6
$\begingroup$

Just-in-time compilation (JIT) is a way of implementing a language where the source code is given as input (like with a typical interpreter), but rather than being directly interpreted, (some or all of) the code is compiled. This is done, for example, in the Python interpreter PyPy, and in most browsers' JavaScript implementations.

What are the advantages and disadvantages to JIT compilation, over typical interpreters, or compilation ahead of time? When would I choose JIT over something else?

$\endgroup$
5
  • 7
    $\begingroup$ This is not the usual definition of just-in-time compilation, which the Wikipedia page you linked to does have. $\endgroup$
    – Michael Homer ♦
    May 22 at 23:54
  • 6
    $\begingroup$ Without reading the wiki article, my understanding is that JIT typically involves compiling individual functions or even branches as-needed, rather than compiling the whole program upfront within the interpreter. $\endgroup$
    – Bbrk24
    May 23 at 4:24
  • $\begingroup$ Note most browsers require a function to be run around ~1000 times before it will be considered for JIT, until then it's interpreted normally $\endgroup$
    – mousetail
    May 23 at 5:58
  • 1
    $\begingroup$ Java uses a JIT compiler without having the source code available at runtime. You could argue that Java bytecode is the source code that the JIT compiles, but "source code" usually means the form in which the software is written or edited. $\endgroup$
    – kaya3
    May 23 at 9:38
  • $\begingroup$ Hi, are the existing answers lacking? I don't want to write a new answer if any of them are acceptable (none are yet accepted); yet, my answer is going to be short: "Sometimes JIT is faster, sometimes slower; profile for your use case, and read VM Startup Blows Hot & Cold. There are no known hard rules." $\endgroup$
    – Corbin
    Sep 21 at 2:11

5 Answers 5

Reset to default
6
$\begingroup$

In certain cases having a JIT-compiler can be beneficial even in compiled code. Think of an image filter where you parametrize your kernel function on some runtime variables. If you compile this function just in time you have a lot more information available and thus more optimization opportunities than if you had compiled it ahead of time. For a function that is called millions of times in a hot loop this can really make a difference in terms of performance.

There is a library that aims to bring jit-compilation to C++ for this purpose.

$\endgroup$
3
  • $\begingroup$ Do you mean that a specific version of the function/kernel is compiled for a specific runtime value of some parameter(s)? So it'd eventually need to be recompiled for other values? $\endgroup$
    – Pablo H
    Aug 23 at 19:37
  • $\begingroup$ @PabloH Yes, exactly $\endgroup$
    – chrysante
    Aug 29 at 17:26
  • $\begingroup$ But even if runtime state is not taken into account, JITted code can be compiled for the exact platform (processor, etc.) and so can still be faster than a compiled program which has to run on a wide variety of platforms. $\endgroup$
    – gidds
    Oct 18 at 14:14
5
$\begingroup$

There is really only one advantage: Speed. Once compiled, JIT-ed code should run just as fast as native code. While there are many disadvantages the speed bonus is often worth it.

You can also incrementally apply optimizations for functions that get called a lot to reduce startup time. This can be done with real-world data about in what way the function gets called which could potentially allow more efficient optimization than when done ahead of time.

Some disadvantages include:

  • Requiring taming of a dynamic type system. If everything can be any type the machine code will need to be littered with branches and not be very fast. Thus JIT-ed code will make assumptions about the types of objects based on watching them over time. However, if the types ever do not match the performance benefit is lost.
  • Security vulnerabilities. Despite being made by major companies with teams of highly skilled professionals all browsers have had many security issues related to their JavaScript JIT compiler. Mostly related to making assumptions when optimizing but not checking them properly.
  • More memory overhead. You need to keep both the original code and JITed versions of many potential type combinations of each function in memory as well as needing both primitive low level and high level versions of objects, as well as constantly needing to keep them in sync.
  • Some types of events can basically invalidate some or all JITed code, requiring a slow recompile. This includes things like:
    • Changing the type of a global variable
    • Monkey patching some type
    • Self-modification
  • Debugging can be harder. Typically debugging is done by just disabling the JIT portion but this would exclude bugs introduced by the JIT compilation itself.

Generally you only JIT specific functions if they are run a lot with the same argument types and the same context, so if this never happens you have no downsides except the book-kepping to keep track of how often each function is run. You can JIT in a seperate thread so as not to slow down the program while compiling and just interpret normally until then.

If your language is less dynamic many of the downsides might not apply and implementation might be more straightforward.

$\endgroup$
6
  • $\begingroup$ "Some types of events can basically invalidate all JITed code" — I don't immediately see why the events you list should invalidate all JITed code and not a small part of it? $\endgroup$
    – Alex Chichigin
    May 24 at 6:50
  • $\begingroup$ @AlexChichigin It can be really hard to prove in highly dynamic languages if a function depends on some global variable $\endgroup$
    – mousetail
    May 24 at 6:52
  • $\begingroup$ In general (contrived) case — yes, but wast majority of cases are not general and don't use global variables at all, thus most of the code stays the same, and don't get invalidated. As far as I know... $\endgroup$
    – Alex Chichigin
    May 24 at 7:10
  • $\begingroup$ The issue is, unless you can 100% prove that the global variable won't effect a function you need to regenerate it. Note the "global variable" can be something like null, true, or object which are also variables in Javascript $\endgroup$
    – mousetail
    May 24 at 9:36
  • $\begingroup$ Another advantage is the ability to handle source languages where a program would specify an infinite number of specialized functions, but only execute a finite subset that is based upon the data a program receives. This kind of pattern can exists with generics in .NET languages like C#. Ahead-of-time generation of machine code would be impossible because nothing in the universe could know exactly what functions would be invoked until after execution had started. $\endgroup$
    – supercat
    Sep 22 at 16:05
4
$\begingroup$

JIT is a way of implementing a language where the source code is given as input

Not necessarily at all. As we know both JVM and .NET JIT-compile byte-code and not the source code. The same is true for most WebAssembly runtimes.

As already mentioned, the main advantage of JIT over both interpretation and AOT is speed. For interpreters it's more evident as natively compiled code runs faster — almost always, there are exceptions though. Still JIT can outperform AOT compiled and optimized code due to availability of runtime information. That most pronounced for dynamically typed languages exactly because type (and object shape) information only fully emerges at runtime, but for statically typed languages too — as was also already mentioned, JIT can specialize (inline) and optimize algorithms for particular runtime parameters. Julia language does this with great success for many numerical algorithms.

The main user-facing disadvantage is start-up time. Again this is vividly exemplified by Julia language: before recent version 1.9 some libraries could take up to a couple of minutes to fully initialize and JIT, which was mitigated by employing (more) of AOT compilation at the library install time.

But most of other JIT-compilers including HotSpot JVM, .NET Core, V8, SpiderMonkey and others mitigate start-up latency problem via tiered compilation: they first run faster, less optimizing baseline JIT, and only after that a slower, better optimizing one.

$\endgroup$
4
  • 1
    $\begingroup$ Another performance advantage of a JIT language like Java is that programs can benefit from improvements in a JIT compiler which take place after a program is published, even if such improvements exploit hardware features that didn't exist when the program was written. $\endgroup$
    – supercat
    Sep 22 at 16:07
  • 1
    $\begingroup$ Nerd remark: a language can't be JIT or not, it's an implementation that can use JIT or not. :) Also, AOT-compiled programs in source code form also benefit from compiler improvements after they were written. And one can argue a bytecode is closer to source code than machine code, so the lines are blurry... $\endgroup$
    – Alex Chichigin
    Sep 22 at 17:41
  • $\begingroup$ Some languages, including Java, have historically been designed with the intention of facilitating JIT-based implementations. Some go further and make it impossible to generate machine code for some constructs until after a program has started execution. Compiling from source may be possible after publication, but in many the time required to compile a source-code program may exceed the time of a typical execution. JIT-based languages are designed to make the JIT process fast. $\endgroup$
    – supercat
    Sep 22 at 17:48
  • $\begingroup$ Historical note: Java wasn't designed for JIT, it was only added in version 1.2. .NET was designed for JIT from the start, yep. $\endgroup$
    – Alex Chichigin
    Sep 22 at 17:53
4
$\begingroup$

It's complicated.

Security

Compiling to native code is much less forgiving -- security-wise -- than interpreting. A stray read or write can very easily lead to an exploit.

As a result, compiling "foreign" code to native code on the fly -- whether via AOT or JIT compilation -- is fraught with peril. All mainstream JS engines have had CVEs filed against them, for example.

JIT (and AOT) compilers are therefore at a clear security disadvantage compared to interpreters when it comes to running "foreign" code.

Size & Complexity

In general, a pure interpreter is (much) smaller than a JIT compiler, and thus much less complex. This ties in with the above security issue: the simpler, the easier to secure.

Portability

In general, a pure interpreter is more portable than a JIT compiler. A JIT compiler must be specialized for everything target platform (both architecture and OS), whereas a pure interpreter can rely on existing toolchains to be compiled for the right platform.

Conversely, a statically compiled binary is stand-alone and can be delivered on its own to any user, whereas for interpreted or JIT compiled language, the user typically must have both the "program" (in some form) and an interpreter or JIT compiler of the right version. Bundlers do exist, but the resulting applications are even bigger than statically compiled applications.

Dynamic Languages

So called dynamic languages -- ie, languages where the type of a value is not known at compile-time -- are a hard target for AOT compilers, so that in general there are only interpreters or JIT for them.

In such a situation, there's typically a start-up time vs run-time trade-off:

  • An interpreter starts up near immediately but may run slower.
  • A JIT compiler starts up slower -- compiling everything it comes across -- but will run faster in the long term.

The above trade-off can be alleviated (somewhat) by using tiered compilation, which is often used in JS engines, or the C# or Java runtimes. Tiered compilation is the idea of starting with the fastest start-up method, and over time to compile more and more optimized JIT code.

Static Languages

While in theory a JIT compiler should be able to outperform anything, in practice it rarely plays out and a JIT compiler typically ends up producing a program which runs slower1 than one produced by an AOT compiler, due to a lower optimization budget.

There a specific exceptions: some workloads benefit from a run-time specialized inner loop.

The difference between theory and practice is perhaps not too surprising: in order for a JIT compiled program to outperform an AOT compiled program, it must take advantage of run-time information. This, in turn, means that a JIT runtime must not only JIT compile a program, but also instrument it to gather the necessary information, and guard it, in case the information it optimized for falls through.

This has 3 downsides:

  1. Information gathering takes up run-time, run-time that could be put to use to actually run the program, which the JIT compiler must then make up for, somehow.
  2. Hot-patching -- used both for de-optimizing and splicing in a more optimized function -- requires to leave some "scaffolding" in place. Scaffolding which takes up run-time, and which once again the JIT compiler has to make up for, somehow.
  3. Guards -- used for de-optimizing -- which are extra run-time checks, which the JIT compiler has to make up for, somehow.

And of course, the very run-time used up to JIT compile also has to be made up for.

All in all, this means that JIT compiler start with a penalty2, compared to AOT compilers, and only if they uncover critical run-time information unlocking worthy optimizations will they able to produce a program which will outperform an AOT program.

And of course, very optimized AOT programs also use run-time information gathering for Profile Guided Optimizations, except ahead of time3 so they don't pay for it at run-time.

1 JIT compiled pre-optimized WASM produced by AOT compilers for C, C++, Rust, or Zig, for example, is routinely 2x slower than the natively compiled program. There's indubitably some overhead from the sandboxing applied, but still...

2 A penalty on top of the hand tied behind their back, that is, as they typically have a much stricter "optimization budget" than AOT compilers, since they're doing the optimization work while the program is running...

3 PGO ahead of time may suffer from artifacts effects, yes. But then again, the latest C# JIT will splice in a non-instrumented optimized version, so that should the workload change after optimization, it also uses the wrong profile. Why? Because keeping the instrumentation in would negatively affect performance...

$\endgroup$
9
  • $\begingroup$ I see no reason a JIT couldn't employ security-in-depth practices in much the same manner as interpreters. A JIT may reap more benefit from identifying and eliminating seemingly-redundant operations, and thus many JIT compilers are more aggressive than interpreters in that regard, but a JIT which e.g. produces machine code that bounds-checks every array access could easily be less vulnerable to exploits than an interpreter which tries to identify and exploit places where checks can be omitted. $\endgroup$
    – supercat
    Sep 22 at 16:11
  • 2
    $\begingroup$ JIT compilers introduce a new security problem, because JITs are self-modifying code. For security reasons, most modern OS use a $W \oplus X$ (write xor execute) discipline for memory locations. But a JIT generates machine code, meaning it needs to write (W) and then later jump to the written code (X). So the OS has to make memory that may have been compromised executable. $\endgroup$
    – Martin Berger
    Sep 22 at 18:38
  • $\begingroup$ @supercat: A JIT is inherently more difficult. You can write an interpreter in a safe language, guaranteeing that no matter how much you mess up, you're still bound by the rules of the underlying language (and its runtime). A JIT generates assembly, the ultimate unsafe language. A single mistake in interpreting an untyped register, and all soundness flies out the window. $\endgroup$
    – Matthieu M.
    Sep 23 at 11:15
  • $\begingroup$ @MatthieuM.: If code consists entirely of operational sequences which validate array indices, and maintain the invariant that anything that might hold an object reference and isn't null will always be a valid object reference, operations that would uphold those invariants individually will also uphold them when combined. Things can get dangerous if an implementation skips bounds checks in cases where there doesn't seem to be any sequence of events where computations could yield out-of-range results, but if bounds checks are consistently performed invariants will be upheld. $\endgroup$
    – supercat
    Sep 25 at 15:18
  • $\begingroup$ @MartinBerger: If there is no mechanism via which code can violate memory-layout invariants, or access things in a manner contrary to those invariants, there would be no means via which control could be transferred to any regions of address space not used by the JIT, nor that those regions of address could be written by anything other than the JIT. Code/data segregation makes it harder to exploit opportunities to e.g. overwrite function return addresses, but on platforms that use variable-length instruction sets even ahead-of-time compiled code may be chance contain sequences that will be... $\endgroup$
    – supercat
    Sep 25 at 15:25
1
$\begingroup$

JIT compilation is an attempt to get the "best of both worlds" of compilation and interpretation.

As with an interpreter, you don't have to perform an explicit compilation step before running the program. This is useful if the program will be delivered to diverse environments, as you don't have to compile for each possible environment, or distribute a build script that the end user must run to compile the program. On the other hand, it does require that the target environment provide the execution tool; this is why we most often find it used in browser-based applications -- everyone has a browser.

But as the other answers say, you get the performance benefit of compiled code. There's a small performance hit from the compilation step before a particular function is executed, but this is usually negligible as it's amortized over the entire execution time of the program. Unused parts of the program never get compiled at all.

$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .