The JIT (Just-In-Time) compiler in Java converts bytecode into native machine code while your program is running, rather than before it starts. This is the mechanism that closes the performance gap between Java and languages like C++ that compile directly to machine code. Instead of interpreting bytecode instructions one at a time, the JIT compiler identifies frequently executed code and compiles it into optimized instructions specific to your hardware.
Why Java Needs a JIT Compiler
When you compile a Java source file, the output isn’t native machine code. It’s bytecode, a platform-independent set of instructions that any Java Virtual Machine (JVM) can read. This is what makes Java portable: the same compiled file runs on Windows, Linux, or macOS without changes. But interpreting bytecode instruction by instruction is slow compared to running native code directly on the processor.
Early Java implementations relied entirely on interpretation, and the performance penalty was significant. Compiling everything to native code ahead of time would solve the speed problem but destroy portability and the security guarantees the JVM provides. The JIT compiler splits the difference: it compiles at runtime, preserving portability while producing fast native code tailored to the exact machine running the program.
How the JIT Compilation Process Works
Your Java program doesn’t get JIT-compiled all at once. When execution begins, the JVM interprets bytecode normally and starts collecting data about how the program behaves. It tracks two key metrics: how many times each method gets called and how many times loops within those methods iterate. When either count crosses a threshold, the JVM flags that code as “hot” and hands it to the JIT compiler.
The compiler then walks through the bytecode of that method, block by block. For operations that must execute in a fixed order per the Java specification, it emits native machine instructions immediately. For everything else, it delays generating code, recording the necessary information and waiting until it reaches the instruction that actually uses the result. This lets the compiler reorder and combine operations for better performance.
Once compilation finishes, future calls to that method execute the native version instead of being interpreted. The cost of compilation has to be paid back through faster subsequent executions, which is why the JVM only bothers compiling code that runs frequently. In a typical program run, less than half of all methods ever execute, so compiling everything upfront would waste time on code that never runs.
The Five Levels of Tiered Compilation
Modern JVMs (Java 8 and later) use a system called tiered compilation that combines two separate JIT compilers, known as C1 and C2, with the interpreter. Together they produce five optimization levels:
- Level 0 (Interpreter): The simplest execution mode. Bytecode is interpreted directly while the JVM begins collecting profile data.
- Level 1 (C1, no profiling): A quick compilation with basic optimizations. Used for simple methods that don’t benefit from further analysis.
- Level 2 (C1, basic profiling): Compiled code that also collects limited runtime data for future optimization decisions.
- Level 3 (C1, full profiling): Compiled code with comprehensive instrumentation. This is the most common stepping stone before heavy optimization.
- Level 4 (C2, full optimization): The final tier. The C2 compiler applies aggressive optimizations using all the profile data collected at earlier levels. This produces the fastest code but is the most CPU-intensive compilation stage.
A method typically starts at level 0, gets compiled to level 3 once it’s called enough times, and eventually reaches level 4 if it remains hot. The JVM’s heuristics decide when to promote code, when to recompile it at a different level, and which specific optimizations to apply. This progression means your application gets faster the longer it runs, as more hot paths reach the highest optimization tier.
Compilation Thresholds
The JVM doesn’t compile a method after just a few calls. By default, the server JVM requires 10,000 interpreted invocations of a method before the JIT compiler kicks in. The older client JVM used a lower threshold of 1,500 invocations. These defaults exist because the JVM needs enough execution data to make good optimization decisions, and because compiling rarely-called methods wastes resources.
You can adjust the threshold with the -XX:CompileThreshold flag, though the defaults work well for most applications. With tiered compilation enabled (the default since Java 8), the system is more nuanced: methods progress through the levels based on a combination of invocation counts, loop iteration counts, and the profile data collected at each stage.
Where Compiled Code Lives: The Code Cache
When the JIT compiler produces native code, it needs somewhere to store it. The JVM maintains a dedicated memory region called the code cache for this purpose. It’s divided into three segments based on code type: one for internal JVM code like the bytecode interpreter (which stays permanently), one for lightly optimized code from C1 that has a short lifespan, and one for fully optimized C2 code that may stick around for the lifetime of the application.
The size of the code cache matters for performance. When compiled code takes up a lot of space, the processor’s instruction cache and branch prediction hardware have to work harder. Testing on server-grade hardware has shown that inflating the code’s spatial footprint, even without changing what the code actually does, can reduce throughput by 4 to 6 percent and increase response time latency by 1 to 3 percent. For large applications, tuning code cache size and monitoring its usage can prevent these front-end CPU bottlenecks.
What the JIT Compiler Actually Optimizes
The JIT compiler does more than just translate bytecode to machine code. Because it has access to runtime profile data, it can make optimization decisions that a traditional compiler never could. Some of the key techniques include:
- Method inlining: Replacing a method call with the method’s actual code, eliminating the overhead of the call itself. The JIT compiler can inline methods that a static compiler wouldn’t risk inlining, because it knows from profiling data which concrete types are actually being used.
- Dead code elimination: Removing code paths that profiling shows are never taken in practice, even if they’re reachable in theory.
- Loop optimization: Unrolling loops, hoisting invariant calculations out of loops, and other transformations based on observed iteration patterns.
- Speculative optimization: Making assumptions based on observed behavior (like “this virtual method call always resolves to the same implementation”) and generating faster code for the common case, with a fallback if the assumption is violated.
This ability to optimize based on actual runtime behavior is the JIT compiler’s biggest advantage. It can customize native code for the specific input and execution patterns of each program run, something that’s impossible when compilation happens before the program starts.
JIT Compilation vs. Ahead-of-Time Compilation
Java also supports ahead-of-time (AOT) compilation, where bytecode is converted to native code before the program runs. Tools like GraalVM’s native image do this. The trade-offs are straightforward.
AOT compilation eliminates the startup penalty entirely. There’s no interpreter phase, no profiling overhead, and no compilation happening while your application is trying to serve requests. This makes it attractive for short-lived programs, command-line tools, and serverless functions where the application may not run long enough for JIT compilation to pay off. AOT-compiled programs also tend to use less memory because they don’t need the JIT compiler infrastructure, code cache, or profiling machinery at runtime.
JIT compilation wins on peak performance for long-running applications. Because the JIT compiler sees how the program actually behaves with real data, it can apply profile-guided optimizations that produce better native code than an AOT compiler working with static analysis alone. A web server or database that runs for hours or days will typically reach higher throughput with JIT compilation than with AOT, because the C2 compiler has had time to optimize every hot path based on real traffic patterns.
The choice comes down to what matters more for your use case: fast startup and predictable performance from the first millisecond, or maximum throughput after a warm-up period.