The JIT Advantage: Optimizing With Assumptions
A common claim about Just-In-Time (JIT) compilers is that they can theoretically outperform ahead-of-time (AOT) compilers because they see how code behaves at runtime. This often leads to hand-wavy explanations about runtime optimizations that static compilers can't perform. The reality is more concrete, especially when you look at a language like Ruby, where the dynamic nature of the language creates unique optimization challenges that JIT compilation is well-suited to address.
Interpreters, Compilers, and the Middle Ground
Interpreters execute human-written source code directly, without a separate compilation step. They see the code for the first time when it runs. AOT compilers, by contrast, transform source code into native machine code ahead of time, allowing the developer to pay the optimization cost once so that end users get a fast binary. A JIT compiler sits in between: it waits until runtime to translate the hot parts of a program into native code, and it does so every time the program runs.
This runtime compilation may sound inefficient compared to AOT. But for highly dynamic languages, compiling ahead of time is often not practical. An AOT compiler must generate code that is correct for every possible program state, and in a language where almost nothing is fixed, that often means the generated code is nearly as slow as interpretation.
Why Ruby Is Hard to Compile
Ruby allows programmers to redefine core operations like +, ==, or iteration at any point, even inside a loop. The interpreter must check every time an addition happens whether the meaning of + has changed. This flexibility is a feature, but it's one that makes static compilation extremely difficult.
An AOT compiler has no choice but to assume the worst: that any operation could be redefined at any moment. It has to generate conservative code that repeatedly checks for changes. In this environment, a traditional compiler can't deliver meaningful speedups. The interpreter, which operates with the same constant uncertainty, spends a significant amount of time performing those checks.
De-optimization: The JIT Cheat Code
JIT compilers get around this constraint by making a different kind of trade-off. Because the compiler is present at runtime, it can generate code based on assumptions that are almost always true—for example, that + hasn't been redefined. This lets it produce fast native code that skips the redundant checks a fully conservative compiler must perform.
What happens when an assumption is wrong? If a program actually redefines +, the JIT's compiled code is no longer correct. At that point, it throws the code away and falls back to interpretation. The next time the method runs, the JIT can compile a new version based on the updated definitions. This process is called de-optimization. It's technically difficult to switch between code versions mid-execution, but it's a solved enough problem that JIT compilers rely on it as a core strategy.
Ahead-of-time compilers can't do this. They can only ship code that must remain valid, often resulting in conservative binaries that check for variability at runtime. Interpreters are similarly constrained, as they must always respond to potential changes. If an interpreter were clever enough to pre-build optimized machine code for current assumptions and discard it as those assumptions change, it would essentially be a JIT compiler.
What This Looks Like in Practice
YJIT, a JIT compiler integrated into CRuby, is built around this technique. It tracks what it has optimized and what conditions would invalidate that code. In Ruby's case, "everything can change" is a real constraint, but YJIT's approach treats such changes as the exceptional case.
YJIT is available by enabling the --yjit flag in recent Ruby versions. The latest development builds, such as ruby-build 3.2.0-dev, include more of this work. Note that YJIT had known bugs in Ruby 3.1 and was not recommended for production use then. Improvements made it closer to production-ready for some workloads in Ruby 3.2, released in December 2022.
The takeaway is that JIT compilers don't just translate code at a better time. They generate code based on runtime assumptions that AOT compilers can't safely make. When those assumptions break, they discard the invalid code. This combination—specialized optimization plus the ability to correct course—is what allows a JIT to outperform a traditional compiler in dynamic languages like Ruby.
