YJIT: A New JIT Compiler for CRuby
Interpreted, dynamically-typed languages like Ruby were born in an era when CPU clock speeds doubled every 18 months and performance could be left to hardware improvements. That era is over. With silicon fabrication hitting physical limits, single-core performance gains have stalled, and energy efficiency has become a real constraint. For a company like Shopify, which runs massive server infrastructure on Ruby on Rails, that means performance work has to happen in software.
That's the context for YJIT, a new just-in-time (JIT) compiler being built inside CRuby by a team of engineers from Shopify and GitHub. CRuby already has a JIT compiler, MJIT, which has been in development for three years. MJIT delivers speedups on small benchmarks but has been less successful on real-world applications such as Ruby on Rails. YJIT takes a data-driven approach, focusing on the performance hotspots of larger applications like Rails and Shopify's main monolith, Shopify Core.
How YJIT Works
YJIT is a project to gradually build a JIT compiler inside CRuby, such that more and more code is executed by the JIT, eventually replacing the interpreter for most execution. The compiler is based on Basic Block Versioning (BBV), a JIT architecture developed during the PhD of Maxime Chevalier-Boisvert, who now leads the project at Shopify.
Building YJIT inside CRuby means the compiler must be written in C and must work with design decisions in the CRuby codebase that weren't made with a high-performance JIT in mind. The trade-off is compatibility: YJIT maintains almost 100% compatibility with existing Ruby code and packages. The project passes the CRuby test suite — about 30,000 tests — as well as all tests in the CI for Shopify Core, a codebase with over three million lines of code and over 500 Ruby gem dependencies, and all tests in GitHub's backend CI. There is also a working deployment to a small percentage of Shopify's production servers.
Results So Far
The project is about one year in, and results have improved significantly since an earlier talk at the MoreVMs 2021 workshop. According to the team's benchmark suite, YJIT achieves speedups over the CRuby interpreter of 20% on railsbench, 39% on liquid template rendering, and 37% on activerecord. Warm-up is fast: YJIT reaches near-peak performance after a single iteration of any benchmark, and performs at least as well as the interpreter on every benchmark, even on the first iteration.
Maxime Chevalier-Boisvert and colleagues believe the BBV architecture offers key advantages for compiling dynamically-typed code. Having end-to-end control over the full code generation pipeline allows YJIT to go beyond what's possible with MJIT's GCC-based architecture. In particular, YJIT can quickly specialize code based on type information and patch code at run time based on observed program behavior. Compilation speed and warm-up time are also difficult to match.
Coming to Ruby 3.1
The Ruby core developers have invited the YJIT team to merge the compiler into Ruby 3.1. That means, in a few months, every Ruby developer will be able to try YJIT by passing a command-line option to the Ruby binary.
The work is not done, however. Currently, only about 79% of instructions in railsbench are executed by YJIT; the rest run in the interpreter. There's a clear path forward, and the team believes YJIT can deliver much better performance than it does now.
In the process of building YJIT, the team has identified several elements of CRuby's architecture that can be improved to unlock higher performance. These improvements would help MJIT and the interpreter as well, so they may be upstreamed separately. Potential improvements include:
- Moving CRuby to an object model based on object shapes.
- Changing the CRuby type tagging scheme to reduce the cost of type checks.
- Implementing a more fine-grained constant caching mechanism.
- A faster, more lightweight calling convention.
- Rewriting C runtime methods in Ruby so that JIT compilers can inline through them.
Matz stated at Euruko 2021 that Ruby would remain conservative with language additions in the near future. The YJIT team believes this is wise, as rapid language changes make it difficult for JIT implementations to get off the ground and stay current. Focusing on internal changes that make the language more robust and deliver competitive performance makes more sense.
YJIT is available on GitHub under the same open source license as CRuby. The team requests that bugs be reported as issues with simple reproductions. More details on trying YJIT and the performance tracking system behind speed.yjit.org are coming in two additional blog posts.
