Optimizing Ruby’s Memory Layout: Variable Width Allocation - Shopify

A Single-Size Slab and Its Costs

CRuby's garbage collector has long managed memory in fixed 40-byte slots called RVALUE. This uniform size keeps the collector simple: it only tracks chunks of one size and allocates them in 16 KB heap pages, each holding 409 slots. But that simplicity carries real penalties for data locality, allocation overhead, memory efficiency, and the ability to experiment with alternative collector designs.

Most Ruby objects—with exceptions only for immediates such as nil, small integers, and floats—live in these slots. Anything that doesn't fit must be allocated externally via malloc. Different types have wildly different needs: arrays can grow or shrink dynamically, while classes stay a static size. Forcing everything into one slot size means some objects waste space whiles others need external allocations, driving up cache misses and malloc overhead.

The cost of going to main memory is steep. On a typical x86 system, fetching data pulls in a 64-byte cache line (128 bytes on Apple silicon); a 40-byte slot underutilizes that line, and external allocations make things worse by scattering related data. Systems also carry multiple cache tiers with sharply varying latency and bandwidth. Avoiding main-memory round trips is critical for interpreter performance.

External allocation brings another burden: each malloc call costs CPU time, which adds up during boot and object churn. And because most Ruby object data lives outside the RVALUE slot, alternative garbage collector implementations in Ruby have few opportunities to optimize; languages like Java allow swapping collectors by use case, something Ruby's layout does not encourage.

Variable Width Allocation: Size Pools

Shopify's Variable Width Allocation (VWA) project changes this. Authored with other CRuby contributors, VWA introduces APIs for allocating dynamic-sized objects and supports variable-width slots inside the garbage collector. The key new structure is the "size pool": a collection of same-sized slots. Each pool preserves fixed sizes, so fragmentation and allocation speed are unaffected.

Five size pools exist, with slot sizes as powers-of-two multiples of RVALUE: 40, 80, 160, 320, and 640 bytes. These sizes guarantee about 75 percent memory utilization on average, confirmed by heap dumps. Powers of two also prevent frequent resizing because objects can expand or shrink within a pool without needing a change. Multiples of RVALUE size ensure a single classic slot fits without wastage in pools for types not yet converted.

An alternative design—making non-RVALUE pools true powers of two like 64 and 128 bytes, with a dedicated RVALUE-sized pool—offered minor efficiency gains but at significant implementation complexity, so it was dropped.

Larger Heap Pages

To support larger slots effectively, the heap page size was raised from 16 KB to 64 KB. With bigger slots—especially the 640-byte ones—the old page size yielded only about 25 slots per page, forcing frequent page allocations and inflating per-page metadata overhead. Larger pages reduce allocation frequency and, when the system page is also 64 KB (e.g., on PowerPC), enable features like compaction to work on those platforms. Decoupling page size from the collector's operation was a prerequisite to avoid performance regressions during the change.

Resizing and Compaction

Objects can expand past their slot width or shrink until memory is wasted. VWA handles both indirectly: when an object must grow, it temporarily falls back to malloc storage. Compaction—a garbage collection phase that moves live objects together—has been modified to also migrate each object into its most fitting size pool. After a compaction cycle, data becomes adjacent again, restoring data locality without abandoning VWA's benefits.

Types Now on VWA

Four types currently participate in Variable Width Allocation, chosen for specific trade-offs.

Classes

Classes were the first type integrated because their size is fixed and known at allocation time, letting the feature be tested without dynamic resizing logic. The rb_classext_struct—which stores instance variables, superclass references, and constants—formerly required a separate malloc call due to its size. Now it's stored adjacent to the class object itself, eliminating that allocation entirely.

Strings

Strings came next, chosen because they're ubiquitous and already supported "embedded" storage—contents stored directly in the slot. Previously embedding was capped at 23 bytes because the slot was a fixed 40 bytes. With VWA, strings up to 615 bytes embed directly in their slot; larger ones still use external malloc storage. Microbenchmarks showed speedups of up to 35 percent from the change.

Arrays

Arrays already had an embedded form, but it held only up to three elements. With variable-width slots, arrays now embed up to 78 elements directly in their slot; larger arrays still defer to the system allocator.

Objects from User-Defined Classes

The final type added is ordinary objects created from user code—distinct from core types like arrays or classes. These objects store their instance variable list; in the past, embedded objects were limited to three instance variables. After VWA, objects can embed up to 78 instance variables. Beyond that, content moves to external storage. Microbenchmarks measured about a 20 percent improvement when reading or writing those instance variables.

Swappable Garbage Collectors

Variable Width Allocation is also the vehicle for bringing the Memory Management Toolkit (MMTk) into Ruby. MMTk, a research project led by Professor Steve Blackburn at the Australian National University, provides a standard set of memory management APIs. This means different garbage collection algorithms can be tried without touching the Ruby source code itself.

Different collectors have different trade-offs. CRuby’s current mark-and-sweep collector favors lower memory usage, whereas a semispace copying collector will trade twice the memory for better throughput. With MMTk in place, users could eventually switch collectors based on the demands of their workload. Shopify is partially sponsoring this research.

Measurements on Real Workloads

Across the board, the benchmarks show improvements. The gains are most pronounced in workloads that hammer data structures with reads and writes — parsing YAML with psych-load and generating PDFs with hexapdf stand out. railsbench posts a 2.1 percent gain. The complete results, along with the benchmark configuration, are in the appendix.

Querying the New Size Pools

A new API, GC.stat_heap, exposes statistics about the size pools. It is intended for diagnosing memory behavior and for spotting a size pool that is causing trouble in a particular program. Because it reports internal garbage collector state, the shape of the returned data is not guaranteed to be stable. Consult the method documentation for current details.

The Escape Hatch

If performance or compatibility problems arise — especially with native gems — Variable Width Allocation can be turned off during the build with CPPFLAGS='-DUSE_RVARGC=0' at the configure step. Bug reports are also welcome at bugs.ruby-lang.org. This flag is temporary: it exists only for Ruby 3.2 and will be removed in later releases.

Benchmark Configuration and Raw Data

All benchmarks ran on Ubuntu 22.04.1 with an AMD Ryzen 5 3600X and 32 GB of RAM, using a development build of Ruby 3.2.0 at commit 80e56d1. The yjit-bench harness was used at commit 0cf6940, with YJIT disabled to measure interpreter performance. Each figure below is milliseconds per single iteration, so lower is better.