To Thread or Not to Thread: An In-Depth Look at Ruby’s Execution Models - Shopify

Ruby's Fork-and-Thread Tradeoff

Threaded Ruby servers are the default deployment target in 2022, per the Ruby on Rails community survey, and Sidekiq dominates the job processor space. The core reason is throughput per unit of memory: threads widen request handling capacity without adding proportional memory overhead. But that simplicity obscures a more nuanced engineering decision between processes and threads, one that hinges on memory accounting, the global VM lock (GVL), and the clean-interruption guarantees a process model gives you.

Modeling Application Memory

Split a web application's memory into two conceptual buckets:

Static memory is the fixed cost of running the app: the Ruby VM, generated bytecode, and long-lived objects such as I18n data. It stays flat whether the server handles one request or a hundred concurrently. Request processing memory is what a single request needs: query results, rendered templates, allocations the GC will reclaim. In a simplified accounting:

Given 512MiB of available memory, an app needing 200MiB static and 150MiB per concurrent request fits comfortably as one process with two threads (500MiB) but blows the budget as two single-threaded processes (700MiB).

That model assumes forked processes duplicate everything. Modern operating systems are far more efficient, thanks to copy on write.

Copy on Write Changes the Math

Forking on contemporary UNIX systems doesn't clone the entire address space. The child's page tables point at the parent's pages; actual copying only happens when either side writes. In the ideal case, forked processes cost almost nothing extra, and the formula collapses to:

static_memory + (processes * threads * processing_memory)

That would erase the memory advantage of threads entirely. Reality lands somewhere in between, since applications do dirty some shared pages. The question is how much, and the answer depends on measuring memory correctly.

Avoid RSS (resident set size) for this. When a forked process touches a shared page, RSS counts it for each process independently, double-reporting the same physical memory. PSS (proportional set size) divides shared regions among the procs that use them, so the sum of PSS across processes is the real system footprint.

Reading Linux smaps

Linux exposes per-process granularity at /proc/$PID/smaps_rollup, with fields for clean versus dirty and shared versus private memory. For example, a Unicorn worker reporting 771,912 kB of addressable memory may only truly own 437,928 kB (56.7%). The rest comes from the parent process. Clean memory — the Ruby binary, native shared libraries — was never written since fork. Dirty memory was written by at least one process; it's only shareable if that write happened before the fork. Comparing the worker's shared memory (18,288 kB plus 315,648 kB, or ~334 MB) against the parent's 508,544 kB RSS yields a CoW sharing efficiency of roughly 66%.

That's the practical cost picture: close to 750 MiB per worker by RSS, more honestly around 427 MiB, versus 257 MiB for an additional thread. Still a process premium, and worth closing further.

Improving CoW Efficiency

Any work done during boot is likely shareable; anything lazily initialized after the fork is not. Guard against the common anti-pattern of memoized class variables, which both delay evaluation and break page sharing. Prefer constants, but when that fails use Rails' eager_load_namespaces config option to force the load during pre-boot. Puma, Unicorn, and Sidekiq Enterprise all support preload_app for exactly this purpose. To spot the stragglers, dump the heap with the Shopify Ruby heap-profiler after fork and again a few requests later; the delta reveals where private allocation is happening.

Why Processes Still Win Some Arguments

Even with accurate memory accounting, process-based servers like Unicorn or Resque justify themselves on two less obvious grounds.

Threads Are Uninterruptible

Whether from malicious abuse or an unexpected data spike, a request sometimes runs wildly long. A process-based server kills the worker and spawns a fresh one. Threads share mutable state — mutexes, database connections — so killing a single thread leaves resources in an unrecoverable state, pushing errors onto sibling threads. There is no clean escape hatch.

The GVL's Hidden Latency Tax

Ruby's GVL means only one thread executes Ruby code at a time, and GC pauses block everyone. Two threads in one process serialize CPU work and add latency when they collide. An N-threaded process is only fully saturated when it spends under 1/N of its time on I/O, so 50% I/O saturates two threads; 75% saturates four. And that assumes coordination — an application with 75% I/O will still regularly queue for the GVL.

Common wisdom labels Ruby apps I/O-heavy, but that's wrong for most web request cycles, where slow operations are deliberately deferred to background jobs. The I/O that remains — database and cache queries — is fast. Job processing pulls in more true I/O, so heavier thread counts in Sidekiq-style workloads are a more natural fit. Web servers still benefit, in throughput dollars, at some latency cost you can't directly measure.

Instrumenting that cost isn't possible today; Ruby exposes no GVL API. Proxy experiments — toggling thread counts and watching latency — aren't enough. I've filed a Ruby 3.2 feature request with a proof-of-concept implementation for a low-level GVL instrumentation API. If accepted, I plan to build a gem exposing simple metrics, and hope APM vendors pick it up. Until then, process-and-thread decisions remain an exercise in indirect measurement.

Why Ractors and Fibers Aren’t an Easy Win

Recent experimentation in the Ruby community has focused on two alternative concurrency constructs: Ractors and Fibers. Neither is poised to replace threads in typical web applications.

Ractors are unique in that they allow true parallel Ruby execution: instead of a single Global VM Lock (GVL), each Ractor has its own lock. In theory this is a game-changer. In practice, however, Ractors forbid sharing any global mutable state. You cannot share a database connection pool or even a logger across Ractors. This architectural constraint would require substantial refactoring of most libraries, and the resulting APIs would likely be less usable. While it’s possible this changes in the future, Ractors are unlikely to become the unit of execution for sizable web applications anytime soon.

Fibers are essentially lighter-weight, cooperatively scheduled threads. All the earlier discussion about threads and the GVL applies equally to them. Fibers suit I/O-intensive applications that mostly shuffle byte streams and spend little time executing code. But if your application doesn’t benefit from more than a handful of threads, fibers won’t help either.

YJIT Could Shift the Trade-offs

Just-in-time compilation may eventually change the calculus for threaded servers. YJIT speeds up code execution at the cost of unshareable memory, which degrades copy-on-write (CoW) performance. However, as code runs faster, an application becomes proportionally more I/O-intensive, which is precisely the scenario where threads shine.

Current YJIT speedups are modest. If future versions achieve even a 2x improvement, application owners could reasonably increase their web thread count to compensate for the higher memory cost.

Practical Guidance for Server Selection

There is no universal answer to the process-versus-threads question. The right choice depends on application metrics. That said, several rules of thumb hold up well:

• Always enable application preloading to maximize CoW benefits.
• Unless your application fits on the smallest hosting offering, prefer fewer, larger containers over more, smaller ones. A single 4CPU 2GiB container is more efficient than four 1CPU 512MiB containers.
• If latency matters more than cost, or if spare memory is available, Unicorn offers a reliable request timeout.
• Note that Unicorn needs a reverse proxy to buffer requests and protect against slow client attacks. If that’s an issue, Puma can be configured to run a single thread per worker.
• When using threads, start with just two unless the application demonstrably spends over half its time waiting on I/O. Job processors are the exception: they tend to be far more I/O-bound and less latency-sensitive, so they benefit from higher thread counts.

What’s Next for Ruby’s Execution Ecosystem

Several improvements are under exploration to strengthen both process- and thread-based servers.

The GVL instrumentation API discussed earlier should let application owners make better-informed throughput-versus-latency decisions. It could even support automatic backpressure — dynamically adjusting concurrency when GVL contention exceeds a threshold.

Threaded web servers could also gain a reliable request timeout. When a request exceeds its limit, the server could stop routing traffic to the affected worker, wait for other requests to finish or time out, then kill and refork the worker. Matthew Draper explored this approach a few years ago, and it appears feasible.

Ruby’s CoW performance is another area with headroom. Numerous patches have improved it over the years, but more work is possible. Ruby’s inline caches are suspected of causing most VM bytecode to become unshared once executed. Inspiration may come from the Instagram engineering team’s work on Python’s CoW behavior. They introduced gc.freeze(), which tells the garbage collector that all existing memory regions will become shared. Python then makes smarter allocation decisions, such as avoiding free slots in shared regions in favor of allocating new pages to prevent dirtying shared ones.