New servers, new tradeoff

Cloudflare’s 12th generation fleet, built on AMD EPYC Genoa-X processors, leaned heavily on those chips’ large 3D V-Cache to feed the company’s request handling layer, FL1. But when evaluating the next generation of hardware, Cloudflare hit a tension: the CPUs with the biggest throughput gains cut the per-core cache dramatically, and the legacy FL1 stack couldn’t absorb that change without latency spiking.

The fix did not come from tweaking the hardware. It came from the FL2 transition — Cloudflare’s Rust-based rewrite of its core request handling layer. FL2 was already underway for security and development-velocity reasons, but it turned out to be the key that unlocked the Gen 13 platform. Cloudflare is now announcing the deployment of Gen 13 servers based on AMD EPYC 5th Gen “Turin” processors, running FL2.

What Turin changes under the hood

AMD’s EPYC 9005-series Turin processors bring several advances over the Genoa-X parts used in Gen 12:

  • 2x core count: up to 192 cores versus Gen 12’s 96, with SMT yielding 384 threads
  • Improved IPC: Zen 5 delivers better instructions-per-cycle than Zen 4
  • Better power efficiency: up to 32% fewer watts per core compared to Genoa-X, despite more cores
  • DDR5-6400 support: higher memory bandwidth to keep those cores fed

But the high-density Turin SKUs make a deliberate trade: throughput over per-core cache. Cloudflare’s analysis shows that a 192-core Turin part shares 384MB of L3 cache — just 2MB per core, one-sixth of the allocation on Gen 12’s Genoa-X processors. For a workload dependent on cache locality, that is a major shift.

Measuring the cache pain

Rather than assume the cache reduction would be a problem, Cloudflare measured it during CPU evaluation using AMD uProf. The data was unambiguous:

  • L3 cache miss rates jumped dramatically versus Gen 12’s 3D V-Cache servers
  • Memory fetch latency came to dominate request processing time as data that previously lived in L3 now required trips to DRAM
  • The latency penalty grew with utilization, as cache contention worsened at higher CPU loads

The numbers behind that: an L3 hit completes in roughly 50 cycles, while a miss that forces a DRAM access takes 350+ cycles — an order-of-magnitude difference. Running FL1 on Gen 13, with 6x less cache per core, meant far more trips to memory and a steep latency penalty.

Initial tests with FL1 on Gen 13 confirmed the counter data. The Turin-based evaluation server did deliver a 60% throughput gain versus Gen 12, which was compelling from a TCO standpoint. But that gain came with a more than 50% latency penalty — not acceptable for customer-facing request processing.

Tuning around the bottleneck

Working with AMD, Cloudflare ran a series of targeted experiments on the Turin 9965:

  • Hardware tuning: adjusting hardware prefetchers and Data Fabric (DF) Probe Filters produced only marginal gains
  • Scaling workers: launching more FL1 workers improved throughput but cannibalized resources from other production services
  • CPU pinning and isolation: workload isolation tweaks had limited success

The most useful lever was AMD’s Platform Quality of Service (PQOS), which allows fine-grained regulation of shared resources like cache and memory bandwidth. Turin processors combine one I/O Die with up to 12 Core Complex Dies (CCDs), each sharing an L3 cache across up to 16 cores. Allocating a dedicated L3 share within a single CCD for FL1 gave minimal gains. But dedicating an entire CCD to FL1 — a socket-level approach — produced meaningful throughput improvements while keeping latency acceptable.

FL2 arrives at the right time

Hardware tuning only got Cloudflare so far. Unlocking the Gen 13 architecture fully required a software rewrite that changed how the stack used system resources. That rewrite, FL2, was already in progress: a complete replacement of 15 years of NGINX and LuaJIT code in Rust, built on the Pingora and Oxy frameworks.

The FL2 project was not started to solve Gen 13’s cache problem. Its goals were better security via Rust’s memory safety, faster development velocity through a strict module system, and improved performance with less CPU and memory usage. But FL2’s cleaner architecture, with better memory access patterns and less dynamic allocation, looked like it might not depend on massive L3 caches the way FL1 did. That made the Gen 13 rollout a natural test: could FL2 realize Turin’s throughput gains without the latency hit?

FL2 on Gen 13: the proof

As FL2 rolled out, production metrics from Gen 13 servers bore out the hypothesis. Before any system optimizations, FL2 cut the latency penalty by 70%, letting Cloudflare push Gen 13 to higher CPU utilization while staying within its latency SLAs — something FL1 could not do.

With the cache bottleneck effectively removed, throughput scales nearly linearly with core count. On the high-density AMD Turin 9965, Cloudflare measured a 2x performance gain. Further system tuning is expected to squeeze out more.

Generational gains, from silicon to rack

Hardware qualification is complete, and Gen 13 servers are shipping at scale for Cloudflare’s global rollout. The business-level impact:

  • Up to 2x throughput vs Gen 12, with latency SLAs intact, allowing traffic spikes to be absorbed without slowing customer applications.
  • 50% better performance per watt vs Gen 12, lowering data center expansion costs and reducing carbon footprint per request.
  • 60% higher rack throughput vs Gen 12 at the same rack power budget, making the fleet easy to deploy across Cloudflare’s edge network.

The Gen 13 deployment is a case study in hardware-software co-design. FL1 hit a cache contention wall on Turin, forcing an unacceptable choice between throughput and latency. FL2, with its leaner memory access pattern, removed the dependency on massive L3 caches and let the hardware scale with core count. Gen 13 servers are now ready to serve the millions of requests that cross Cloudflare’s network.