Performance and power gains

Gen 12 systems serve more than twice as many requests per second as Gen 11, a result Cloudflare attributes to close engineering collaboration with AMD on optimization. Higher density per server lowers the cost of building out Cloudflare's infrastructure.

Efficiency moves in the same direction: RPS per watt improved by more than 60% over the prior generation. As the fleet grows, that gain reduces operational expenditure and carbon footprint as a share of fleet size.

Thermal redesign for GPU workloads

Demand for AI capabilities prompted changes to the thermal-mechanical design of the Gen 12 server so it can accommodate more powerful GPUs. This supports the Workers AI objective of running larger large language models and raising throughput for smaller ones.

Hardware trust and an open management controller

Security is built in at the firmware level. Gen 12 servers integrate hardware root of trust (HRoT) capabilities that verify the integrity of boot firmware and board management controller firmware.

The baseboard management and security controller is an OCP DC-SCM module that Cloudflare designed into its systems. It is modular and vendor-agnostic, which allows a single unified openBMC image, faster prototyping and component reuse.

Supply assurance

A robust multi-vendor strategy mitigates supply chain risk, supporting continuity and resiliency in infrastructure deployment.

Gen 12 hardware configuration

Each Gen 12 server pairs a 4th generation AMD EPYC Processor with 384 GB of DDR5 RAM, 16 TB of NVMe storage, a dual-port 25 GbE NIC, and two 800 watt power supply units.

Generation Gen 12 Compute Previous Gen 11 Compute
Form Factor 2U1N - Single socket 1U1N - Single socket
Processor AMD EPYC 9684X Genoa-X 96-Core Processor AMD EPYC 7713 Milan 64-Core Processor
Memory 384GB of DDR5-4800
x12 memory channel
384GB of DDR4-3200
x8 memory channel
Storage x2 E1.S NVMe
Samsung PM9A3 7.68TB / Micron 7450 Pro 7.68TB
x2 M.2 NVMe
2x Samsung PM9A3 x 1.92TB
Network Dual 25 Gbe OCP 3.0
Intel Ethernet Network Adapter E810-XXVDA2 / NVIDIA Mellanox ConnectX-6 Lx
Dual 25 Gbe OCP 2.0
Mellanox ConnectX-4 dual-port 25G
System Management DC-SCM 2.0
ASPEED AST2600 (BMC) + AST1060 (HRoT)
ASPEED AST2500 (BMC)
Power Supply 800W - Titanium Grade 650W - Titanium Grade
BLOG-2116 2

CPU selection and production evaluation

Cloudflare surveyed the CPU market during the design phase and lab-tested many candidates before short-listing three 4th generation AMD EPYC parts for production evaluation: Genoa 9654, Bergamo 9754, and Genoa-X 9684X. The table below compares their specifications against the AMD EPYC 7713 used in Gen 11 servers. All three candidates raise core count substantially and lift all-core boost clock frequency markedly.

CPU Model AMD EPYC 7713 AMD EPYC 9654 AMD EPYC 9754 AMD EPYC 9684X
Series Milan Genoa Bergamo Genoa-X
# of CPU Cores 64 96 128 96
# of Threads 128 192 256 192
Base Clock 2.0 GHz 2.4 GHz 2.25 GHz 2.4 GHz
Max Boost Clock 3.67 GHz 3.7 Ghz 3.1 Ghz 3.7 Ghz
All Core Boost Clock 2.7 GHz * 3.55 GHz 3.1GHz 3.42 GHz
Total L3 Cache 256 MB 384 MB 256 MB 1152 MB
L3 cache per core 4MB / core 4MB / core 2MB / core 12MB / core
Maximum configurable TDP 240W 400W 400W 400W

*Note: AMD EPYC 7713 all core boost clock frequency of 2.7 GHz is not an official specification of the CPU but based on data collected at Cloudflare production fleet.

Each short-listed CPU was tuned as far as the team's knowledge allowed, with thermal design power (TDP) set to 400W for maximum performance. The servers ran the same processes and services as any other production machine, enabling a direct side-by-side comparison.

Milan 7713 Genoa 9654 Bergamo 9754 Genoa-X 9684X
Production performance (request per second) multiplier 1x 2x 2.15x 2.45x
Production efficiency (request per second per watt) multiplier 1x 1.33x 1.38x 1.63x

Why Genoa-X 9684X won

Every candidate more than doubles the performance of the previous generation. The AMD EPYC 9684X Genoa-X, with 3D V-cache technology, posted the largest gain: 2.45x against Gen 11 servers built on the AMD EPYC 7713 Milan.

Genoa-X 9684X leads Genoa 9654 by roughly 22.5%. The main difference between the two parts is L3 cache: 1152 MB on Genoa-X 9684X against 384 MB on Genoa 9654, a threefold gap. Cloudflare workloads benefit from more low-level cache being reachable and thus avoid the much larger latency penalty of fetching data from memory.

The 3x larger L3 cache does draw additional power, but at 400W Genoa-X 9684X still delivered about 22.5% better performance than Genoa 9654, giving up only 3% of its highest achievable all-core boost frequency — a favorable trade-off for these workloads.

Overall, Genoa-X 9684X produced a 145% performance improvement for a 50% rise in system power, a 63% power efficiency gain with a substantial effect on operational expenditure. A large share of that efficiency comes from the CPU, but it only materializes with the right thermal-mechanical design. Cloudflare earlier doubled the server chassis height from 1U to 2U to improve rack density and cooling efficiency. That move was estimated to cut fan power by 150W, taking system power from 750 watts to 600 watts — and a Gen 12 server indeed consumes 600 watts per system at a typical ambient temperature of 25°C.

High performance usually carries a high price, but the AMD EPYC 9684X balances cost and capability well: top-tier performance without a huge financial outlay, and a solid Total Cost of Ownership improvement for Cloudflare.

Memory: 384 GB across twelve DDR5 channels

The AMD Genoa-X CPU supports twelve memory channels of DDR5 at up to 4800 mega transfers per second (MT/s), for per-socket memory bandwidth of 460.8 GB/s. All twelve channels are populated with 32 GB ECC 2Rx8 DDR5 RDIMMs in a one-DIMM-per-channel layout, for 384 GB total.

Sizing memory is a balancing act: the memory-to-core ratio must keep either CPU capacity or memory capacity from going to waste. Gen 11 servers with 64-core AMD EPYC 7713 CPUs also carry 384 GB, about 6 GB per core. Gen 12 raises core count to 96 but keeps 384 GB. The reason is memory optimization work done since Gen 11 — including Bot Management code optimization and the transition to the highly efficient Pingora — plus per-service memory allocations sized for optimal performance and managed through Linux control group resource management features. In sizing Gen 12, the team consulted the group that monitors resource allocation and reviewed memory utilization metrics from the fleet; the analysis put the optimal ratio at 4 GB per CPU core, or 384 GB total, and the configuration is validated in production. Dual rank modules were chosen over single rank for their higher memory throughput.

Using the Intel Memory Latency Checker (MLC), peak memory bandwidth was measured for 12 channels of dual-rank (2Rx8) 32 GB DIMMs against 12 channels of single rank (1Rx4) 32 GB DIMMs. Dual rank parts show slightly higher (1.8%) read bandwidth and a noticeably higher write bandwidth: as write ratios rose from 25% to 50%, the throughput delta grew by 10%.

Benchmark Dual rank advantage over single rank
Intel MLC ALL Reads 101.8%
Intel MLC 3:1 Reads-Writes 107.7%
Intel MLC 2:1 Reads-Writes 112.9%
Intel MLC 1:1 Reads-Writes 117.8%
Intel MLC Stream-triad like 108.6%

The AMD STREAM benchmark, which measures sustainable main memory bandwidth in MB/s and the matching computation rate for simple vector kernels, shows dual rank DIMMs ahead of single rank in all four kernel types.

Benchmark Dual rank advantage over single rank
Stream Copy 115.44%
Stream Scale 111.22%
Stream Add 109.06%
Stream Triad 107.70%

Storage: moving from M.2 to E1.S

Gen X and Gen 11 servers used M.2 form factor drives, valued mainly for compactness. But the M.2 connector, introduced in 2012, is now dated, and the industry has concerns about its signal integrity at the speeds defined by PCIe 5.0 and PCIe 6.0. The 8.25W thermal limit of M.2 also caps the number of flash dies per drive and therefore maximum capacity. The industry's answer is the E1.S specification, and the transition away from M.2 is underway.

Gen 12 adopts the EDSFF E1 form factor, specifically E1.S 15mm. It remains compact while offering more room for flash dies and larger capacities, and its better cooling design supports more than 25W of sustained power.

The AMD Genoa-X CPU offers 128 PCIe 5.0 lanes, but Gen 12 keeps NVMe devices on PCIe Gen 4.0 x4 lanes: Gen 4.0 throughput meets drive bandwidth requirements and keeps server design costs optimal. Two 8 TB NVMe drives provide 16 TB total. Two 8 TB drives were chosen over four 4 TB drives because the dual configuration already supplies sufficient I/O bandwidth for every Cloudflare workload running on each server.

Sequential Read (MB/s) : 6,700
Sequential Write (MB/s) : 4,000
Random Read IOPS: 1,000,000
Random Write IOPS: 200,000
Endurance 1 DWPD
PCIe GEN4 x4 lane throughput 7880 MB/s

Networking and NIC dual-sourcing

Cloudflare servers and top-of-rack (ToR) network equipment run at 25 GbE. Gen 12 uses a DC-MHS motherboard-inspired design and upgrades from the OCP 2.0 form factor to OCP 3.0, which allows tool-less NIC serviceability. OCP 3.0 also takes less space in the 2U chassis than PCIe-attached NICs, improving airflow and freeing room for other application-specific PCIe cards such as GPUs.

Since the Gen 9 servers in 2018, Cloudflare relied on the Mellanox CX4-Lx EN dual port 25 GbE NIC — a single source. Pandemic-era supply constraints and extremely long lead times forced the team to qualify the Broadcom M225P dual port 25 GbE NIC as a second source in 2022 so servers could keep coming online. Applying that lesson, Gen 12 is dual-sourced on the Intel Ethernet Network Adapter E810 and the NVIDIA Mellanox ConnectX-6 Lx. Both comply with the OCP 3.0 specification and expose more MSI-X queues, which can be mapped to the higher core count of the AMD EPYC 9684X. The Intel E810 adds full Generic Segmentation Offload (GSO) support, including VLAN-tagged encapsulated traffic, where many vendors offer only Partial GSO or none at all. With full GSO, the kernel spends noticeably less time in softirq segmenting packets, and servers with Intel E810 NICs process approximately 2% more requests per second.

Security: DC-SCM 2.0 and Project Argus

BLOG-3226 image 3

Gen 12 integrates Project Argus, one of the industry's first implementations of Data Center Secure Control Module 2.0 (DC-SCM 2.0), which decouples server management and security functions from the motherboard. The baseboard management controller (BMC), hardware root of trust (HRoT), trusted platform module (TPM), and dual BMC/BIOS flash chips all sit on the DC-SCM.

On Gen X and Gen 11, Cloudflare moved the secure boot trust anchor from the system BIOS or UEFI firmware to hardware-rooted boot integrity — AMD's implementation of Platform Secure Boot (PSB) or Ampere's implementation of Single Domain Secure Boot. Those solutions hardened the infrastructure against BIOS/UEFI firmware attacks but left exposure to out-of-band attacks that compromise BMC firmware; a compromised BMC can, for example, read processor console logs and control server power states. On Gen 12, the HRoT on the DC-SCM serves as the trust store for cryptographic keys and authenticates both the BIOS/UEFI firmware (independent of CPU vendor) and the BMC firmware for the secure boot process. The DC-SCM also carries additional flash storage for backup BIOS/UEFI and BMC firmware, enabling rapid recovery from corrupted or malicious firmware and resilience to flash chip failure from aging. The result is a server more secure and more resilient to firmware attacks.

Power delivery and efficiency

At 25°C ambient, a Gen 12 server draws 600 watts. That is 50% above the 400 watts of Gen 11, but a relatively small price for a 145% performance increase. The server uses dual 800W common redundant power supplies (CRPS) with 80 PLUS Titanium grade efficiency. Both PSUs operate actively with distributed power and current, and both are hot-pluggable, so the server runs with redundancy and maximizes uptime.

80 PLUS is a PSU efficiency certification program. Titanium grade units are 2% more efficient than Platinum grade between a typical operating load of 25% to 50%. Across a fleet of Cloudflare's size, that 2% over the lifetime of all Gen 12 deployments amounts to a reduction of more than 7 GWh, equivalent to carbon sequestered by more than 3400 acres of U.S. forests in one year. The upgrade also brings the Gen 12 server into compliance with EU Lot9 requirements, allowing deployment in the EU region.

80 PLUS certification 10% 20% 50% 100%
80 PLUS Platinum - 92% 94% 90%
80 PLUS Titanium 90% 94% 96% 91%

GPU flexibility for Workers AI

Demand for machine learning and AI workloads exploded in 2023, and Cloudflare introduced Workers AI. GPUs were retrofitted or deployed worldwide in part of the Gen 11 fleet to support its growth. Gen 12 is likewise designed to take more powerful GPUs, giving Cloudflare flexibility to support Workers AI in all regions and to place GPUs strategically to cut inference latency. A GPU-equipped server can still run Cloudflare's full software stack, serving general web requests when GPU utilization is low.

The motherboard's electrical design supports up to two PCIe add-in cards, and the power distribution board is sized for an additional 400W. Mechanically, the chassis accepts either one FHFL (full height, full length) double width GPU PCIe card or two FHFL single width GPU PCIe cards. Component placement, fans, and air duct design are sized to support GPUs with TDP up to 400W.

What comes next

Gen 12 servers are deployed and live in multiple Cloudflare data centers worldwide, already processing millions of requests per second. The EPYC journey continues: 5th-gen AMD EPYC CPUs (code name "Turin") are already available for testing, and architecture planning and design discussions for the Gen 13 server are set to begin.