Finding Headroom in the EPYC 7642

Cloudflare's tenth-generation edge servers are built around AMD's 2nd Gen EPYC 7642, a chip chosen for its frequency headroom and cache-to-core ratio. In partnership with AMD, we've been tuning this processor to squeeze out additional throughput beyond the factory settings. Our efforts to date have yielded an extra 6% in performance, achieved through a combination of specific power and determinism settings.

TDP Is Not a Power Cap

A common misconception is that Thermal Design Power (TDP) represents the maximum or average power draw of a processor. In reality, TDP is a guideline for the thermal solution—the amount of heat the cooling system should be able to dissipate indefinitely. For the 48-core EPYC 7642, the 225W TDP is the same as the 64-core EPYC 7742. The 7642 doesn't use that budget for more cores; instead, it uses it to retain its full 256 MiB of L3 cache and maintain a higher sustained clock frequency, particularly during peak loads.

The actual power drawn is determined by more than just the rated TDP. Dynamic power is a function of capacitance, supply voltage, frequency, and the activity factor of the workloads. Workloads like Cloudflare Workers or Cloudflare for Teams have different hotspots that utilize distinct parts of the processor, which directly affects the activity factor and thus the power draw.

Dialing in Power and Performance

The EPYC 7642 supports AMD's Precision Boost, which opportunistically raises frequency above the base clock. To control this behavior, AMD provides determinism modes that manage the trade-off between power and frequency.

  • Performance Determinism: In this mode, power becomes a function of frequency. The processor stabilizes its frequency to match its lowest-performing core, ensuring predictable cycles per second across all cores.
  • Power Determinism: Here, frequency is a function of power. The processor maximizes both power and frequency as much as the operating envelope allows.

Another available knob is Configurable TDP (cTDP). While the EPYC 7642 is rated at 225W, AMD guidance indicates this specific part can be reconfigured up to 240W. We validated that our cooling solution—fans paired with a heatsink rated for 240W—can dissipate that amount of heat even at higher ambient temperatures.

Production Testing on Real Assets

We tested these configurations using 10 KiB web assets over HTTPS for a sustained period to properly heat the processor. Our measurements tracked average operating frequency across all 48 cores, total socket power, and the highest temperature reported across the eight dies. Performance determinism emphasized frequency stability but showed an unexpected frequency jitter during the warm-up phase, which contradicted its purpose of predictability. Power determinism, by contrast, continuously pushed the processor to its limits.

Regarding thermal performance, our cooling solution worked as intended: fan speed ramped up over time to prevent the processor from reaching critical temperatures. The processor operated within safe limits, and we have yet to observe throttling in production.

Real-World Gains

When comparing production results against our baseline—power determinism with a default 225W TDP—none of our data showed a negative trend. The deltas we achieved are as follows:

  • Power determinism alone provided an additional 2% in requests per second.
  • Reconfiguring the TDP to 240W yielded another 4% in throughput.

NUMA Domain Options

The EPYC 7642's eight dies are laid out across four quadrants on a single package. AMD supports dividing these dies into multiple NUMA domains via the Nodes Per Socket (NPS) feature, which is configurable between 1, 2, or 4 nodes per socket on this part. We explored these options, but observed no significant performance deltas between them.

Our work with AMD has successfully identified an additional 6% throughput out of the box. Some processor parts will offer more tuning headroom than others, and we will continue investigating further areas of improvement with our internal teams and AMD.