Revisiting SMT and Boost on AMD EPYC

Cloudflare's engineering team recently set out to quantify the impact of simultaneous multithreading (SMT) and Core Performance Boost (CPB) on its AMD-based Gen X servers. The last time these technologies were evaluated in depth was during the company's deployment of Intel Xeon (Skylake/Purley) systems. Since then, the fleet has transitioned to AMD EPYC (Zen 2/Rome) processors, making a fresh assessment necessary.

While Intel introduced Hyper-Threading with NetBurst in 2002 and later paired it with Turbo Boost on Nehalem in 2008, AMD's equivalents—SMT and CPB—have been present since the Zen architecture launched in 2017. AMD's boost technology actually traces back further, to the K10 (Thuban) era in 2010 when it was called Turbo Core. Though the vendors' implementations differ at the hardware level, the core concepts are the same: SMT creates a second logical hardware thread per physical core by duplicating context-supporting components, allowing two threads to execute in parallel across shared resources. Boost technologies let the processor exceed its rated base frequency as long as thermal and power guidelines are met.

Lab Results: Controlled Testing

The test platform was the AMD EPYC 7642, a 48-core Zen 2 processor capable of running 96 hardware threads with SMT enabled. Its base frequency is 2.3 GHz, with CPB allowing operation up to 3.3 GHz. Benchmarks used the standard simulated traffic pattern of 10 KiB cached assets over HTTPS, sufficient to saturate the CPU to 100% utilization.

Feature-by-Feature Gains

Starting from a baseline with both SMT and CPB disabled, the team enabled one feature at a time:

  • Core Performance Boost alone: The processor ran near its peak frequency (3.2–3.3 GHz), over 39% above base. This translated directly into a 40% increase in requests per second.
  • SMT alone: Similarly delivered a 43% improvement in throughput.
  • Both enabled: Combined for an 86% gain in requests per second over the baseline.

Latency generally improved with either feature active. CPB held a consistent latency advantage over the baseline across all percentiles. SMT, however, showed gradually increasing latency toward the tail. Beyond the 99.99th percentile, SMT—even when paired with CPB—produced latencies more than 150% above baseline, likely due to shared-resource contention between hardware threads within a core.

Production Testing: A Skewed Baseline

For the production experiment, four identical Gen X servers were run in parallel during peak traffic hours across two regions. The only variable was enabling or disabling SMT and CPB in a full test matrix. Both regions produced matching trends.

An issue emerged: the baseline server (with both features disabled) operated at a higher CPU utilization than the others. Cloudflare's load balancer, Unimog, normally distributes connections based on server generation to balance CPU load. With SMT and CPB disabled, the baseline server's performance degraded to the point where Unimog hit its lower limit on requests sent, forcing the server to run at higher utilization to meet the minimum throughput threshold.

What the Data Showed

The skewed baseline muddied the results. In production:

  • CPB alone: Showed only a 7% gain in requests per second.
  • SMT alone: Improved throughput by 41%.
  • Both features: Delivered an 86% improvement.

By normalizing requests per second against CPU utilization to correct for the baseline imbalance, the estimated improvements were 36% for CPB, 80% for SMT, and 136% with both enabled. Latency results were also confounded by the baseline's higher utilization, which produced elevated tail latencies. The other servers maintained lower latencies thanks to lower CPU load combined with CPB, SMT, or both.

Power Efficiency Findings

Adding power measurements provided the most useful insights of the experiment. The baseline server, operating at higher utilization and serving more requests, consumed more power than necessary. Enabling CPB increased power draw by 35% over the skewed baseline, as the processor ran at its peak turbo frequency. Enabling SMT alone raised consumption by just 7%, yet delivered a 41% throughput gain—a notably power-efficient outcome.

Combining both features resulted in a 58% increase in power consumption. When bridging performance and power into a per-watt metric, the best results came from enabling SMT alone. The higher frequency from CPB delivers meaningful latency reduction and additional throughput, but its power cost is substantial and worth weighing carefully.

In the end, the original experiment design did not hold up in production, producing only partial answers. The power analysis, however, yielded actionable conclusions: SMT is a power-efficient way to extract additional request throughput, with tail-latency drawbacks that CPB currently helps mitigate. The tradeoff is that CPB's frequency boost brings significant power demands.