Performance’s Hardware Horizon: Beyond BPF

Brendan Gregg’s recent plenary at USENIX LISA 2021 turned the spotlight away from the BPF observability work he is best known for, and toward the broader hardware and software trends that will define server performance in the coming years. The talk, “Computing Performance: On the Horizon,” covers developments across processors, memory, storage, and networking, and offers a set of forward-looking predictions from the perspective of a senior performance engineer.

The full session is available on YouTube, with slides published online and as a PDF. The material also expands on chapters from the second edition of Gregg’s Systems Performance book.

What’s Changing at the Silicon Level

The talk leads with the CPU. New processor designs are increasingly leveraging 3D die stacking to overcome the limits of traditional 2D scaling. Cloud vendors are also entering the silicon fray with in-house designs like the AWS Graviton2, which have shifted the performance conversation from raw clock speed to core count, memory bandwidth, and power efficiency.

Close behind the CPU are memory advances. DDR5 is beginning deployment, offering higher bandwidth and capacity per channel. More transformative, however, is the arrival of High Bandwidth Memory (HBM) integrated on-package. HBM provides a generational jump in memory bandwidth, which has become the binding constraint for many modern workloads, particularly in analytics and machine learning inference where data movement outpaces compute.

The Storage and Network Landscape

On the storage side, Gregg highlights new uses for 3D XPoint as an accelerator for 3D NAND, rather than as a permanent replacement for it. In this arrangement, the faster persistent memory acts as a caching or metadata layer in front of the dense but slower NAND. This is a practical evolution for workloads that need persistence without paying a full latency penalty.

Networking also earns a mention, driven by the rise of QUIC as a transport protocol and eXpress Data Path (XDP) as a programmable kernel networking hook. XDP, in particular, moves packet processing earlier in the stack, enabling high-throughput applications to run at near line rate with lower overhead than traditional socket-based paths.

A Prediction, With a Caveat

Gregg’s talk is not shy about looking ahead, and he notes that these predictions are intended to provoke thought rather than to serve as unassailable forecasts. The central theme for the server side is a move away from squeezing single-thread performance and toward orchestrating parallelism, memory tiers, and fast I/O stacks in a way that systems software can reliably exploit.

The full walkthrough of these trends, along with the reasoning and data behind them, is well worth watching for engineers involved in capacity planning, kernel development, or infrastructure design.

Digging Deeper: Reference Material and Further Reading

For those who want to explore the topics covered in Brendan Gregg's LISA 2021 presentation, the talk's reference list is a valuable resource. It spans over a decade of pioneering work across hardware, software, and networking. Below is a selection of the cited sources, which are a mix of blog posts, academic papers, and industry announcements that trace the evolution of the technologies discussed.

The list includes foundational work on analysis methodologies, such as Gregg’s own writing on visualization techniques and flame graphs. On the systems software side, sources cover the introduction of new I/O schedulers in the Linux kernel, the development of advanced congestion control algorithms for TCP, and the tooling built with BPF. Hardware developments are also well documented, covering the roadmap from HBM and DDR4 through to DDR5, the introduction of high-capacity storage technologies, and the rise of custom silicon for the cloud, including FPGA-based infrastructure and area-specific processors.

Systems Software and Analysis

Networking and Protocol Evolution

Memory, Processors, and Accelerators

Storage Evolution

Gregg notes that he made a point to include author names alongside titles and dates for all sources, rather than simply listing URLs. This practice, which he adopted after feeling it was unfair in his earlier books that some references had author names and others didn't, provides better attribution for the reader. For those interested in his other work from the same conference, he also presented a talk on BPF Internals.