How we could save petabytes of cache storage with Zstandard and Pingora

Stretching Cache Headroom by Compressing at Rest

With memory and storage prices climbing, Cloudflare’s distributed CDN faces pressure to extract more value from the hardware it already runs. A prototype engineering project, called Cache Transcoding, tackles that by changing how eligible objects are stored in its Pingora-based proxy. The idea is straightforward: compress an asset with Zstandard before it is written to the cache, keep it compressed as it moves between tiers, and decompress it only when it is served to a client.

The trade-off is a deliberate one. Encoding costs CPU once, at cache fill time. The payoff — fewer bytes on disk and smaller transfers across the backbone — is realized on every subsequent cache hit. In early testing, eligible assets shrank to roughly one-third of their original size on average, while the projected CPU overhead stayed in the low single digits.

Why Zstandard Fits the Cache Path

Zstandard (zstd) is a lossless compression algorithm open sourced in 2016. Losslessness is a hard requirement here: decoding must reproduce the original bytes exactly. The algorithm’s design goal is balancing compression ratio against speed, which matters when the codec runs over a large share of cache traffic.

Cloudflare’s prior browser compression testing found zstd compressed data 42% faster than Brotli at a comparable output size, and produced files 11.3% smaller than gzip at similar speeds. The Cache Transcoding prototype runs zstd at level 3, a setting that captures most of the compression benefit without making cache fills a CPU bottleneck. The level and the size threshold are configurable parameters, not fixed limits.

Traditionally, Cloudflare stores an asset using whatever Content-Encoding the origin supplied. That means uncompressed origin responses are written to disk and moved between data centers in their original form. Cache Transcoding injects a compression step into the cache itself, changing the on-disk representation without altering the asset’s identity.

Keying Off What Actually Compresses

Not all bytes are worth re-encoding. Media files — images, video, fonts — are already compressed formats. In Cloudflare’s traffic sample, that content accounted for 21.4% of requests but 63.3% of bytes. Running zstd over those files would burn CPU for negligible gain.

Text is the target. HTML, JSON, CSS, and JavaScript represented 67.3% of requests and 22.3% of bytes. Roughly 71% of that text arrived with no Content-Encoding header, meaning it was uncompressed and highly compressible. In the controlled test corpus, those eligible assets compressed by roughly 2.8 times.

The encoding cost is not symmetric with the benefit. Assets are served far more often than they are filled, so paying compression once at ingest buys a persistent reduction in storage footprint. That means each server can retain more unique objects, improving cache density and reducing evictions caused by an inefficiently large on-disk representation. The same smaller representation also trims the bytes transferred between Cloudflare data centers when an asset travels through Tiered Cache.

Weighing the Cost of Compression

Compression consumes CPU on both the encode and decode paths. The question the team tested was whether the byte savings justified that processing. At zstd level 3, and under the traffic and reuse assumptions of the model, the extra CPU cost landed at a few percent.

An early idea was to restrict transcoding to the most popular content, assuming hot assets would amortize the cost better. That did not help. Since decoding runs on every cache hit, limiting the feature to popular objects cut the storage savings without a proportionate drop in CPU usage. The simpler blanket policy won: transcode all eligible compressible text at or above 4 KiB. That threshold captured nearly all of the measured storage benefit while keeping CPU inside budget.

Flow Through the Cache Tiers

The mechanics differ by cache state:

• On a full miss, the upper tier fetches identity bytes from the origin, encodes them with zstd once, and stores the compressed form. It transfers the zstd bytes to the lower tier, which also stores them compressed and decodes only for the client-facing response.
• On a tiered miss where the upper tier has the object, the origin is untouched. The compressed object moves directly between cache tiers, staying compressed both on the wire and on disk until it is decoded at the lower tier.
• On a local hit, no network transfer or encoding occurs. The lower tier reads the zstd bytes from disk, decodes them, and serves the original asset.

A storage encoding marker is key to keeping this efficient. When a cache layer receives an object from another tier, that marker tells it the asset is already stored as zstd, preventing any double-encoding.

Eligibility: Not All Text Is Created Equal

The prototype routes a response through a series of checks before transcoding. The response must be a 200 OK with no Content-Encoding set, a compressible text Content-Type, and a known Content-Length of at least 4 KiB. Slice subrequests, responses with active upstream compression, range requests, precompressed content, unknown-length bodies, and binary types all pass through unchanged.

The 4 KiB floor is a deliberate filter. It removes a vast number of tiny requests while giving up only about 1% of the otherwise eligible bytes. Dropping the threshold lower would add per-object overhead with marginal storage return.

Validation Across a Million Requests

The team exercised the prototype against a controlled test zone, correlating every request through logs, Prometheus metrics, and Jaeger traces to verify the code paths. Tests covered cache misses, hits, single-hop fills, and Tiered Cache fills, with trace data confirming exactly where encoding and decoding happened.

One performance run pushed over one million requests across 10 cache servers. Half ran with Tiered Cache disabled, half with it enabled, isolating local cache behavior from inter-tier transfer costs. The two test assets — about 195 KiB and 272 KiB — both compressed by roughly 2.8 times. That corpus was intentionally compressible to give a clear architectural signal; the team notes that a broader content sample is needed before treating that ratio as a fleet-wide figure.

The Next Compression Frontier

The experiment showed that measurable gains remain available in a large-scale caching service. The architecture preserved content fidelity and stayed within its CPU budget under the tested conditions. Further work will evaluate higher zstd levels, a wider range of content types and object sizes, and adjustments to the eligibility parameters. Range requests, pre-compressed origin responses, and passing compressed bytes straight through to downstream components that already support them are also on the roadmap.