Compression: Zstandard's Trade-offs

Zstandard (zstd) has emerged as a promising option for web performance work. Cloudflare reports that zstd compresses data 42% faster than Brotli while maintaining almost the same compression levels. It also cuts file sizes by 11.3% relative to GZIP at comparable speeds. Because compression speed and efficiency translate directly into latency, the algorithm matters for the user experience across the web.

That work fits a longer pattern at Cloudflare, which has spent 14 years trying to lower the cost of adopting new web technologies. The company points to being first to offer free SSL certificates through Universal SSL, followed by prompt support for TLS 1.3, IPv6, and HTTP/3.

Encrypted Client Hello Resumes Rollout

Cloudflare is restarting the rollout of Encrypted Client Hello (ECH), a new proposed standard that keeps networks from snooping on which websites a user visits. ECH succeeds ESNI and masks the Server Name Indication (SNI) used to negotiate a TLS handshake. When a user visits a Cloudflare-hosted site with ECH enabled, only the user, Cloudflare, and the site owner can determine which site was visited.

Beyond the Handshake

Cloudflare is also measuring the impact of HTTP/3 prioritization and developing Bottleneck Bandwidth and Round-trip propagation time (BBR) congestion control to further optimize network performance.

Where Zstandard fits among the existing algorithms

Compression remains the default performance lever for most of the web. W3 Technologies measured on September 12, 2024 that 88.6% of sites compress their responses; GZIP, defined in 1996, is still the default for many and serves 57.0% of sites, while Brotli, released by Google in 2016, is used by 45.5% and produces smaller files, particularly for static assets such as JavaScript and CSS. The remaining 11.4% of sites send nothing compressed at all.

Every algorithm trades compression time against compression ratio: a 10:1 ratio means the compressed output is one-tenth the original size, and pushing that number higher generally costs processing time on both ends, while faster methods leave files larger and consume more bandwidth. Zstandard targets the middle of that curve — ratios in GZIP's range, with compression and decompression considerably faster than Brotli — which suits real-time workloads that need both speed and a decent ratio.

The algorithm was created by Yann Collet at Facebook and open sourced in August 2016 for large-scale data processing. Chromium-based browsers and Firefox added support in March 2024, and Cloudflare's implementation is now generally available between its network and browsers.

Why Zstandard is fast: memory, parallelism, branches, entropy coding

Zstandard's lineage runs through Zlib, the mid-1990s implementation of DEFLATE (LZ77 plus Huffman coding) that still underpins Cloudflare's open-source GZIP. Zlib is held back by a 32 KB sliding window, a limit inherited from the memory available when it was designed, so it cannot exploit the far larger memory of current hardware. Zstandard imposes no such fixed window and can in theory address terabytes, though at lower compression levels it typically uses around 1 MB. That larger buffer lets it find repeating patterns more effectively, and repcode modeling handles structured, repetitive data at lower cost.

The implementation is also built for modern CPUs. Data is split into parallel streams processed concurrently, and the Huff0 Huffman decoder can decode multiple symbols at once on a single core; combined with multithreading, this speeds up both directions.

Branchless code matters just as much. Pipelined CPUs work like a production line, and a conditional branch forces a branch prediction that, when wrong, drains and restarts the pipeline. By avoiding conditional branching altogether, Zstandard keeps instructions flowing and the pipeline full.

For entropy coding, Zstandard uses Finite State Entropy (FSE), built on the Asymmetric Numeral System (ANS). Unlike Huffman coding, which spends whole bits per symbol, FSE encodes with fractional bits, so highly repetitive data packs tighter without a loss of efficiency.

Measured against Brotli and GZIP

In Q3 2024 Cloudflare ran the new module over a 24-hour window on Free-plan traffic, switching the default from Brotli to Zstandard across billions of requests spanning HTML, CSS, JavaScript and other types and sizes.

Average compression ratios:

Compression Algorithm

Average Compression Ratio

GZIP

2.56

Zstandard

2.86

Brotli

3.08

BLOG-2562 image 2

Zstandard averaged 2.86:1, ahead of gzip at 2.56:1 and just behind Brotli at 3.08:1. Notably, the test ran at Zstandard's default level of 3 out of 22; higher levels and multithreading were slated for experimentation in Q4 2024.

Average compression times:

Compression Algorithm

Average Time to Compress (ms)

GZIP

0.872

Zstandard

0.848

Brotli

1.544

BLOG-2562 image 3

Zstandard compressed 42% faster than Brotli — 0.848 ms against 1.544 ms — and was slightly quicker than gzip's 0.872 ms, attaining Brotli-sized output in roughly half the time. The combination makes it a good fit for dynamic responses such as HTML and non-cacheable sensitive data: faster than Brotli at the origin, with better ratios than GZIP.

Building it into Nginx

The module lives alongside the existing GZIP and Brotli handling in Cloudflare's Nginx-based service, hooking into Nginx's header filter and body filter.

The header filter inspects and modifies response headers. Responses must exceed a minimum size — 50 bytes for Zstandard — checked via Content-Length:

if (r->headers_out.content_length_n != -1 &&
    r->headers_out.content_length_n < conf->min_length) {
    return ngx_http_next_header_filter(r);
}

Below that threshold, compression is skipped and Nginx moves to the next module. Already-encoded content is likewise left alone, detected through Content-Encoding:

if (r->headers_out.content_encoding &&
    r->headers_out.content_encoding->value.len) {
    return ngx_http_next_header_filter(r);
}

The body filter does the real work: run chunks through the Zstandard encoder and stream the output to the client. Large responses cannot be held in memory, so internal buffers are managed carefully. The library's streaming interface, ZSTD_compressStream2, accepts input and output buffers plus a ZSTD_EndDirective endOp operation that says whether to continue feeding data, flush, or finalize:

ZSTDLIB_API size_t ZSTD_compressStream2(ZSTD_CCtx* cctx,
                                        ZSTD_outBuffer* output,
                                        ZSTD_inBuffer* input,
                                        ZSTD_EndDirective endOp);

Nginx's buffer "flush" flag maps onto that operation, switching the encoder from ZSTD_e_continue to ZSTD_e_flush to draw compressed bytes out:

switch (zstd_operation) {
    case ZSTD_e_continue: {
        if (flush) {
            zstd_operation = ZSTD_e_flush;
        }
    }
}

A request then cycles through four steps:

  1. Receive uncompressed data.
  2. Locate an internal buffer to store compressed data.
  3. Compress the data with Zstandard.
  4. Send the compressed data back to the client.

Filled buffers pass down the Nginx chain and are recycled once released, avoiding repeated allocation:

if (free) {
    // A free buffer is available, so use it
    buffer = free;
} else if (buffers_used < maximum_buffers) {
    // No free buffers, but we're under the limit, so allocate a new one
    buffer = create_buf();
} else {
    // No free buffers and can't allocate more
    err = no_memory;
}

If every buffer is in use, the encoder can outpace the client and data stalls inside Nginx — backpressure. The module then stops compressing and hands the next Nginx module an empty buffer so it can retry the send; success releases buffers for reuse and streaming continues without buffering the whole response.

Compression Rules and enabling Zstandard

Compression Rules are now available to all pay-as-you-go customers and arrive on free plans in October 2024, after previously being an Enterprise-only feature. Built on the same framework as Origin Rules, Custom Firewall Rules and Cache Rules, they add fields for Media Type and Extension Type so customers can specify exactly what gets compressed; Enterprise plans now allow 125 rules, matching other rule products' quotas. By default Cloudflare compresses certain content types according to their Content-Type headers.

Zstandard is selectable within that framework, and choosing it enables the module in Nginx while disabling the other compression modules where needed. Negotiation happens through Accept-Encoding: when the browser advertises zstd and both Cloudflare and the zone have the feature on, the response comes back Zstandard-compressed:

BLOG-2562 image 4

Clients without Zstandard support fall back to Brotli, GZIP, or uncompressed content. A single rule suffices to turn it on site-wide or restrict it to particular file types:

BLOG-2562 image 5

Traffic sent to clients can currently use Zstandard, Brotli or GZIP; data from origins supports GZIP and Brotli, the latter since 2023. Full end-to-end Zstandard support is planned for 2025 as another way to cut egress costs. Once enabled, the response's content-encoding header can be verified in the browser's Network Activity log.

BLOG-2562 image 5
BLOG-2562 image 6

Next: compression dictionaries

Dictionaries storing common patterns — defined statically by an RFC, or custom-built for the content being served — let both Brotli and Zstandard reference sequences instead of repeating them; against static dictionaries, they have shown up to 90% improvement in compression. A custom dictionary tuned to frequently used terms could compress them more efficiently and still decode correctly in a browser holding the same dictionary.

O

M

W

O

n

M

y

W

a

y

Future work includes letting origins supply their own dictionaries for both algorithms, and exploring AI-generated dictionaries built from data streams in real time, so dictionaries are context-aware without being produced at the origin — potentially improving both ratio and processing speed.

Encrypted Client Hello (ECH)

BLOG-2562 image 7

Faster transfer alone does not complete the picture: the connections carrying that data still leak metadata. The last major leak in a TLS session is the Server Name Indication (SNI), which travels in plaintext during the handshake. Encrypted Client Hello (ECH) closes that gap, and it is the piece Cloudflare is now shipping at scale. Cloudflare first announced ECH support in 2023 and has since refined the implementation together with its browser partners.

The plaintext handshake problem

To open a secure connection, a client must name the site it wants. That name is the SNI, and on its own path it is readable, much like a letter whose envelope shows the address to anyone following the courier. ECH replaces that with delivery to a shared P.O. Box: sensitive details sit inside a sealed inner envelope, while the outer one is addressed to a public drop point. The outer envelope is visible to all; only the intended server can open the inner one.

Encrypting the server name is the headline benefit, but the scope is larger. ECH encrypts the whole Client Hello, which also lets TLS keep evolving without exposing connection metadata along the way.

Outer and inner messages

ECH splits the ClientHello message into two parts:

  • ClientHelloOuter: sent cleartext, with innocuous values in place of sensitive TLS extensions. Its SNI names Cloudflare's public domain, currently cloudflare-ech.com, which Cloudflare manages and holds certificates for, so it can complete the TLS negotiation.
  • ClientHelloInner: encrypted under a public key and carrying the real destination name plus the other sensitive extensions. Only the client-facing server — Cloudflare, in this case — can decrypt it.

An observer during the handshake therefore sees only the public name; the requested site stays hidden.

BLOG-2562 image 7

ECH depends on other IETF work to be deployable. The HTTPS DNS resource record (RFC 9460) lets an HTTPS endpoint publish its TLS capabilities, which simplifies key distribution, and Encrypted DNS lets clients query those records anonymously. Handshake encryption itself uses Hybrid Public Key Encryption (HPKE).

Before opening a layer 4 connection, the browser looks up the site's HTTPS record. A zone with ECH enabled publishes an ECH configuration in that record, containing the encryption public key and associated metadata. The cloudflare-ech.com zone, for example, returns the following:

dig cloudflare-ech.com https +short

1 . alpn="h3,h2" ipv4hint=104.18.10.118,104.18.11.118 ech=AEX+DQBB2gAgACD1W1B+GxY3nZ53Rigpsp0xlL6+80qcvZtgwjsIs4YoOwAEAAEAAQASY2xvdWRmbGFyZS1lY2guY29tAAA= ipv6hint=2606:4700::6812:a76,2606:4700::6812:b76

Alongside the public key and configuration parameters, the record carries the configured public name:

Y2xvdWRmbGFyZS1lY2guY29t

Decoded, that string reveals:

cloudflare-ech.com

The client places this name in the ClientHelloOuter.

What observers see

Traffic between a client and Cloudflare looks like a uniform TLS handshake aimed at cloudflare-ech.com, whatever site is actually being fetched. A user visiting example.com is invisible as such to an intermediary; only the public name appears in the visible handshake data.

BLOG-2562 image

Middleboxes and fallback

Not every connection runs straight from browser to origin. TLS-terminating forward proxies sit in the path, decrypting traffic and re-encrypting it toward the destination, typically after a root certificate has been installed on the device so the proxy can present a trusted certificate. Administrators use such setups, as with Cloudflare WARP, to keep sensitive company data from reaching unsanctioned destinations.

These proxies are frequently not ECH-aware. Because a proxy terminates the TLS connection without ECH support, it may hand back a valid certificate for the public name while being unable to decrypt the ClientHelloInner or supply the client with a new public key. The client then treats ECH as disabled — losing ECH and absorbing the cost of an extra round trip.

Two Cloudflare-specific configurations could also break ECH: CNAME Flattening and Orange-to-Orange setups. In both, the final destination did not support TLS 1.3, so ECH could not be processed. Working with its browser partners, Cloudflare added a fallback to the BoringSSL implementation that handles TLS terminations, which lets browsers retry over TLS 1.2 without ECH rather than fail the connection.

ECH is now on by default for all Free plans; other plan types can enable it in the Cloudflare dashboard or through the API.

Experimenting with priorities and congestion control

HTTP/3 Prioritization and QUIC congestion control are two further areas of investment aimed at improving performance for all customers. The first manages the order in which web assets load so that critical assets arrive sooner. It relies on Extensible Priorities, which reduces prioritization to two parameters: an urgency value from 0 to 7 and a true/false flag indicating whether a resource can be processed progressively. HTML, CSS and images can therefore be ranked by importance. QUIC congestion control, by contrast, targets the flow of data itself, preventing bottlenecks and keeping transmission smooth and reliable under heavy traffic.

Both changes affect how Cloudflare's network serves requests to clients. Before deploying them across a global network that handles peak volumes of over 80 million requests per second, the company first built a reliable way to measure their impact through rigorous experimentation.

Validating changes with real-world data

Measurement matters because optimizations in performance, security or reliability must deliver the intended benefit without introducing new problems. Aggregated real-world data is one of the best ways to capture performance changes. Cloudflare Web Analytics provides free, privacy-first analytics, while Real User Metrics (RUM) captures load times, resource usage and user interactions directly from real users' devices — reflecting the varied environments in which sites are actually accessed.

A key RUM indicator is Google's Core Web Vitals (CWV), which covers loading performance, interactivity and visual stability through Largest Contentful Paint (LCP), First Input Delay (FID) and Cumulative Layout Shift (CLS).

Over the last three months, Cloudflare developed the capability to add valuable information to Server-Timing response headers. When a page using Web Analytics loads, the privacy-first client-side script collects browser metrics and server-timing headers and sends this performance data back, where it is ingested, aggregated and made queryable. The header carries Layer 4 information such as Round-Trip Time (RTT) and protocol type (TCP or QUIC). Combined with Core Web Vitals, it lets engineers determine whether an optimization improved a request relative to a control sample — making it possible to ship large-scale changes like HTTP/3 Prioritization or BBR with a clear view of their global impact.

A sample header exposes several server-observed network properties:

server-timing: cfL4;desc="?proto=TCP&rtt=7337&sent=8&recv=8&lost=0&retrans=0&sent_bytes=3419&recv_bytes=832&delivery_rate=548023&cwnd=25&unsent_bytes=0&cid=94dae6b578f91145&ts=225
  • proto: the transport protocol in use.
  • rtt: round-trip time measured by the layer 4 connection, using a smoothing algorithm.
  • sent / recv: packets sent and received.
  • lost / retrans: packets lost and retransmitted.
  • sent_bytes / recv_bytes: total bytes sent and received.
  • delivery_rate: instantaneous data delivery rate in bytes per second.
  • cwnd: congestion window, an instantaneous packet or byte count depending on protocol.
  • unsent_bytes: bytes not yet sent.
  • cid: a 16-byte hexadecimal opaque connection ID.
  • ts: timestamp in milliseconds marking when the data was captured.

Continuous analysis of these network and performance insights lets Cloudflare make data-driven decisions and confirm that optimizations such as HTTP/3 Prioritization or BBR deployment deliver tangible benefits.

Shipping HTTP/3 Prioritization to every plan

While integrating the server-timing header for observability, Cloudflare made several minor changes to optimize QUIC handshakes. Telemetry showed positive improvements thanks to the Layer 4 observability the header provides, and these internal findings matched third-party measurements showing similar handshake gains.

In the fourth quarter of 2024, the same experimental methodology will be applied to the HTTP/3 Prioritization support announced during Speed Week 2023. The feature manages the delivery order of a page's many assets — images, scripts, stylesheets — so that critical elements such as primary content and layout arrive first. The testing framework will quantify improvements before the feature is enabled across all plan types, and, above all, verify that no performance regressions occur.

Evaluating congestion control algorithms

Once the HTTP/3 Prioritization experiments conclude, testing will begin on different congestion control algorithms, with a focus on BBR (Bottleneck Bandwidth and Round-trip propagation time) version 3. Congestion control regulates how much data travels over a network: sending too much too quickly causes congestion, packet loss, delays and lower overall performance. Each side of a connection runs such an algorithm in real time, adjusting the data flow to current and predicted network conditions.

BLOG-2562 image

BBR, initially developed by Google, estimates actual available bandwidth and minimum RTT to set the optimal flow, sustaining high throughput while minimizing latency. BBR v3 refines the bandwidth estimation techniques of BBRv1 and BBRv2 and adapts further to varying network conditions. Cloudflare found it faster than its previous CUBIC implementation in several cases, and — most importantly — it reduced loss and retransmission rates in the company's Oxy proxy implementation.

BLOG-2562 image

With those results, Cloudflare will test various congestion control algorithms, including BBRv3, for quiche, its QUIC implementation, across HTTP/3 traffic. Pairing Layer 4 server-timing information with these experiments makes it possible to control and measure the impact on real-world metrics.