Part 1: Rethinking Cache Purge, Fast and Scalable Global Cache Invalidation

Cache Invalidation at Scale: Where the Bottlenecks Live

Among the oft-quoted hard problems in computer science, cache invalidation is frequently listed as one of the two most difficult. In a global network, that difficulty is amplified by three factors: the sheer scale of the fleet, the speed required to avoid serving stale data, and the scope of what gets invalidated.

The scale issue is straightforward: with data centers in over 270 cities, a purge request must propagate to every node where an asset is cached. Each data center has to locate the content and force subsequent requests to re-fetch from origin. Maintaining consistency when some of those centers may be offline adds another layer of complexity.

Speed is the second challenge. Requests can arrive at any moment, so the window between sending a purge command and seeing it executed around the world is full of race conditions. This is especially problematic when a website updates a non-backwards-compatible asset; users hitting a mix of old and new versions can break pages entirely. Physical limits also play a role. A light-speed transit between Cloudflare locations in Tokyo and Cape Town takes over 180ms before any processing even begins. Shrinking the footprint reduces that problem but sacrifices performance for users elsewhere.

Finally, scope matters. Invalidation must be surgical. Purging everything risks a thundering herd back to the origin, while purging too little leaves visitors with outdated content. Cloudflare's existing options attempt to navigate this balance:

• Purge everything clears all cached content associated with a website.
• Purge by prefix targets a URL prefix.
• Purge by hostname invalidates based on a hostname.
• Purge by URL focuses on specific files.
• Purge by tag invalidates assets marked with Cache-Tag headers.

How the Current Pipeline Processes a Purge

The current system routes purge requests through a set of core data centers before distribution. The journey looks like this:

When an API or UI request arrives at any Cloudflare data center, it is identified as a purge and forwarded to a core data center tasked with network management. There, internal services handle authorization. After passing those checks, the request is fanned out to every data center using a distribution service. Each receiving data center then removes or marks the matching assets as stale, forcing a fresh pull from origin on the next request.

Benchmarking Current Purge Latency

To measure how this pipeline performs in the real world, Cloudflare sampled real-user-monitoring purge data over 48 hours in May 2022. The clock started when a purge request was received by a data center, including any retries required for centers that missed the first broadcast. The clock stopped when the asset was removed from cache and a miss was logged.

Results were segmented by purge type and by region, with data confined to specific servers within each geographic area to avoid clock skew. The regions covered included Africa, Asia Pacific, Eastern and Western Europe, Eastern and Western North America, Oceania, and South America, representing roughly 90 cities. A "global" category reflected all data centers worldwide, where outliers and retries had an outsized effect.

URL purge was isolated as a distinct category because it identifies a single target asset, even if that asset is stored across multiple locations. All other purge types -- everything, tag, prefix, hostname -- were grouped together because the amount of data they remove is highly variable, ranging from a single tag to an entire website.

Targeting the Speed of Light

Those numbers show worst-case global purge completion in under seven seconds, with most hovering under one second. Cloudflare's explicit goal is to push this closer to the theoretical speed of light for its network size, which it estimates at 200ms. Interestingly, the company notes that LEO satellite networks with laser links could potentially beat fiber-optic latency in some cases due to the straightness of their paths.

Coreless Purge: A Decentralized Rearchitecture

The path to faster invalidation runs through a complete redesign of the purge service, which has not received a systematic overhaul in over a decade. The current architecture leans on central core services for authorization, authentication, and distribution. These components have become a bottleneck as the network expands.

Cloudflare's planned replacement, called coreless purge, eliminates the reliance on core data centers. Instead, all the functionality moves to every data center via a proof of concept built on Cloudflare Workers and Durable Objects. Durable Objects act as a horizontally scalable queuing mechanism, allowing the system to scale by adding more objects as needed and enabling quicker regional fanouts.

The new flow is simplified in practice:

• The purge request is routed to the nearest data center, where a Worker performs core functions like authorization and filtering.
• The Worker passes the request to a Durable Object in that data center, which queues it.
• The Durable Object broadcasts the request to every data center.
• Another Worker at each destination passes the request to the service that executes the invalidation.

Cloudflare frames this as part of an ongoing series, promising incremental updates as it benchmarks the new system against the performance data gathered from the current pipeline.