KV’s hot path gets a cold look
Latency is the currency of the web: every extra millisecond spent waiting for a response is a user more likely to leave. Cloudflare’s Workers KV is the key-value store backing many of its own services — Pages stores static assets, Access holds authentication credentials, and AI Gateway and Images keep configuration and assets nearby. Because KV sits behind so many products, its response time shapes the latency of nearly every request those services handle.
Cloudflare has now cut KV hot read latency by up to 3x, with every KV operation faster by up to 20 ms. The gains come from two architectural changes: removing a network layer from the request path and adding a regional cache tier in front of the storage backends.
Two layers removed from every KV read
At a high level, Workers KV is just another Worker that makes requests back into Cloudflare’s storage and caching infrastructure. A read from your Worker to KV traversed several hops: first to Cloudflare’s network, then through the Front Line (FL) request-processing service, then to the KV Worker itself. The KV Worker would then do its own cache lookup by routing through FL and Pingora again to reach cache backends.
When they traced these operations, Cloudflare’s engineers found that most of the latency wasn’t in the cache lookups or storage at all — it was the overhead of passing through FL, a service designed to handle requests coming from outside the network. A Worker-to-KV request didn’t need that treatment.
Two optimizations eliminated most of that overhead:
- Direct binding from the Worker to the KV Worker. The KV binding in your Worker now connects straight to the KV Worker inside the same Cloudflare data center. To do this, the Workers host uses a C++ subpipeline that lets bindings establish a direct connection with the appropriate routing config and authorization already loaded, bypassing FL entirely.
- Direct binding from the KV Worker to cache. On the other side, the KV Worker now binds directly to the internal Pingora-based cache service using a Workers-internal binding that lets service Workers reach non-Worker services. The KV Worker sets the proper cache-control headers and connects without leaving Cloudflare’s network.
Together, those two changes remove about 20 ms from every KV operation.
Tiered cache moves misses closer to users
The second part of the latency equation was cache misses. When a key wasn’t found in the local cache, the request had to travel all the way to centralized storage backends, which incur substantially more network latency. Cloudflare reduced how often that happens by fronting the storage with Tiered Cache, using a topology tuned to KV’s access patterns.
With tiered caching, a request that misses the local cache is first checked against a regional tier before falling through to centralized storage. That matters most for workloads with traffic spread across multiple data centers in the same region. The result: about 30% of requests that previously had to hit storage backends are now resolved at the regional tier, where they’re up to 100 ms faster than if they had reached central storage.
What the new architecture costs
The simplified KV path is now:
- The KV binding connects directly to the KV Worker within the same Cloudflare location, saving 10 ms.
- The KV Worker binds directly to the Tiered Cache service, saving another 10 ms.
- Tiered Cache resolves local misses regionally before the request ever reaches a storage backend.
KV Worker wall time, which measures the duration of a key-value retrieval from cache, reflects the direct binding to cache: the 90th percentile of KV Worker invocations now resolves in under 12 ms, down from 22 ms before the change.
The tiering change appears in the read path that wasn’t resolved locally. Reading from tiered cache has cut latency substantially versus central storage backends.
Cloudflare’s own products see the compound effect. Pages, which serves static assets from KV, reported asset load times improving by up to 68%. Workers asset hosting, announced alongside this work, benefits from day one. Queues and Access also saw KV operation latencies drop, with read operations now 2–5x faster — those services depend on KV for configuration and routing data on every request.
== In-memory cache for the hottest keys ==A small set of keys drives a disproportionate share of KV traffic: less than 0.03% of keys account for nearly half of all requests across namespaces. These keys are read thousands of times per second, so their speed has an outsized impact. Almost all of them are under 100 KB, making them candidates for the KV Worker’s in-memory cache — the memory inside the main runtime process of the Worker sandbox.
For those highest-throughput keys, Cloudflare has now made it possible for reads to resolve directly from that in-memory cache, without any network hops beyond the original request. Keys representing over 40% of global Workers KV requests now resolve in under a millisecond. The change is still in internal testing, with a rollout expected in October.
Latency already in the dashboard
Most of these improvements are already live without any customer action. KV-backed sites and Cloudflare services that depend on KV are responding faster now. New KV analytics in the dashboard show latency and cache status per namespace, making the improvements measurable per key space.
