The case for caching API traffic
APIs have become the standard way for applications to exchange data and trigger actions across the internet. They avoid the overhead of shipping bulky files, transferring only the essential payload needed to render a page or update state. But speed is still bounded by network distance: if the origin server sits far from the user, round-trip time can add noticeable latency to every call.
For web content, serving requests from a nearby data center and caching responses has long been a standard performance technique. Until now, that treatment has rarely extended to API calls. The reasons are understandable: API responses are often dynamic, cache invalidation has to be fast and reliable, dedicated caching tools frequently sit close to the origin rather than close to users, and serving cached user data to the wrong person is an unacceptable risk. As a result, many teams simply assume API caching requires too much effort and analysis to be worth it.
Ricochet for API Gateway aims to change that. Announced for 2024, it brings Cloudflare’s caching infrastructure to API endpoints managed by API Gateway, with no changes required at the origin. Customers configure their API session identifiers and Ricochet identifies repeatable, cacheable requests. The expected outcome: lower average latency and fewer requests reaching origin, which in turn improves response times even for cache misses.
Handling safe and non-safe methods
The mechanics of HTTP make some requests straightforward to cache and others problematic. GET, OPTIONS, and HEAD are “safe” methods, meaning they should not change state at the origin, so a cached response can be served without risk of stale server data. POST, PUT, and DELETE are “non-safe”—they exist to modify resources, and serving them from cache would leave the origin out of sync.
Ricochet applies standard rules for safe methods and then layer in automatic invalidation. When a non-safe method request arrives for a RESTful endpoint that API Gateway manages, the cached entries for that endpoint are invalidated. This handles the common pattern where endpoints remain constant across operations while the method signals the intent. For cases where an update on one endpoint affects data served by another, the short default TTL on API Gateway can be configured to tolerate a brief delay between an origin update and its appearance from cache.
Securing the cache key
Caching authenticated API responses safely comes down to the cache key. Ricochet incorporates the user’s API session identifier as part of the key, ensuring that a given user’s cached responses are only ever served back to that same user. For endpoints that do not require authentication, the request parameters themselves are hashed into the key. That preserves privacy for the request body while still producing a unique, reproducible key for endpoints with static paths and variable inputs. In both cases, the extra key material prevents cross-user data leakage even when the underlying request path is identical.
Caching endpoints that look dynamic
Two categories of API traffic are prime candidates for this treatment, and both are among the use cases Cloudflare cites as customer requests. The first is mixed-authentication APIs where correct data isolation is essential. The second is single endpoints that return varied results based on inputs—RESTful endpoints with query parameters, as well as GraphQL POST requests, which would not traditionally be considered cacheable.
Examples include weather forecasts, flight status and tracking, live sports scores, and collaborative tools with many concurrent users. Even a short cache-control timer can produce meaningful origin offload. A popular public endpoint receiving 1,000 requests per second, or 60,000 requests per minute, could set a cache control of just 1 second. In that scenario, 999 of every 1,000 requests per second would be served from cache, cutting origin traffic to roughly 60 requests per minute.
Measured impact in the field
A profiling exercise on five major airline website APIs reinforces the potential gains. Flight status retrieval across all five was on average about 7x slower than retrieving the airline’s static logo—data that is readily cacheable. Beyond cutting latency for repeat requests, offloading origin traffic helps the requests that do miss cache, since the origin is under less load and can respond faster overall.
Ricochet for API Gateway is scheduled for release in 2024. Cloudflare is maintaining a waitlist for customers who want early access. For API Gateway users, it is designed to require minimal setup—and no origin-side changes—to bring API responses closer to end users.
