Preventing the stampede: Request collapsing in the Vercel CDN

Why concurrent misses are expensive

Incremental Static Regeneration (ISR) lets Next.js apps serve static pages that refresh on a schedule. But every time a page's cache expires, there's a window where the old content is gone and the new content hasn't been generated yet. If a page is popular, many users can hit that window at the same time.

Without coordination, each of those requests sees a cache miss and triggers its own function invocation, regenerating the same page from scratch. That parallel work burns compute and can overwhelm the backend. The Vercel CDN now mitigates this with a request collapsing mechanism: when multiple requests hit the same uncached path, only one request per region invokes the function. The rest wait briefly and then receive the cached response.

Cacheability is inferred automatically from framework-defined infrastructure. When you deploy, Vercel analyzes which routes use ISR, static generation, or dynamic rendering, and distributes that metadata to every CDN region. No manual configuration is required.

How the multi-layer cache normally serves requests

The CDN starts with the ISR cache, which sits alongside functions and stores the results of static regeneration. That cache is the source of truth: content replicates from it to each region's Vercel cache. When a request arrives, the nearest region serves it, handled by one node (a server instance) within that region. Nodes scale with traffic, and each node runs multiple workers, each maintaining a small in-memory cache for hot content.

In the normal flow, content is served immediately from a node's in-memory cache when it's hot. If it's not there, the request checks the regional CDN cache, which pulls from the ISR cache. If the ISR cache still has a valid page, it returns without invoking a function.

The trouble starts when every layer misses, which happens with an expired page or a brand-new route. Concurrent requests each see an empty cache, and each one triggers a regeneration.

Synchronizing those misses is the core of request collapsing. Instead of dozens of invocations for the same path, the system coordinates: only one request per region runs the function, and all the others wait for that single result.

When everything works, the benefits are clear: one invocation regenerates the page, every other request gets served from cache once it completes, and cache coherency improves because only one invocation writes the response at a time.

When collapsing is safe

Request collapsing only applies when a request is known to produce a cacheable response. An ISR page that renders the same content for all users qualifies. Dynamic API routes returning user-specific data do not, and pages with random content or timestamps should not be collapsed. Since the route's behavior is already known from the framework metadata, the CDN can decide which path is safe to collapse before any request reaches a function.

Distributed locking at two levels

Collapsing is implemented as a two-level distributed lock. At the node level, each CDN node keeps an in-memory lock per path. When multiple requests for the same uncached path land on one node, that lock ensures exactly one proceeds; the rest wait until the cache is populated. At the regional level, each region enforces a lock across all of its nodes. After acquiring the node lock, a request must also acquire the regional lock. Holding both locks at once is what guarantees only one function invocation per region per path.

This hierarchy keeps the coordination itself scalable. Without node-level grouping, hundreds of concurrent requests would compete for the regional lock simultaneously, turning lock coordination into a thundering herd bottleneck. With node locks, the number of waiters at the regional level stays proportional to the number of nodes in a region, not the total request count. Within a node, waiters scale only with the requests hitting that specific instance.

function createDistributedLock(cacheKey) {
  const nodeLock = createNodeLock(cacheKey);
  const regionalLock = createRegionalLock(cacheKey);
  return combineLocks([nodeLock, regionalLock]);
}
async function respond(request) {
  const cacheKey = getCacheKey(request);
  const cachedResponse = await cache.get(cacheKey);
  if (cachedResponse) return cachedResponse;
  const lock = createDistributedLock(cacheKey);
  await lock.lock();
  const response = await invokeFunction(request);
  lock.unlock();
  return response;
}

Double-checked locking for correctness

Locks alone don't collapse requests. If every waiter invoked the function after acquiring the lock, work would still be duplicated. The CDN therefore applies the classic double-checked locking pattern, checking the cache twice around lock acquisition.

1. First check: On arrival, if the cache has content, return it immediately without taking a lock.
2. Acquire lock: If the cache is empty, acquire both the node and regional locks.
3. Second check: After acquiring the lock, check the cache again. While this request waited, another may have completed a regeneration and populated the cache. If so, skip the work and return the cached value.
4. Regeneration: Only if the cache is still empty does the request invoke the function, set the cache, and release the lock.

async function respond(request) {
  const cacheKey = getCacheKey(request);
  const cachedResponse = await cache.get(cacheKey);
  if (cachedResponse) return cachedResponse;
  const lock = createDistributedLock(cacheKey);
  await lock.lock();
  let cachedResponse = await cache.get(cacheKey);
  if (cachedResponse) return cachedResponse;
  const functionResponse = await invokeFunction(request);
  // set cache in background so we can return response immediately
  (async () => {
	  await cache.set(cacheKey, functionResponse);
	  lock.unlock();
  })()
  return functionResponse;
}

Cache writes are asynchronous: the function's response is sent to the user immediately, without waiting for the cache-set operation to finish, which keeps time to first byte low. The lock is released as soon as the cache is populated, so waiting requests can proceed quickly.

Failure modes

Function errors are handled cleanly. If an invocation throws, nothing is cached. The second cache check still finds nothing, so the next lock holder simply retries regeneration. Collapsing doesn't help in that case (there is no valid response to share), but errors never poison the cache.

Timeouts are the more dangerous failure. A slow lock holder could leave every waiting request stuck indefinitely. To prevent that, locks carry explicit timeouts. If a request can't acquire a lock within a fixed window, it abandons the wait and invokes the function itself, a technique called hedging. Slow regenerations never block an entire route's traffic; the worst case is a return to multiple invocations.

function createDistributedLock(cacheKey) {
  const nodeLock = createNodeLock(cacheKey, { timeout: 3000 });
  const regionalLock = createRegionalLock(cacheKey, { timeout: 3000 });
  return combineLocks([nodeLock, regionalLock]);
}

With the configuration shown above, each request waits at most three seconds for both the node and regional locks before proceeding independently. That balances the common-case benefits of collapsing against resilience to slow or unstable functions.

Impact in production

The number of function calls elided by request collapsing varies significantly over time. One production sample shows the collapse rate jumping from 30 to 120 requests per second during a short window.

Overall, the Vercel CDN collapses more than 3 million requests per day on cache miss, on top of 90 million collapsed requests from background revalidations. The feature is enabled for all projects on Vercel, so any ISR deployment benefits automatically.