How Waiting Room makes queueing decisions on Cloudflare's highly distributed network

How Waiting Room decides who gets in

Waiting Room’s core decision — send a user to the origin or hold them in a queue — happens at the edge, in every Cloudflare data center that receives a request. A customer’s traffic can arrive at any of the hundreds of locations in Cloudflare’s network, and each is optimized for the lowest possible latency. That means the request doesn’t wait for a round trip to a central coordinator. Instead, each data center relies on its own snapshot of the waiting room’s state, which is continuously updated in the background via a pipeline built on Durable Objects. That pipeline propagates changes in global traffic patterns within seconds, so each location has a reasonably current view without adding per-request latency.

The tricky part is balancing two goals: don’t overload the origin, but don’t queue users prematurely based on stale regional data. A site might be popular in one geography at a given time, and a naive global algorithm could shut out users from a less-active region when the state has already changed. The current algorithm is the result of iterating on this challenge, reducing false positives while still respecting the customer’s configured limits.

The traffic limits that drive queueing

When a customer sets up a waiting room, they define two thresholds: total active users and new users per minute. The first is a cap on simultaneous users on the protected pages; the second limits the rate of new arrivals. The session duration setting determines how long a user is counted as active after their last request to a covered page.

Consider a real customer with both limits set to 200. Their traffic pattern over two days shows queuing began when total active users hit the 200 mark at 11:45. Once active users dropped around 12:30, the queue drained. The same cycle repeated at 15:00 the same day.

Users who are admitted receive an encrypted cookie — essentially a ticket. It contains a bucketId identifying their group, along with acceptedAt and lastCheckInTime timestamps. On each subsequent request, the waiting room worker checks whether the cookie is still valid against the configured session duration. Valid cookies allow continued browsing; expired ones are treated as new entries, and if queueing is active, those users join the back of the line.

How local slots are calculated

The waiting room state is not a single global number. Each data center sees its own view, which includes the current active user count, the distribution of recent traffic, and how many workers are active — both locally and globally. For example, a state snapshot at a given moment might show 50 active users against a 200-user limit, leaving 150 slots. If those 50 users came from San Jose (20) and London (30), the state at San Jose reflects that history.

When a new request hits a data center, the worker looks at the traffic distribution from the previous minute. Suppose San Jose had 20 of 200 users that minute — 10% of the load. The worker allocates 10% of the available 150 slots to San Jose, or 15 users. That pool is then divided evenly among the workers active in that data center. With three workers, each can admit up to five users in that minute.

Not every data center has a local allocation. A request arriving in Delhi, for instance, might see zero slots assigned to that region in the traffic history. In cases where the global state is far from the limit, the worker falls back to reserved Anywhere slots — a shared pool for regions without a specific allocation. In the example, 75% of the remaining 150 slots (113) are available as Anywhere capacity. These are split among all active workers worldwide; with 10 workers, each gets roughly 11 admits per minute.

Combining the two mechanisms, a worker in San Jose could admit up to 16 users that minute: five from its local allocation plus its share of the Anywhere pool. New users beyond that are queued. This approach lets the waiting room react quickly to regional traffic patterns, avoid queueing based on stale state, and still enforce the customer’s overall limits with minimal added latency per request.

Why worker-level decisions add up to premature queueing

Because queueing decisions happen per worker, the number of new users arriving at each worker can vary widely — even within a single data center. Consider a San Jose data center with ten slots available and two workers running there. If seven users arrive at worker1 and one at worker2, worker1 will admit five users and queue two (since its allocation is only five), while worker2 lets its single user through. Total admitted: six. Total queued: two — despite there being capacity for ten users from that data center and only eight total arrivals.

This is the cost of dividing slots evenly across workers: queueing can begin well before a waiting room's configured traffic limits are reached, typically within 20–30% of those limits. The trade-off is intentional — it keeps waiting rooms operating safely — but it is not ideal. The team has since refined the allocation algorithm to reduce how often queueing happens outside that 20–30% window, aiming to queue as close to the limits as possible while still being ready for sudden spikes.

The primary advantage of worker-level decision-making is latency. Because the data needed to admit or queue a user lives entirely within the local data center, no request needs to leave the data center to resolve a waiting room decision. With no additional round-trips, customers can leave a waiting room enabled at all times without imposing extra latency on their users.

That latency benefit comes with a safety requirement. Waiting Room's core priority is keeping customer sites online even during unexpected traffic surges. If a spike occurs at one data center — say San Jose — the local state there can lag by a few seconds before it propagates to other locations like Delhi. Splitting slots across workers ensures that acting on slightly stale data does not cause a meaningful overshoot of the overall limit. If activeUsers reads 26 in San Jose and 100 in another data center where a spike is underway, sending extra users from Delhi cannot overshoot the global ceiling by much, since Delhi only controls a fractional share of the total capacity. Queueing before limits are reached is therefore a deliberate protective measure, not a defect.

Overprovisioning slots when headroom is plentiful

The first improvement targets the rare case where users get queued even though the waiting room is far from its traffic limits. In that situation, utilization is low — meaning there is ample headroom between current traffic and the configured maximum. The fix was to allocate more generous slot budgets per worker when utilization is low, and tighten them as utilization climbs.

Returning to the two-worker example: at a low utilization of 10%, the allocated slots per worker (eight) are much closer to the data center's total slotsAvailable of ten. With that wider allocation, all eight arriving users are admitted, and nobody queues despite the uneven distribution across workers.

The relationship between slack and allocation is visible in the chart below: at 10% utilization, each worker can draw nearly the full data center capacity. As utilization approaches 100%, the per-worker allocation converges toward the data center's total slots divided by the number of workers in that data center.

The math behind adaptive slot allocation

To understand how the adaptive allocation is computed, consider traffic arriving at the Delhi data center, where `activeUsers` is 50 out of a 200-user limit — roughly 25% utilization.

{
  "activeUsers": 50,
  "globalWorkersActive": 10,
  "dataCenterWorkersActive": 1,
  "trafficHistory": {
    "Mon, 11 Sep 2023 11:44:00 GMT": {
       San Jose: 20/200, // 10%
       London: 30/200, // 15%
       Anywhere: 150/200 // 75%
    }
  }
}

The mechanism introduces a parameter called workerMultiplier, which scales based on utilization. At lower utilization the multiplier is smaller; it approaches one as utilization nears maximum. The value is derived from a configurable exponent, curveFactor, which controls how aggressively extra budget is distributed when total worker count is low. The relationship follows the same logic as comparing y = x to y = x^2 between 0 and 1: the higher the exponent, the slower the multiplier grows relative to utilization for values below 1.

workerMultiplier = (utilization)^curveFactor
adaptedWorkerCount = actualWorkerCount * workerMultiplier

Concretely, utilization is how far the waiting room is from its limits, and curveFactor is the exponent that sets the allocation curve's aggressiveness. When curveFactor equals 1, workerMultiplier equals utilization exactly.

Applying this to the Delhi example: utilization is 25%, so the remaining 75% of 150 slots (~113) represents the "anywhere" pool available to all active workers globally. If globalWorkersActive is 10, the effective worker count is not 10 but globalWorkersActive * workerMultiplier. With curveFactor of 1 and utilization of 0.25, the effective worker count becomes 10 * 0.25 = 2.5.

Each active worker can then admit up to 113 / 2.5 — approximately 45 users. The first 45 users arriving at a worker during the minute Mon, 11 Sep 2023 11:45:00 GMT are admitted; the rest wait in the queue. The result is that each worker gets substantially more slots when the waiting room is far from its limit. The catch, however, is that summing the slots across all workers now creates a real risk of surpassing the global limit if many workers admit users simultaneously.

Overprovisioning risk and the shift to shared state

Giving workers more slots at lower utilization reduces unnecessary queueing, but it introduces a new failure mode: a uniform spike from a low-utilization baseline could push far more users to the origin than intended. In a ten-slot data center at 10% utilization, each of two workers could receive eight slots. If eight users hit one worker and seven hit the other, fifteen users would be sent to the origin — well beyond the data center's maximum of ten.

This is not just a theoretical concern. Across the variety of customer traffic profiles, this scenario did occur. Spikes originating from low utilization levels could overshoot the global limits because the allocation was too generous at precisely the utilization levels where overshoot risk was highest. A safer method was needed — one that kept queueing close to limits while reducing the chance of exceeding them.

A deeper problem also surfaced in an underlying assumption: that the number of workers in a data center correlates with the traffic the data center receives. In practice, that is not true for all customers. Even where worker count does correlate with traffic volume, the volume of new users — the population that slots actually govern — often does not. The traffic in a data center is a mix of users already on the site and new arrivals attempting to enter.

Rather than relying on worker counts as a proxy for demand, the next iteration removes worker counts from the decision entirely. Instead, workers within a data center communicate with one another through a new service based on a durable object counter, ensuring slot accounting reflects the data center's actual real-time admission state.

A Shared Counter Instead of Per-Worker Quotas

The first approach of dividing available slots evenly across workers in a data center led to overprovisioning at the worker level when an uneven number of new users arrived at different workers. If a data center has 10 slots and 3 workers, dividing the slots means each worker gets fewer than 4 slots — and a worker that receives five new users quickly will queue some of them even though the data center hasn't reached its limit. The opposite problem occurs when workers are given more slots at low utilization: a sudden spike after a quiet period can overshoot the customer's configured limit.

The solution is to remove worker counts from the division entirely. Waiting Room uses Data Center Counters — small durable object instances that track how many users a set of workers in a given data center have let through. Workers no longer store the count locally. Instead, each new user request triggers a call to the counter to fetch the current value. The counter increments after each fetch, so every worker sees a unique sequence number.

In the example below, the counter returns 9 for the first new request, meaning the worker has a slot available if the data center has 10 slots. A second worker that receives a new user just after gets a value of 10, and that user is queued because the data center's allocation is exhausted.

This design is essentially a shared ticket counter. Workers don't communicate directly with one another; they synchronize through the counter. If one worker receives seven new users and another receives just one, all eight get through as long as the counter value stays below the data center's slotsAvailable. When the counter value equals the limit, no extra users are admitted. If fifteen users show up at the workers and the data center has ten slots, ten reach the website and five are queued — exactly what the configured limit requires.

This also eliminates the overprovisioning risk at lower utilization, since workers no longer need pre-allocated quotas. Consider a customer state that reserves a portion of slots for a specific data center:

{  
  "activeUsers": 50,
  "globalWorkersActive": 10,
  "dataCenterWorkersActive": 3,
  "trafficHistory": {
    "Mon, 11 Sep 2023 11:44:00 GMT": {
       San Jose: 20/200, // 10%
       London: 30/200, // 15%
       Anywhere: 150/200 // 75%
    }
  }
}

At time Mon, 11 Sep 2023 11:45:54 GMT, requests arrive at San Jose. A data center with 150 slots and a 10% reserve can admit 15 users. The counter returns an incremented value for each new request. The first 15 users get values below 15, so they are admitted using the San Jose allocation. Once the counter returns 15 or higher, the worker knows the local allocation is consumed and checks the Anywhere pool — slots not reserved for any particular location. Anywhere slots comprise 75% of the remaining 150 slots (113) and are tracked in a durable object that workers from all data centers can query. Even if 128 users (113 Anywhere plus 15 San Jose) all arrive at the same worker, none are queued prematurely.

The Latency Trade-off of Counters

The original Waiting Room design kept entry decisions entirely in the worker — no extra service calls while a request is in flight. Adding counters introduces a coordination point, which means a call to a durable object. For data center counters, that object lives in the same data center, so the added latency is typically under 10 ms. Requests for Anywhere slots may travel across oceans to a farther durable object, pushing latency to 60 or 70 ms in those cases. The 95th percentile figures are higher because of those long-distance calls.

The extra latency applies only to new users. Once a user has a proof-of-entry cookie from the origin, workers admit them directly without consulting a counter. The trade-off is acceptable because it reduces the number of users who get queued before reaching the customer's limit — a worse experience than a few extra milliseconds on the first request.

Counters are deliberately simple: they do nothing but count. That keeps their memory and CPU footprint minimal. Because many counters run around the world, each handling coordination for a subset of workers, the synchronization load is distributed broadly enough that counters remain viable under production traffic.

Why This Approach Sticks

Waiting Room runs on every server in Cloudflare's network, spanning more than 300 cities in over 100 countries. The goal is to decide — quickly and at the right time — whether each new user goes to the website or the queue. Queuing too early violates the customer's configured limits; queuing too late risks overshooting them.

Dividing slots evenly among workers respected limits but occasionally queued users early. Granting workers more slots at low utilization avoided early queuing but introduced overshoot risk when traffic spiked after a quiet period. Counters avoid the division problem altogether. Workers don't need to know how many siblings they have or how traffic is distributed. They just ask the counter, which reflects the true data center-wide state.

The cost is a little latency on each new-user request. In practice, that is negligible compared to the alternative of sending legitimate users to a queue before they should be there.