Cloudflare’s Traffic Manager: Keeping the Network Fast When Things Break

Cloudflare operates in over 300 cities worldwide, with each location designed to handle user traffic for every product it offers. The goal is always to serve each user from the closest data center. In an ideal world, that's simple. But hardware fails, maintenance windows close, and power outages happen. When a local data center can't handle its load, traffic needs to move elsewhere—and doing that manually doesn't scale.

Originally, re-routing traffic was a manual task for network engineers. When a data center failed, user requests would start failing with 499 or 500 errors. That would page an engineer, who would then remove some Anycast routes for that location. Because Anycast sends users to the nearest advertised location, withdrawing the prefix would push traffic to a different data center. When the problem was resolved, the routes were put back and traffic returned naturally. But with hundreds of locations and constant issues, this process was slow, error-prone, and a poor experience for customers.

Automation with a View of Consequences

To fix this, Cloudflare built Traffic Manager: an automated service that detects when users cannot reach a data center and withdraws routes until the problem clears. But moving traffic isn't as simple as removing a prefix. The engineers needed to know exactly where traffic would go—and make sure that destination could absorb it. That's the job of Traffic Predictor.

Because Cloudflare can't control which proportion of traffic each data center receives, it had to measure the network's behavior empirically. Traffic Predictor runs continuous tests to map failover paths. For a given data center—say, Christchurch, New Zealand—it builds a list of IPs that normally connect there, tests them with ping requests to a special Anycast range, then withdraws that route and runs the test again. The responses now arrive at other data centers, showing exactly where traffic would go if Christchurch went offline.

These tests scale across all data centers, and even account for cascading failures. If Christchurch fails, traffic shifts to Auckland—and Traffic Predictor also models what happens if Auckland fails under that extra load. The result is a set of failover policies for multiple layers of regional failures. For Christchurch, the model predicts 99.8% of traffic would move to Auckland, which can handle the load. This precomputation lets Traffic Manager avoid a "thundering herd" scenario, where one failed data center creates a spike that kills its neighbor.

From Broad Prefixes to Fine-Grained Control

The original approach of withdrawing prefixes—each typically a /24 subnet—was heavy-handed. A /24 represents thousands of IP addresses, and with Cloudflare mapping many customers to each IP, removing a single prefix can move far more traffic than necessary. This "stranding capacity" makes a data center unable to serve users it could actually handle.

To move only what's needed, Cloudflare built Plurimog, an extension of its Layer 4 load balancer Unimog. Unimog works within a data center: every machine knows—through constant inter-service communication—whether it can take a request, and forwards to a machine that can when it can't. Plurimog does the same between data centers. If a request arrives in Philadelphia and that location can't serve it, Plurimog forwards the individual TCP or UDP request to a data center that can, like Ashburn. Because it operates per-connection, Plurimog can shift precise percentages of traffic, not just entire prefixes.

For larger locations—Multi-Colo Points of Presence (MCPs) with distinct logical subsections of compute—a related tool called Duomog provides the same balancing. It automatically shifts traffic between sections within a data center, keeping an MCP online even if one entire subsection suffers hardware issues. When the subsection recovers, traffic shifts back automatically.

When moving traffic, Traffic Manager works through these steps:

  • It identifies the proportion of requests that must leave a data center so all requests can be served.
  • It calculates the available space in each potential target data center.
  • It determines how much traffic to move per target, using Plurimog or Duomog to shift whole plans or portions of plans.

Tier selection matters: Traffic Manager moves Free customers first, then Pro, then Business, paying most attention to keeping higher-paying plans running as locally as possible.

What Metric Matters: CPU Time, Not Requests

Moving traffic isn't based on requests per second. Different customers have hugely different resource costs—a request served from cache at low CPU, versus a request running heavy WAF rules at high CPU. Cloudflare measures instead the aggregate CPU time each request consumes, summing it in milliseconds per second across each plan.

CPU time correlates tightly with what customers experience. Cloudflare tracks the cfcheck latency—the time for a request to traverse its front-line servers. When CPU utilization is high, users notice the slowdown. Similarly, when servers fail in a data center, the remaining machines take on more work and CPU utilization rises across the datacenter, warning Traffic Manager that action is needed.

Three thresholds drive each data center:

  • Maximum: the CPU level where performance degrades, triggering active moves out.
  • Target: the level Traffic Manager aims to return to, so users get optimal service.
  • Acceptable: the level below which a data center can receive forwarded traffic from other locations.

The math scales across differently-sized data centers. If a data center runs at 90% CPU, target at 85%, and total CPU time is 18,000 ms/s, then Traffic Manager computes it needs to move 1,000 ms/s of CPU time to hit the target.

Setting Thresholds with Machine Learning

Choosing the right maximum threshold is tricky because requirements vary. For its service-level indicator (SLI), Cloudflare uses the 95th percentile cfcheck latency, and defines a 20ms service-level objective (SLO). If cfcheck latency tops 20ms, customers are unhappy. Since high CPU causes that, the goal of Traffic Manager's thresholds is to start moving traffic before that impact occurs.

A machine learning service handles this automatically. At scheduled intervals, it gathers metrics for each data center, cleans outliers, and fits a model—currently a quadratic curve fit, with a linear regression in testing. The models predict the CPU usage that corresponds to the 20ms SLO, and that point becomes the maximum threshold. This has proven highly accurate. Before deployment in Lisbon, for example, cfcheck latency was constantly spiking with no Traffic Manager actions because the static threshold was set too high. After the dynamic threshold was introduced, cfcheck latency never hit the SLO again.

Choosing Where to Send Traffic

When a data center must offload traffic, Traffic Manager identifies its available CPU time. It then orders destination candidates by latency and, starting with the lowest latency, fills each one up to its threshold before moving on. At the same time, it moves the lowest-priority traffic first—those plans are the most flexible. Destination selection works in the opposite order; the most important traffic gets the nearest available capacity.

Suppose data center A needs to move 1,000 ms/s of CPU time. Its plan usage breaks down as: Free at 500 ms/s, Pro at 400 ms/s, Business at 200 ms/s, Enterprise at 1,000 ms/s. Traffic Manager moves all Free traffic, all Pro, and half of Business, leaving Enterprise alone. Nearby data centers B, C, and D have space. Traffic Manager prioritizes by latency: B gets all the moved Business traffic and half the Pro; C gets the rest of Pro plus some Free; D, being furthest, gets the final share of Free traffic.

Reverting moves is just as important as making them. When a data center recovers capacity below its acceptable threshold, Traffic Manager uses that available CPU time to first pull back traffic it is forwarding out to other locations—reverting active moves always takes priority over accepting new traffic from elsewhere.

Always On, Automatically

The end result is a system that runs around the clock, measuring health metrics and shifting traffic long before users see errors. Looking at a global map, red and yellow dots show data centers under load gracefully forwarding to greener zones. In one example, Toronto, Detroit, New York, and Kansas City were dealing with hardware issues, so they sent requests to Dallas, Chicago, and Ashburn. Once Detroit recovered, it gradually stopped forwarding until traffic returned to normal. End users never saw the incident.

Because Traffic Manager directly oversees load, its integration benefits product-level health too. Browser Isolation is a prime example: when a user connects to start a hosted browser, it first asks Traffic Manager if the local data center has enough capacity. If not, Traffic Manager directs the request to the closest data center that does have room, keeping the user on the best-performing server available.

Humans used to solve these problems—and every fix came at the cost of availability. Traffic Manager handles hardware and network failures automatically, faster than any person could react. It keeps customers online with minimal disruption, powered by constant data and precise, surgical rerouting. For Cloudflare's engineers, that also means a full night of sleep.