Cloudflare’s Egress Traffic No Longer Depends on Per-Server IPs

Cloudflare has long explained how traffic reaches its network: anycast IP ranges, advertised from every data center, pull eyeball connections toward the closest possible location. Far less documented is the opposite path — how Cloudflare servers initiate outbound connections to the rest of the Internet. That egress traffic, which powers origin fetches, WARP, Workers subrequests, and Spectrum, has gone through a quiet architectural shift. Servers no longer need their own unicast IP addresses to talk to the outside world.

Ingress and Egress Operate Under Different Rules

Every Cloudflare server handles two broad categories of traffic. Ingress connections are inbound, initiated by clients to Cloudflare. Egress connections are outbound, initiated by Cloudflare servers to other hosts on the Internet. The distinction matters because the networking designs for the two directions are fundamentally different.

Ingress traffic uses anycast: every data center announces the same IP ranges, and BGP routing sends each client’s packets to the nearest data center by routing metrics. That model breaks in reverse. If a server initiated a connection from an anycast address, the response packets would be routed back to whatever data center is closest to the sender of the response — not necessarily the data center that originated the connection. Return traffic would land in the wrong place.

The traditional fix was straightforward: give each server its own unicast IP address, unique in the world, so return packets always find their way back to the exact machine. Cloudflare ran that way for years, and it worked. But as the product line expanded, the number of egress IPs a single server needed grew far beyond one or two.

Segmentation Drove Up IP Consumption

Initially, egress traffic was almost entirely HTTP fetches to origin servers, which treated Cloudflare as a trusted source. Newer products changed that. WARP, for instance, operates as a forward proxy, sending end-user traffic onward to the Internet without the same level of intermediation as the CDN. Origin servers and third-party services needed a way to distinguish trusted Cloudflare traffic from untrusted WARP user traffic. The conventional solution is IP-based segmentation: different ranges for different traffic categories.

That added one more address per server — an untrusted WARP IP alongside the trusted egress IP. Then geolocation requirements made things worse. WARP users in the UK expect BBC services to work, and the BBC geofences by looking up the country associated with a public IP. To serve users from many countries, Cloudflare needed egress IPs tagged with different country codes.

But anycast ingress means Cloudflare does not control which data center a user lands in. A London data center might receive traffic from Ireland and Saudi Arabia as well as the UK. To give every user a correctly geolocated egress address, servers in London needed WARP egress IPs tagged for many countries, not just one.

The math became unpleasant. Thousands of servers, each needing dozens of egress IPs with different country tags — and IPv4 addresses are not cheap. The problem space exploded.

Why Not Abandon Anycast?

One possible answer was to ditch anycast ingress entirely and route each user to a specific data center or server via DNS. Cloudflare considered the idea and rejected it. Anycast provides concrete advantages:

  • Performance: users route to the closest data center by BGP metrics.
  • Automatic failover: an unavailable data center is bypassed instantly as routes shift.
  • DDoS resilience: attack traffic spreads across many data centers automatically.
  • Uniform software: every server can perform any function, simplifying debugging and scaling.

Port Slicing Instead of Full IPs per Server

The challenge stayed: any server, in any data center, must be able to egress from an IP with any required tag. Two naive designs bracket the problem. The first gives every server every IP it might need — fast and reliable, but prohibitively expensive in IPv4. The second puts each specialized IP on one server and forwards traffic to it — cheap on IP space but worse on performance and reliability.

Cloudflare’s solution splits the difference by changing what gets assigned. Rather than giving each server a full /32, it assigns a /32 per data center and shares it across servers by port range. Each server owns a slice of the available source ports for a given egress IP.

Return packets are routed back to the right machine by Unimog, Cloudflare’s XDP-based L4 load balancer, which inspects the destination port and forwards accordingly. With port slices of 2,048 ports, a single IP can serve 31 servers. Aggressive reuse of ephemeral ports, covered in Cloudflare’s earlier work on long-lived connections, reduces the chance of port exhaustion.

Soft-Unicast: Anycast Routing, Unicast Semantics

Sharing one address across servers inside a data center solves only part of the problem. The public Internet generally cannot route a single /32 — routing granularity typically stops at /24, or 256 addresses. Announcing entire /24s for egress ranges that are only partially used would waste valuable IP space.

Cloudflare’s answer: announce egress ranges as anycast. Unimog was extended to forward packets across Cloudflare’s backbone network to the correct data center when they arrive at the wrong one. Unimog maintains a mapping of which data center currently owns each address, and corrects misrouted return traffic in software.

At the BGP layer, the ranges are anycast. Logically, though, each IP identifies a single data center at any moment, and each IP plus port range identifies a specific server. Cloudflare calls this stack soft-unicast — unicast semantics implemented in software on top of anycast routing.

Operational Gains and Residual Quirks

Soft-unicast delivers concrete benefits:

  • A single /32 egress IP is shared across many servers.
  • A subnet can be spread across many data centers and reshuffled on the fly, allowing full utilization of egress IPv4 space.
  • Similar IPs can be grouped into contiguous ranges, reducing the size of published geofeeds.
  • New egress ranges — including customer IPs for products like Zero Trust — can be onboarded easily.

Most of the time, users still egress from their nearest data center, so performance is not sacrificed. IPs can be allocated or released as demand shifts, so Cloudflare does not need to pay upfront for address space it may not use. Running multiple potential egress locations keeps reliability intact.

The design has side effects. One is philosophical: ask where a particular egress IP physically exists, and there is no clean answer. At the BGP layer the range is everywhere; at any given time the address is logically used from one data center, but that can change on demand.

Another side effect surfaced with third-party CDNs. A UK-tagged WARP user, rerouted to Paris during London maintenance, might trigger a reverse geolocation lookup of the egress IP. Finding the UK tag, a CDN could direct the user to a London server — while the user was actually egressing from Paris. The result was a suboptimal path spanning the UK, France, and back. Cloudflare mitigates this by performing DNS resolutions in the egressing data center, using IPs tagged with the actual data center location rather than the user’s intended geolocation. That fixes most cases, though exceptions remain.

The bigger picture is a shift in how Cloudflare thinks about address ownership. Servers no longer need true unicast IPs. Egress addresses are assigned dynamically, shared by port slices, and movable between data centers. Traditional unicast is no longer the foundation of outbound connectivity.