Anatomy of a BGP zombie

The term "BGP zombie" describes a route that has effectively died — its prefix withdrawal was issued or should have occurred — but the route lingers in the Default-Free Zone (DFZ), the set of internet routers that operate without a default route. The cause is typically not a single failure but a combination of slow route processing, software quirks, or missed withdrawals. Whatever the origin, the practical effect is that packets destined for that prefix can end up trapped in loops or taking suboptimal paths for as long as the zombie persists.

To grasp why routes become stuck, it helps to understand path hunting. When a more-specific BGP prefix is withdrawn, every router that previously used it must fall back to the next best match, usually a less-specific prefix. That fallback is not instantaneous; routers must search for the best path again based on longest prefix matching (LPM) and BGP attributes. The process is further stretched by the Minimum Route Advertisement Interval (MRAI). RFC4271 recommends a 30-second MRAI for eBGP updates, which reduces chatter and oscillation but deliberately slows down convergence.

Consider a simple case where 2001:db8::/48 is actively used for traffic toward a customer, routing through AS13335. The customer also has a backup path via another ISP, AS64510, but that path remains unused because the /48 is a longer prefix match than the 2001:db8::/32 that AS64510 would otherwise use.

When the customer signals AS13335 to withdraw the /48 — for example, after a DDoS attack subsides — AS13335 sends a withdrawal to its neighbors. The problem is that not all routers process that withdrawal at the same speed. If AS64510 receives it faster than the ISP's own next hop, traffic can arrive at AS13335 for a prefix it no longer advertises, and the search for a valid path begins. On the next cycle, the routers that were still holding the old /48 should converge on the /32, but only if they process the withdrawal at all. A router that is slow, buggy, or overloaded can keep the old path alive, effectively becoming a zombie.

In practice, two cycles are rarely enough. Given how well-connected a network like AS13335 is to Tier-1 providers and peers globally, convergence can take many more rounds of path hunting. That extended hunting is often the first warning sign that a more-specific withdrawal is about to leave behind an undead route.

Arise, zombie

Zombies become a practical concern for customers using Bring-Your-Own-IP (BYOIP) in "on-demand" mode, where prefixes are announced only when needed — say, during a DDoS attack — and withdrawn afterward. For these customers, announcement and withdrawal cycles are frequent by design, making them more exposed to the effects of path hunting and stuck routes.

In testing with spare IPv4 and IPv6 address blocks, the pattern is consistent. Announcing a more-specific prefix and then withdrawing it produces the same behavior: some networks hold onto the old route well after the withdrawal should have propagated.

Two common zombie variants emerge from these tests.

Variant A: Between providers. Zombies frequently appear at the interconnection points between upstream ISPs. A single router in one provider's network that is slower to process the withdrawal can cause a loop, where traffic destined for the withdrawn prefix bounces between two or more providers that each believe the other still has the route.

Variant B: Inside a single network. Zombies also live entirely within one network. When a route is withdrawn from Cloudflare's infrastructure, every internal router must process the update independently. If one router lags, external partners may still send traffic toward it, even though no host is reachable behind it. The result is an internal loop that persists until that router re-converges.

The observable lifetime of these zombies varies. In one test, most major networks re-converged around the less-specific prefix after roughly six minutes. That sounds long, but it is on the shorter end — some zombies have caused reachability problems for more than ten minutes. Even after 30 minutes, remnants were still visible in route-views public collectors, confirming that the problem is real and not merely a startup artifact.

Faster convergence, shorter life

One interesting observation is that convergence delay is worse on IPv4 than on IPv6. In the same tests, the longest IPv6 impact observed in Tier-1 networks was just over four minutes, while IPv4 took longer. A likely explanation is the sheer size difference: the global IPv4 table carries far more prefixes than the IPv6 table, and BGP speakers handle them separately, so each more-specific withdrawal on IPv4 triggers a larger amount of path hunting.

The high level of interconnectivity of the announcing network also plays a role. A network that peers with a large portion of the internet has more places where a route can get stuck, making a long-lived zombie more probable. That suggests a workaround. If, instead of announcing a more-specific prefix persistently from the highly-connected network, you announce it only from a less-connected local ISP — while the less-specific prefix remains advertised from the highly-connected network — the withdrawal propagation is likely faster.

Testing this inversion of roles confirmed the intuition. The zombie still appears, but it lives for only about 20 seconds in the Tier-1 network layer. That is still an impactful 20 seconds of potential packet loss or delay, and the exact duration would vary depending on the local ISP's own network structure. But the difference between a 20-second hiccup and a multi-minute outage is meaningful for any network operator relying on on-demand DDoS protection.

A useful takeaway from these tests is that new route announcements behave differently from withdrawals. In both test scenarios, the initial announcement did not create a zombie or cause path hunting. Zombies emerged only when a previously known more-specific prefix was fully withdrawn — newly announced routes propagate cleanly, but the removal of a known route is where the undead find their foothold.

Mitigating stuck routes

The analysis suggests that withdrawing a more-specific prefix can leave zombies active for longer. Cloudflare is therefore testing changes that reduce the impact of stuck routes on customers using its on-demand BGP service.

For traffic that does reach Cloudflare despite a stuck route, the plan is to introduce internal forwarding adjustments that allow for a more graceful withdrawal, even when routes incorrectly point to Cloudflare. This behavior would resemble the BGP no-export community’s functionality from Cloudflare’s BGP servers. Even if external parties send traffic due to stuck routing, Cloudflare would still be able to deliver it to far-end customers via a tunneled connection or a Cloudflare Network Interconnect (CNI). The goal is to make draining traffic more graceful by default, with results to be reported after implementation.

For traffic that never reaches Cloudflare and instead loops between providers, a different approach is needed. Given that fallback from more-specific to less-specific prefixes is more prone to zombie outbreaks, customers are advised to use a multi-step draining process when removing an on-demand prefix to avoid route loops or blackholes. The recommended procedure is:

  1. The customer is already announcing a prefix such as 198.18.0.0/24 from Cloudflare.
  2. The customer begins natively announcing 198.18.0.0/24 — the same-length prefix they are advertising via Cloudflare — from their own network to the ISPs they want to fail over to.
  3. After a few minutes, the customer withdraws the 198.18.0.0/24 announcement from Cloudflare.

This provides a clean cutover: zombies are avoided because the same-length prefix remains in the global routing table, and excessive path hunting is avoided because routers can fall back to the persistent native announcement instead of aggressively seeking a missing more-specific route.

BLOG-3059 8

Looking ahead

Cloudflare plans to continue refining how it measures BGP zombies and expects to share further findings. Other efforts in the community, such as BGP stuck route observatories, are also producing useful measurement data.

On the software side, routing vendors should implement RFC9687, the BGP SendHoldTimer, to reduce the duration of zombie routes. The SendHoldTimer lets a local router detect when a far-end router has stopped processing BGP messages unexpectedly, lowering the chance that zombies remain stuck for long periods.

Operators who rely on more-specific prefix announcements for failover or traffic engineering should keep the observations about excessive path hunting in mind: those routes can stay stuck longer before full BGP convergence. Anyone interested in working on problems like BGP zombies is encouraged to explore Cloudflare careers or internship opportunities.