BGP as the Routing Backbone for Hyperscale Data Centers
Border Gateway Protocol (BGP) was built to connect autonomous systems across the global internet, but its scalability and granular policy controls have made it an increasingly attractive option inside the data center. A new study from Facebook engineers details how the company adapted BGP for its own large-scale fabrics, going beyond a simple protocol deployment to a full routing architecture codesigned with topology, switch software, and operational tooling.
The design goal was straightforward: build networks quickly, offer high availability, and keep the system scalable even as failures occur—an inevitability at this scale. The result is a routing foundation that minimizes the blast radius of any single failure while supporting continuous service changes.
Design Principles for Scale
Facebook’s approach reuses a uniform autonomous system (AS) numbering scheme across multiple data center fabrics, which simplifies ASN management considerably. Route summarization is applied hierarchically at every level of the topology, keeping hardware forwarding tables small even as the network grows. This baseline connectivity configuration sits on top of the existing fabric topology and provides a consistent starting point for all routing behavior.
Policy configuration is not an afterthought; it is tightly integrated with the baseline connectivity. Route propagation scopes define where routes can travel, while predefined backup paths ensure traffic can be rerouted quickly when a device fails. The same policies allow operators to gracefully drain traffic from problematic hardware for maintenance, and they guarantee that services stay reachable when instances are added, removed, or migrated.
An In-House BGP Agent
To keep pace with new features, optimizations, and bug fixes, the team needed a switch-level BGP agent that could be updated frequently without disrupting production. Rather than adopt a full-featured third-party implementation, Facebook built its own in-house BGP agent. The codebase is intentionally simple, implementing only the protocol features required in the data center—without deviating from the BGP specification itself.
Shipping updates to that agent at high velocity required a purpose-built testing and deployment framework. The pipeline combines unit testing, emulation, and canary testing, then rolls changes out through a multi-phase deployment process designed to minimize impact on live traffic.
Why This Matters for Data Center Routing
BGP’s move into the data center is not new, but data center requirements differ sharply from those of the public internet. Operators need fast convergence, strict isolation between tenants and services, and the ability to treat routing software like any other deployable component. Facebook’s design shows that BGP can meet those demands when the protocol is paired with disciplined topology design and robust operational processes.
The company has run this routing design across its entire data center fleet for over two years, and the operational experience has directly shaped its ongoing routing work. The key takeaway for other large-scale operators: BGP is a viable, effective option for data center routing when the implementation and surrounding tooling receive the same engineering rigor as the network hardware itself. The full details were presented at NSDI 2021 and published in the research paper “Running BGP in Data Centers at Scale.”
