One Network Layer to Bind AI Clusters Together

Meta has shared new details on the backend aggregation (BAG) layer underpinning its gigawatt-scale AI cluster, Prometheus. The cluster, which will span multiple data center buildings in a single region and interconnect tens of thousands of GPUs, relies on BAG as the centralized Ethernet-based super spine layer that joins spine fabrics across data centers and regions. The BAG layer functions as the aggregation point between regional networks and Meta's backbone, with inter-BAG capacities reaching the petabit range (16–48 Pbps per region pair).

The design bridges two separate network fabric technologies: Disaggregated Schedule Fabric (DSF) and Non-Scheduled Fabric (NSF). Modular hardware, advanced routing, and resilient topologies are the core mechanisms that make the architecture work at scale.

We use backend aggregation (BAG) to interconnect data center regions to share compute and other resources into large clusters.

Distributing the Aggregation Layer

BAG layers are not monolithic. They are distributed regionally to serve subsets of Layer 2 (L2) fabrics, with placement governed by distance, buffer, and latency limits. Inter-BAG connectivity uses one of two topologies depending on site size and fiber availability:

  • Planar topology makes one-to-one connections between BAG switches following the plane, simplifying management but concentrating potential failure domains.
  • Spread connection topology spreads links across multiple BAG switches and planes, enhancing path diversity and resilience.
An example of an inter-BAG network topology.

Connecting Downstream to L2 Fabrics

On the downstream side, BAG layers connect to L2 fabrics built with either DSF or NSF. In a DSF example, L2 zones across five data center buildings attach to the BAG layer through a dedicated backend edge pod in each building.

A BAG inter-building connection for DSF fabric across five data centers.

With NSF, each BAG plane connects to matching Spine Training Switches (STSWs) from all spine planes, producing an effective oversubscription of 4.98:1.

A BAG inter-building connection for NSF fabric.

Oversubscription ratios are tuned to balance scale against performance. Typical L2-to-BAG oversubscription sits around 4.5:1, while BAG-to-BAG ratios vary by regional needs and link capacity.

Hardware, Routing, and Resilience

Meta's BAG uses modular chassis with Jericho3 (J3) ASIC line cards, each offering up to 432x800G ports. The central hub runs a larger chassis to accommodate numerous spokes and long-distance links with varied cable lengths, optimizing buffer usage. Routing is handled via eBGP with link bandwidth attributes, enabling Unequal Cost Multipath (UCMP) for load balancing and robust failure handling. MACsec secures BAG-to-BAG links.

Resilience is engineered at multiple levels. Port striping, IP addressing, and failure domain analysis—covering BAG, data hall, and power distribution—are all explicitly mapped. To prevent blackholing, the team drains affected BAG planes and uses conditional route aggregation.

Long cable runs also factor into the design. The distributed architecture keeps L2-to-edge distances short, which is key for shallow-buffer NSF switches. Longer BAG-to-BAG links instead rely on deep-buffer switches for the BAG role, providing headroom to support lossless congestion control protocols such as PFC.

With the hardware and topology choices set, BAG is positioned as the connective tissue for Prometheus and the broader scaling roadmap for Meta's AI infrastructure, intended to carry the network through future generations of demand.