- Meta's OCP 2026 disclosures span three scaling domains: liquid-cooled 102.4T switching, the Flattened Planar Fabric for backend scale-out, and petabit-scale aggregation for multi-site clusters.
- The new hardware generation pairs the IcePack rack with IceCube switch blades; the fabric work introduces FPF, MetaRoCE, and the OSPA initiative co-led with Microsoft and Alibaba.
The 102.4T Generation: IcePack and IceCube
Gigawatt-scale AI clusters demand far more switch bandwidth inside power and cooling envelopes that are already tight. Meta's response is IcePack, its first-generation liquid-cooled network rack, built on OCP's ORV3-HPRv1 rack standard. Each rack holds IceCube switch blades — 102.4-Tbps fabric switches offering 128×800G non-blocking switching capacity.
IceCube ships in two variants, one using Broadcom's Tomahawk6C switch ASIC and one using NVIDIA's Spectrum-6 Switch ASIC. Both run Meta's open-source FBOSS network operating system on top of OCP SAI (Switch Abstraction Interface). The platform doubles bandwidth relative to Meta's 51.2T generation while meeting the thermal density next-generation AI data centers require.
Flattened Planar Fabric
Flattened Planar Fabric (FPF) is Meta's multiplane backend scale-out fabric. Rather than a single-plane uplink architecture, each GPU connects over eight independent 100G links to eight different switches, one per network plane. That topology yields four properties:
- Larger non-blocking GPU clusters than single-plane topologies support, which is the scale Meta's largest training workloads need.
- A flattened topology: a 2-tier Clos fabric in place of the 3-tier Clos used by Meta's NSF fabric, removing a tier of switches and with it network hops, latency, and infrastructure cost.
- NIC-level hardware load balancing. Functions previously handled by switches now start at the NIC, which sprays every packet across all eight planes in hardware.
- A combined control plane spanning switches and hosts, giving the host the ability to steer traffic away from downed planes.
Meta developed FPF inside its own infrastructure with NVIDIA, drawing on NVIDIA Spectrum-X Multiplane innovations to keep performance stable under varied failure conditions and operational lifecycle scenarios. The two companies are also working with joint cloud partners to standardize multiplane fabric implementations based on the FPF learnings.
Scale-Across: BAG and the Regional Backbone
Meta's committed capacity — Prometheus at roughly 1 gigawatt in 2026, with Hyperion scaling toward 5 gigawatts — exceeds what any single region can hold. The backend aggregation (BAG) layer is a centralized, Ethernet-based super-spine that joins many spine-layer fabrics into one mega cluster across buildings, campuses, and regions. Inter-BAG capacity between a pair of regions reaches tens of petabits per second, letting Prometheus operate as a single cluster even though its capacity spans conventional data center buildings, weatherproof outdoor structures, and adjacent colocation space.
At this reach, distance becomes a design constraint. Lossless Ethernet needs buffer headroom proportional to cable length, so across links stretching tens of kilometers, buffer management is a first-order concern; get it wrong and congestion spreads beyond the originating link. To avoid provisioning every link for worst-case distance, FBOSS measures actual cable length in place, timing probe traffic on an uncongested queue to derive round-trip distance and allocating headroom against that measurement.
Load balancing must also account for paths that are degraded rather than down. BAG uses eBGP link-bandwidth signaling to spread traffic in proportion to the capacity actually available on each path, with damping so a single link failure does not trigger routing churn domain-wide. Because inter-fabric latency runs several times intra-fabric latency, collective libraries that select algorithms without network-distance awareness can hit a performance cliff. Meta is addressing this from both directions: adaptive routing improvements to cut the latency penalty, and scheduling that reduces how much collective traffic must cross BAG at all. A failed replica is automatically excluded, and surviving replicas continue synchronizing across BAG.
For regional backbones, Meta has deployed Segment Routing over IPv6 (SRv6) using FBOSS, enabling more flexible traffic engineering and simplified operations as AI clusters interconnect across geographies. The backbone runs on open shallow-buffer switch platforms powered by Cisco SiliconOne, with ongoing work to onboard additional silicon families, all managed under FBOSS using OCP SAI.
Bridging On-Prem and Public Cloud
Meta's AI capacity plans cannot be met by data center construction alone: construction runs on multi-year lead times while demand moves in months. Leased facilities and public cloud capacity fill the gap alongside Meta's own campuses — but only if the network behaves identically wherever the GPUs sit.
Cloud networking differs from owning the rack in ways that complicate this. Fabric topologies vary by provider instead of following one design. Buffering is often shallower, changing congestion and loss behavior. RDMA transports are provider-specific, and some clouds still run InfiniBand rather than Ethernet and RoCE. Approaches that work for a few cloud instances tend to break at fleet scale.
Meta's contribution at OCP is how it converges these environments into one operational model. A standardized performance validation framework has cloud partners run a common suite of point-to-point and collective benchmarks, scaling from tens to over a thousand GPUs, before a cluster is handed over, so performance is proven rather than assumed. Heterogeneous, often multiplane fabrics are normalized by mapping divergent traffic-class schemes and congestion-control parameters onto Meta's lossless RDMA classes, so end-to-end paths behave predictably across different vendors' NICs and switches. The underlying principle: heterogeneity is manageable, but standardization is what makes it affordable. Testing and qualification, RDMA transport and congestion behavior, and telemetry still diverge across providers, and every divergence becomes engineering work.
Scale-Up Ethernet and OSPA
Within Meta's scale-up work, the MTIA 300 system uses Tahan, an Ethernet-based 51.2T scale-up switch delivering 6.4 Tbps per accelerator. Meta previewed a next-generation 72-accelerator-per-rack design on 102.4T silicon, moving toward an ESUN-based implementation with future chips.
The Open Switch Platform Architecture (OSPA) initiative standardizes the switch platform layer and is co-led by Meta, Microsoft, and Alibaba; specification v1.0 has been submitted to OCP. MetaRoCE, Meta's RDMA transport protocol for AI networks, now has open-source implementations.
Ethernet Moves Into Scale-Up
Scale-up — the high-bandwidth, ultra-low-latency interconnect between accelerators inside a rack or pod — has traditionally depended on proprietary technologies. OCP 2026 serves as proof that Ethernet can take on this domain as well.
Tahan, an Ethernet scale-up switch
Tahan is Meta's Ethernet-based scale-up switch, purpose-built for its MTIA 300 training accelerators. It uses Broadcom's 51.2 Tbps Tomahawk 5 ASIC and provides a total capacity of 64x800G, half of which carries scale-up connectivity to the accelerators over the backplane. Four Tahan switches link 16 accelerators in a non-blocking topology.
Tahan also reuses the same lossless RoCE transport — routed IPv6, PFC and AI-tuned QoS — that serves the scale-out network, so both scale-up and scale-out fabrics run under FBOSS.
72 accelerators per rack on 102.4T silicon
A next-generation scale-up blade has been previewed: a 216×400G switch built on Broadcom's 102.4 Tbps Tomahawk6C silicon. Eight such switches interconnect 72 accelerators per rack, a density jump enabled by the 102.4T generation's bandwidth. A later generation of the blade is planned to adopt OCP Ethernet for Scale-Up Networking (ESUN) as its scale-up data plane, extending the commitment to open standards across the full scale-up stack.
ESUN specifications advance
ESUN, the initiative founded within OCP last year, is working with AMD, Arista, ARM, Broadcom, Cisco, HPE, Marvell, Microsoft and NVIDIA on open specifications for:
- Protocol headers and framing optimized for scale-up topologies.
- Error handling mechanisms for lossless multi-hop fabrics.
- Alignment with UEC and IEEE standards.
The work has already yielded the ESUN 1.0 Network Operator Requirements Base Specification, released earlier this year. ESUN 1.0 deliberately focuses on the fabric — defining what the network must deliver while leaving implementation flexibility to GPU and accelerator vendors, which broadens adoption. It also coordinates with existing standards bodies, building on IEEE and Ultra Ethernet Consortium (UEC) work instead of duplicating it.
The group is now close to ESUN 1.1, which adds features such as switch profiles and inter-packet gap reduction to further tune Ethernet for low-latency scale-up use cases.
Taken together: Tahan shipping today, a 72-accelerator-per-rack design on the way that adopts ESUN as its data plane, and ESUN's specifications continuing to evolve.
Opening the Switch Platform and Transport Layers
Openness spans the networking stack, from switch platform hardware up through transport protocols.
OSPA: a reference design for the switch platform layer
SAI made switch data plane software portable across silicon, but the switch platform layer beneath it has stayed proprietary and fragmented. Each new hardware platform demands months of duplicated custom engineering to bring up, and the absence of a standard platform design multiplies long-term maintenance and troubleshooting cost — a tax that no longer fits compressed AI hardware cycles.
Meta is co-leading a new OCP initiative with Microsoft and Alibaba, the Open Switch Platform Architecture (OSPA), a unified reference design for that layer. OSPA standardizes hardware and software interfaces for:
- Power management and power sequencing.
- Sensor and environment control, including thermal and fan management.
- Hardware inventory management and device enumeration.
- Out-of-band hardware interfaces (management Ethernet, serial console, USB, LED indicators).
- Firmware maintenance and updates.
- Platform troubleshooting and diagnostics.
Self-onboarding for FBOSS
Over the past year, work has focused on extending open-source FBOSS so switch vendors can self-onboard their own platforms without Meta engineers in the loop for the port. The aim is a self-service model that allows multiple new switch platforms to be developed concurrently, giving vendors access to a hyperscale-proven NOS as they build FBOSS-compatible hardware. Vendors and developers can start at the FBOSS documentation site and the FBOSS GitHub repository.
MetaRoCE as an open transport
MetaRoCE is an open-source RDMA transport protocol optimized for AI cluster networks over Ethernet. It supplies the congestion control and reliability semantics distributed AI training workloads require, and is purpose-built for the traffic patterns of collective communication across large-scale Ethernet fabrics. The open-source MetaRoCE implementations are being announced at OCP so the wider AI infrastructure community can adopt a production-proven transport stack and build on it.
An Open Platform for AI Networking
AI infrastructure continues to evolve at an accelerating pace, and the problems of building open, efficient and sustainable AI networking exceed what any one company can solve. Engineers, silicon vendors, system integrators and operators are invited to join the OCP community in defining what comes next — through the OCP 2026 presentations, the FBOSS and MetaRoCE open-source contributions, and ongoing collaboration on the networking infrastructure the AI era demands.
The authors are members of the Network Infrastructure team at Meta.
