Meta's OCP Summit 2022 announcements: A new AI platform, a revised rack standard, and a storage refresh
Meta's hardware team used this year's Open Compute Project Global Summit to push open designs for large-scale AI infrastructure. The centerpiece is Grand Teton, the company's next-generation GPU-based platform, which will be contributed to the OCP community. The announcement comes alongside a new version of Open Rack, a cold-plate cooling design, and an HDD storage system, all released under open-source licenses.
These releases continue a pattern of open hardware contributions that dates back to the Big Sur server design in 2015. The new systems are a direct response to the trajectory of Meta's own AI models, particularly deep learning recommendation models (DLRMs) with tens of trillions of parameters that can require a zettaflop of compute to train.
Grand Teton: Consolidating the AI server chassis
Grand Teton is the successor to the Zion-EX platform, with significant gains across the board: 4x the host-to-GPU bandwidth, 2x the compute and data network bandwidth, and a doubling of the power envelope. Rather than being a fundamental performance leap, much of the value here lies in how the system is packaged.
The previous Zion generation was built as three separate boxes — a CPU head node, a switch sync system, and a GPU system — that required external cabling. Grand Teton folds all of those into a single integrated chassis. That means power, control, compute, and fabric interfaces are all part of one system, which Meta states improves overall performance, signal integrity, and thermal behavior.
That integration is also the key operational differentiator. A single chassis means fewer external connections, fewer potential points of failure, and a simplified deployment process that should get the platforms into fleets faster. The higher compute capacity is aimed at both memory-bandwidth-bound workloads like DLRMs and the compute-bound jobs associated with content understanding.
Open Rack v3: A flexible power architecture
Meta's addition to the wider OCP rack and power ecosystem, Open Rack v3 (ORv3), is designed to bridge current data center layouts and future ones that will be dominated by AI accelerators. The frame and power infrastructure aim to be flexible enough to cover both.
A notable mechanical change sees the power shelf decoupled from the busbar. It now installs anywhere in the rack, which permits variable rack configurations. For scale, multiple shelves can hang off a single busbar to support 30kW racks. The shift to a 48VDC output is intended to serve the higher power transmission needs of future accelerators.
The battery backup unit has also been upgraded. Where the old model provided 90 seconds of power, the new unit offers four minutes at 15kW per shelf. It installs anywhere in the rack like the power shelf and delivers 30kW when two are deployed as a pair.
From a design process standpoint, ORv3 took a different route than typical in-house hardware. Meta chose to develop nearly every component in the open through OCP from the start. That approach extends the development cycle, but the resulting ecosystem benefits are improved supplier interoperability and a diverse toolkit of second sources. The rack is built to accommodate Grand Teton and similarly powerful future systems.
Cooling strategies for higher socket power
Higher socket power brings a correspondingly complex thermal management problem. The ORv3 ecosystem is designed to accommodate multiple liquid-cooling strategies, including air-assisted liquid cooling and facility water cooling, alongside an optional blind mate liquid cooling interface. The blind mate design provides dripless connections between IT gear and the liquid manifold to simplify field service.
The blind mate work is part of a dedicated ORv3 Blind Mate Interfaces Group, formed in 2020 with industry experts, suppliers, and partners. The focus group is developing specifications for rack interfaces, structural enhancements for liquid cooling, blind mate quick connectors and manifolds, hose and tubing requirements, and IT gear design concepts.
The chart above illustrates the projected power growth for high-bandwidth memory (HBM) and training modules over several years, along with the point where standard air cooling gives way to liquid-based approaches.
Grand Canyon: Density upgrades for HDD storage
AI infrastructure is not just the compute. Supporting the data requirements means steady improvement on the storage side. Grand Canyon is Meta's next-generation storage platform, designed for denser HDDs without a penalty in performance or power utilization. It also comes with improved hardware security.
PyTorch under Linux Foundation stewardship
PyTorch has moved under the Linux Foundation as an independent PyTorch Foundation, a change the company announced in September 2022. Meta will continue using PyTorch as its primary framework for AI research and production, but the new foundation is responsible for its stewardship, including training courses and conference support. The goal is a vendor-neutral ecosystem that encourages wider industry adoption of AI tooling.
