From LAMP to Large Clusters: Meta's Infrastructure Arc
Meta's infrastructure story begins in 2004 with a handful of servers in a few co-location facilities running the open source LAMP stack. Over 21 years, that footprint grew into a globally networked operation serving more than 3.4 billion people across multiple apps and hardware products. The journey has been marked by a series of scaling challenges — first in software, then in physical infrastructure, and most recently in the entirely new domain of AI workloads.
True to its open source roots, Meta has shared much of its engineering work through research papers and open source hardware and software systems. That commitment extends to the AI era, with an open standards approach to silicon and hardware systems as the company pushes the frontiers of computer science.
The First Scaling Era: Software and Social Graphs (2004–2010)
In the earliest days, each university had its own database, with common web servers connecting students to their respective university's data. The realization that students wanted to connect with friends at other universities gave birth to the social graph that interconnected everyone on the platform.
As Facebook expanded beyond universities to high schools and the general public, database load was managed by scaling Memcache deployments and building entirely new software systems. These included the TAO social graph, a ranking service for News Feed, and a photo and video sharing service. New caching and data management systems followed.
Expansion into Europe signaled that scaling software alone was insufficient. Meta moved down the stack to physical infrastructure, expanding beyond Bay Area co-los to a facility in Ashburn, Va., and building its first purpose-built data centers in Prineville, Ore., and Forest City, N.C.
With multiple data centers came two new problems. The first was connecting a geographically distributed user base to those centers, solved by aggressively building edge infrastructure with compute capacity beside local internet service providers (ISPs) and buying into the peering network connecting ISPs to data centers. The second was replicating the entire software stack to each data center so users got the same experience regardless of physical location. That required a high-bandwidth, multipath backbone network interconnecting data centers, initially over terrestrial fiber linking California and Virginia co-los to the new Oregon and North Carolina data centers.
Global growth eventually pushed Meta beyond single data center buildings into data center regions composed of multiple buildings. Edge presence expanded to hundreds of points-of-presence (POPs) worldwide.
Distributed Systems Problems at Scale (2010–2020)
A global infrastructure forced Meta to confront the hard corner cases of distributed computer science.
Cache Consistency
Users in one region might receive a notification about being tagged in a photo they couldn't yet see, while users in chat threads could receive out-of-order messages. These inconsistencies arose because a fraction of the user base was served out of each data center region, with data updates replicated asynchronously across the distributed fleet. Meta solved this with novel software systems delivering cache invalidations, culminating in a consistency API for distributed systems.
Fleet Management
New abstractions were needed to manage a growing machine fleet across data center regions. Key systems included:
- Twine: a cluster management system scaling to millions of machines in a data center region.
- Tectonic: a data center scale distributed file system.
- ZippyDB: a strongly consistent distributed key value store.
- Shard Manager: a global system managing tens of millions of shards of data across hundreds of thousands of servers for hundreds of applications.
- Delos: a new control plane for global infrastructure.
- Service Router: managing the global service mesh.
Masking Hardware Failure
More machines meant more failures. Meta built systems to mask failures from users and maintain high availability, including:
- Kraken: leveraging live traffic load tests to identify and resolve resource utilization bottlenecks.
- Taiji: managing user traffic load balancing.
- Maelstrom: handling data center-scale disasters safely while minimizing user impact.
Investment in reliability and fault tolerance remains critical for the stability of services connecting people with friends, family, and businesses.
When AI Workloads Arrived (2020)
The late 2010s brought the first AI-induced infrastructure challenges via the rise of short-form video. This content format demanded personalized recommendations — a fundamentally different ranking problem. Previously, content surfaced based on what a user's community of a few hundred friends interacted with. Short-form video required ranking all uploaded content — orders of magnitude larger — and producing a custom ranking for each person for each piece of content.
That workload is where GPUs and other AI accelerators entered the picture. Where a CPU is primarily a load-store machine, a GPU is a vector and matrix processing machine performing orders of magnitude more computation. With a large corpus such as a video library, Meta builds embeddings: mathematical representations of each video as a vector of numbers capturing semantic context in a lower-dimensional space so similar content sits close together. A model tracks a user's click sequence and predicts future videos of interest, combining mathematical similarity with GPU computational power.
Internet services of the 2000s and 2010s scaled by buying cost-efficient but unreliable CPUs, memory, and hard drives, then masking failures with software. An AI cluster is fundamentally different: a high performance computational system of hundreds or thousands of powerful GPUs with ample memory, interconnected via high-bandwidth, low-latency networking, running a custom software stack optimized for maximum performance.
Meta's initial AI clusters interconnected 4k GPUs for training ranking and recommendation models.
The LLM Shift: Synchronous Training at Scale (2022)
Large language models (LLMs) changed assumptions about scale when they took off in 2022. Previously, while AI clusters were 4k in size, individual training jobs typically ran on 128 GPUs.
LLM pretraining demanded dramatically more compute, and more compute produced better models. Within weeks, training job sizes scaled from 128 GPUs to 2k and then 4k GPUs.
For the first time, Meta regularly ran training jobs requiring thousands of GPUs synchronously, where any single straggling GPU held up the entire cluster. Scaling brought challenges: GPUs can fail, memory can have errors, the network can experience jitter. And as with traditional web workloads, more machines meant more failures. The difference was that retrying on a different machine — the standard approach for web requests — wasn't viable when the entire cluster was running one job. A single failure could halt the job, and if jobs failed too frequently, progress stopped because of the time required to checkpoint and restart.
Through collaboration with industry and partners, Meta drove the interruption rate down approximately 50x based on normalized interruption and reliability metrics. As clusters grew, investment in fundamental research and development across AI infrastructure continued. LLMs also influenced how ranking and recommendation models were developed; Hierarchical Sequential Transduction Units (HSTU) accelerated training and inference by 10-1000x for Generative Recommenders.
From 24K to 129K: Scaling Meta's AI Clusters
After getting 4k jobs running efficiently, Meta's infrastructure team faced a new challenge: building clusters an order of magnitude larger. In late 2023, they constructed two clusters of 24,000 H100 GPUs each—one using Infiniband, the other using RoCE—to explore different network technologies while providing capacity for training models like Llama 3.
The results reinforced a simple truth: more compute dedicated to pre-training produces higher-quality LLMs. So infrastructure engineers were tasked with scaling up by another factor of ten. Meta accomplished this by doing something unprecedented in its history: emptying out five production data centers and consolidating the hardware into a single AI cluster with 129,000 H100 GPUs—all built within months.
The Efficiency Problem
The next frontier is efficiency: finding hardware and software solutions that best support Meta's workloads and maximize data center utilization. This is complicated by the heterogeneity of AI workloads. Ranking and recommendation models that power personalized user experiences have different needs than LLMs. And LLMs themselves are evolving rapidly, moving beyond pre-training into reinforcement learning, supervised fine-tuning, test-time inference, and reasoning—each requiring custom hardware and software support.
Given the scale of Meta's AI ambitions, the company works with multiple vendors to encourage market diversity. Solutions come from partners like AMD and NVIDIA, as well as Meta's own custom silicon. One pod design shows six racks, with the middle two housing 72 NVIDIA Blackwell GPUs consuming approximately 140kW of power. Since traditional data centers lack facility liquid cooling, four air-assisted liquid cooling (AALC) racks are deployed adjacent to prevent overheating.
This pod delivers 360 PFLOPS of FP16 compute capacity—over 800x the power of a typical CPU with hundreds of thousands of times the compute. Meta is also beginning work with the GB300, which improves on the GB200 in several respects. AMD's MI300 accelerators serve various workloads, and the software layer—built on open source stacks like PyTorch and Triton—abstracts hardware differences from developers.
MTIA and Custom Silicon
Meta's custom chip program includes the Meta Training and Inference Accelerator (MTIA), optimized for ranking and recommendation inference workloads. Now deployed at scale in data centers—primarily serving ads workloads—MTIA delivers massive efficiency gains over vendor silicon. This marks only the beginning: a training chip for ranking and recommendations is ramping to production, with multiple additional chips in development for deployment over the next couple of years.
Advanced Packaging Constraints
Developing custom silicon has surfaced scaling challenges. Transistors aren't scaling at the same pace as performance demands. Reticle size is limited to 830 mm², so exceeding single-die performance requires investing in more dies. LLM workloads make this scaling need so innate that larger die arrays become unavoidable with each model generation. But dies can only be placed adjacently through advanced 2.5D and 3D packaging, which limits array sizes and creates energy efficiency and cooling concerns. Advanced packaging techniques—integrating multiple chiplets or diverse capabilities like compute, memory, and I/O—paired with advanced cooling solutions, may help overcome these hurdles.
Memory Disaggregation
Reasoning models, test-time inference, and reinforcement learning all pressure memory subsystems. Meta is stacking high-bandwidth memory (HBM) adjacent to compute chiplets to maximize I/O bandwidth. But "silicon beachfront" is limited, forcing tradeoffs between computational capability, memory size, and network bandwidth—while additional HBMs create more cooling concerns. Investing in higher-performance networks and locating HBM off-chip, or even off-machine, might mitigate these issues.
Silicon Photonics
Rack-level power budgets have grown dramatically as Meta builds larger, more interconnected chips. Silicon photonics—which enable faster signaling over larger distances—could significantly reduce rack power consumption. Advanced optical solutions are also the only viable path to increasing shoreline beyond 3.2T and escaping backplane constraints on connected endpoints. These solutions introduce their own challenges: higher power consumption and lower reliability than electrical signaling. Future solutions must be interoperable across vendors, more reliable than electrical signaling, and manufacturable at high volume. Meta is actively researching these hardware challenges and collaborating with the industry ecosystem.
Why Open Standards Matter
Hardware proliferation offers options and lets operators match customized solutions to workload needs—but it creates management challenges for hyperscalers, cloud operators, and developers alike.
Operating 5-6 different hardware SKUs deployed every year makes moving workloads difficult, leading to underutilized hardware. Software engineers face challenges building and optimizing for multiple hardware types; if adoption requires rewriting libraries, kernels, and applications, resistance is strong. Meanwhile, hardware companies struggle to design products without clear data center, rack, or power specifications to target.
The answer lies in open standards, open weight models, and open source software. PyTorch and Triton provide a consistent programming interface for ML developers. Open weight models give application developers cost-efficient access to high-quality LLMs while giving infrastructure and hardware engineers a standard workload to optimize against.
Meta was a founding member of the Open Compute Project and remains a leading contributor, with 187 contributions (approximately 25% of all technical contributions) since inception. This collaboration has improved fleet consistency, delivered economies of scale, and enabled companies to debate solutions collectively. At the 2024 OCP Global Summit, Meta unveiled Catalina, an open-design, high-powered rack for AI workloads, and a new version of Grand Teton, featuring a single monolithic system design with integrated power, control, compute, and fabric interfaces.
Standardization gaps remain. Rack power density increases demand standardized systems, racks, and power specifications—including recent adaptations of Open Compute rack standards for AI needs. Scale-up and scale-out networks need standardization so customers can mix and match GPUs and accelerators. Software innovations must allow jobs to run across heterogeneous hardware spread across geographic locations. These standards need to exist throughout the stack.
Looking to 2026 and Beyond
Computational capability remains key to building higher-quality models. Meta's goal: models that deliver engaging experiences and act as personal assistants for billions of daily users. Building infrastructure for such models requires addressing challenges across data centers—advanced packaging, thermal management, power delivery, memory disaggregation, and scalable networks through optics.
The next AI cluster, Prometheus, will be a 1-gigawatt cluster spanning multiple data center buildings. Construction spans five or more buildings in a single region—a small fraction of a gigawatt facility—using traditional data center buildings, weatherproof tents, and adjacent colocation facilities. The software stack, including Twine and MAST, is evolving to support long-distance training across geographically distributed data centers.
An even larger cluster, Hyperion, is expected to come online beginning in 2028, with capacity scaling to 5 gigawatts.
AI workloads are still early in their evolution. The past few years have been demanding, but the pace is accelerating. The hardware demands AI pushes forward show no signs of slowing.
