What the 2026 Andromeda Engine Changes
Meta Andromeda handles personalized ads retrieval, the first stage of the ads recommendation pipeline. Retrieval trims a large pool of eligible ads to a small set of strong candidates in milliseconds; a later ranking stage scores those survivors in detail at far greater compute cost. The deep modeling quality of ranking has historically been too expensive to run at retrieval's scale.
Building on the foundation from Andromeda's 2024 launch on the NVIDIA GH200 Grace Hopper Superchip, this iteration co-designs model, system, and hardware together to bring near ranking-level performance to retrieval scale. In production the system scores every eligible ad at over one billion ads per second — a 100x lift over the 2024 system — and since 2025 it has delivered a 2.4% increase in conversions on Facebook.
Where the Previous Design Runs Out of Room
The 2024 deployment grew model capacity by four orders of magnitude through hierarchical indexing, GPU-native operators, and a purpose-built ML model. Since then, generative AI creative tools have let advertisers create hundreds of millions of new ads, growing the candidate pool to an unprecedented scale.
Model quality
Retrieval has outsized ROI on incremental quality: small model improvements drive large advertiser gains. The 2024 retrieval models rest on a GPU-optimized deep neural network architecture, and continuing to improve them at sustainable compute cost requires going beyond that paradigm — significantly better model performance without proportional cost growth.
System bottlenecks
The 2024 system evaluates tens of millions of ads per second per GPU card; business demand calls for multi-billion ads per second per card. At that throughput, memory bandwidth becomes the bottleneck: intermediate tensors between GPU kernels account for terabytes per second of memory traffic, and shipping ads between local CPU hosts and remote accelerators saturates the network link. The fragmented, distributed workloads need to be unified into a single GPU pipeline.
Hardware limits
With retrieval on GPU, the key question is how close the system sits to the hardware ceiling and how much room remains for model growth. Answering it means pushing every pipeline component to the limits of the current hardware generation — the motivation for a joint collaboration with NVIDIA.
A Retrieval Mixture-of-Experts Architecture
The latest Andromeda brings three synergistic, LLM-inspired mixture-of-experts (MoE) innovations into retrieval. The result approaches ranking-level performance in a typical retrieval-then-ranking system at approximately 300x fewer inference-time FLOPs than the 2024 retrieval model.
- User modeling: a single user tower gave way to an ensemble of specialized user towers as experts, integrating sequential ad engagement events, user-cohort representations, LLM-derived semantic features, and cross-surface co-engagement signals to capture diverse, time-evolving, contextualized interests.
- Candidate ads tower: a joint encoding strategy incorporates real-time serving context so downstream interaction models receive context-aware representations.
- Interaction overarch layer: the interaction model moved from dense neural networks to an MoE attention-based gating network that dynamically routes candidates to the most relevant user experts.
The architecture's performance is co-designed with its hardware execution model. Inference throughput reaches over 10 billion ads per second on a single GH200 GPU, so capacity can grow — richer feature interactions, additional expert sub-networks, deeper architectures — without a proportional increase in GPU cost.
Collapsing the Pipeline Onto One GPU
The 2026 system adds two innovations: a single GPU kernel fusing model and system workloads per user-ad interaction, and co-location of per-user request logic with per user-ad GPU kernels in a single GPU box.
The fused kernel
Per user-ad inference is a compact sequence — gathering embeddings, executing user-ad interaction models, and scoring. The ~300x FLOPs reduction in the MoE gating architecture frees enough headroom to fuse model execution and non-model system workloads into a single kernel, whereas the 2024 launch fused only operations inside model execution. The fused kernel now runs at over 10 billion ads per second on one GH200.
Retrieval workloads are memory-bandwidth bound: at billions of ads per second, even a small per-ad intermediate tensor multiplies into terabytes per second of high bandwidth memory (HBM) traffic, and that traffic — not arithmetic — dominates multi-kernel pipelines. The fused kernel executes the entire per user-ad computation in one pass, keeping intermediates in GPU registers and thread block shared memory so they are never materialized to HBM, using peak memory bandwidth rather than wasting it.
Co-location
Measured in isolation, the fused kernel runs at multi-billion ads per second. But index matching (enforcing advertisers' targeting rules) and top-K selection ran on separate local CPU hosts, communicating with the remote scoring service over the network. At multi-billion ads per second, that traffic saturates the link, making network bandwidth — not GPU compute — the bottleneck.
The fix was to collapse the system architecture to match the model architecture: index matching, scoring, and top-K selection now share one GPU. Only a compact request payload — user features and a targeting query — reaches a single GPU server, which runs the whole pipeline locally and returns just the top-K results, cutting network traffic by ~100x and returning the system to a GPU-compute-bound regime.
Putting index matching on GPU is difficult: it evaluates complex advertiser-defined targeting logic across tens of feature categories and must support real-time index updates as ads are created, modified, or expired during the day. This required redesigning the index structure for GPU-parallel execution while preserving full production targeting semantics.
Together, the two innovations yield a fully co-located single GPU pipeline that processes over one billion ads per second on one GPU server while scoring every eligible ad — a new throughput benchmark for ads retrieval-stage models, with significant headroom remaining.
Pushing Toward the Hardware Ceiling With NVIDIA
In early 2026 Meta and NVIDIA launched a joint collaboration to measure how close the production pipeline is to current-generation hardware limits. NVIDIA contributed optimized GPU kernels; Meta integrated and benchmarked the full pipeline. Each component achieved 2x to 5x speedup over the baseline.
Retrieval MoE gating network
The optimized gating kernel improved 5.3x, from 2.4 billion to over 10 billion ads per second, reaching ~85% of peak HBM bandwidth. Its latency now matches a simple dot-product scoring kernel on the same hardware while being substantially more expressive — the additional model complexity is essentially free on Tensor Core hardware. INT8 quantization added up to 2x throughput over the optimized FP16 kernels, with accuracy on par with 16-bit precision.
Index matching
Targeted GPU kernel optimizations improved index matching over 2x to over 100 billion ads per second, an order of magnitude above the scoring network — enough headroom to re-run index matching across multiple retrieval pathways without creating a bottleneck.
End-to-end pipeline
Profiling the full flow showed that data movement between pipeline components, not computation, accounted for a significant fraction of end-to-end latency. Co-designing data layouts across index matching and scoring removed the unnecessary indirection. With all optimizations applied, the full pipeline benchmarks at multiple times current production throughput on a single GPU server, with further gains expected as more optimizations land.
Moving to Grace Blackwell
Meta is extending its optimization and co-design work to Grace Blackwell, focusing on platform design optimizations that enable different CPU-to-GPU compute ratios — an important factor for Andromeda-like workloads based on results so far. The team is evaluating end-to-end sensitivity to these ratios, pairing Grace-class CPU performance with Blackwell-class GPU performance for their respective parts of the workload, and exploring how greater flexibility in CPU+GPU execution, resource ratios, and coupling could apply to future platform generations.
The pipeline is not yet at its ceiling: quantization opens further scoring throughput, index matching runs an order of magnitude ahead of the scoring network, and the architecture can absorb deeper interaction networks, richer feature spaces, and additional retrieval pathways without proportional infrastructure growth.
The results above rest on a single design principle: model, system, and hardware are designed as one integrated stack rather than optimized separately. Each change reinforces the others, and the gains compound accordingly.
Three outcomes stand out. First, the retrieval MoE gating architecture reaches near-ranking-level expressiveness without adding serving cost. Second, placing the full retrieval pipeline on one GPU removes network and memory bottlenecks. Third, the pipeline still sits below its hardware ceiling — meaning substantial headroom is available for further scaling without matching the growth in infrastructure.
The same co-design approach is expected to carry over to next-generation model architectures and hardware platforms, and the measured headroom lets the system continue scaling as ad volume rises under Advantage+ automation and generative AI creative tools.
What the design changes deliver
- For users: more relevant ads within the same latency budget.
- For advertisers: better matches to their customers and more conversions.
- For Meta: more advertiser business with lower costs, since each GPU now evaluates more ads per second.
Acknowledgements
We would like to thank Gosh Arzumanyan, Habiya Beg, Andrew Bocking, Zain Brohi, Rohit Chandel, Claire Chen, Jason Chen, Wenlin Chen, Zhenyu Chen, Yuze Dai, Zhou Fang, Junjie Feng, Stephen Fitz, Chunli Fu, Yicong Fu, Xiaorui Gan, Sohan Gantala, Yingsheng Gao, Golnaz Ghasemiesfeh, Ishu Goyal, Wentian Guo, George Han, Yihan He, Jun Hu, Xuan Hu, Kun Huang, Qin Huang, Tianchen Ji, Crystal Jin, Tao Ju, Rajat Karnwal, Santanu Kolay, Michael Kwan, Ilia Lewis, Baiyu Li, Chao Li, Jim Li, Li Li, Lin Li, Liyuan Li, Mingda Li, Ted Li, Xiaotian Li, Yanyu Liang, Yue Liang, Irene Liu, Gaoxiang Liu, Jason Liu, Siqi Liu, Wei Liu, Xiao Liu, Yuheng Long, Yining Lu, Siyi Luo, Anuj Madan, Phalguna Mankar, Jiachen Mao, Yuji Mo, GP Musumeci, Guruswamy Namasivayam, Kexin Nie, Sandeep Pandey, Uladzimir Pashkevich, Clementine Plati, Heng Qiao, Hang Qu, Lei Qu, Anand Rajan, Jatin Sethi, Puneet Sharma, Ruchit Sharma, Yan Shi, Chang Shu, Sean Song, Alec Stashevsky, Matt Steiner, Zheng Sun, Nikhil Vinayak Supekar, Jim Tao, Chintan Thakkar, Tian Tong, Aleksandr Ulanov, Daksh Vasistha, Simha Venkataramaiah, Abhishek Verma, Ruud Vinke, Vidhoon Viswanathan, Sasha Vorontsov, Eric Walden, Dong Wang, Feng Wang, Haole Wang, Jeff Wang, Senhong Wang, Ye Wang, Di Wu, Haocheng Wu, Shawn Wu, Jimmy Xiang, Nan Xiao, Ruichao Xiao, Chao Xie, Yudi Xie, Fangzhou Xu, Nathan Yan, Tak Yan, Hang Yang, Song Yang, Yuhan Yang, Yavuz Yetim, Bowen Yu, Li Yu, Qifan Yuan, Xiaodong Yuan, Lin Zeng, Siqi (Cindy) Zeng, Caijie Zhang, Joanna Zhang, Min Zhang, Qing Zhang, Zhengkai Zhang, Fangyu Zou, and the entire v-team behind the success of Meta Andromeda.
