Capacity Efficiency at Hyperscale: Reusing One AI Platform for Both Sides of Performance Work
Meta’s Capacity Efficiency organization operates at a scale where a 0.1% performance regression across code serving over 3 billion people translates into meaningful additional power consumption. The team’s mandate splits into two directions: offense (proactively finding and shipping optimizations) and defense (detecting regressions in production, root-causing them, and mitigating them). Both have historically been bottlenecked by the same constraint: finite human engineering time.
To break that bottleneck, Meta built a unified AI agent platform. The system encodes the domain expertise of senior efficiency engineers into reusable skills layered on standardized tool interfaces. These agents now handle both detection and resolution of performance issues across the fleet, recovering hundreds of megawatts of power and compressing roughly 10 hours of manual regression investigation into about 30 minutes. On the offensive side, agents can take an efficiency opportunity all the way from analysis to a ready-to-review pull request.
Why Offense and Defense Needed One Platform
At first glance, hunting for optimization opportunities and triaging production regressions look like different jobs. The insight that made a unified system practical was recognizing both problems share identical structure:
Rather than building two separate AI systems, Meta built one platform with two layers:
- MCP Tools: Standardized interfaces for LLMs to invoke code. Each tool performs one function: query profiling data, fetch experiment results, retrieve configuration history, search code, or extract documentation.
- Skills: Encoded domain expertise about performance efficiency. A skill instructs an LLM on which tools to use and how to interpret results, capturing reasoning patterns such as "consult the top GraphQL endpoints for endpoint latency regressions" or "look for recent schema changes if the affected function handles serialization."
This two-layer design promotes a generalized language model into something capable of applying senior-level domain expertise. The same tools power both offense and defense; only the skills differ.
Defense: Automated Regression Fix-Forward
FBDetect, Meta's in-house regression detection tool, catches performance regressions as small as 0.005% in noisy production environments by analyzing time series data. When FBDetect flags a regression, the system attempts to root-cause it to a code or configuration change, primarily by correlating regression functions with recent pull requests. Detection and root-causing were already automated; the bottleneck was what happened next. Traditionally, a root-caused regression pull request was either rolled back (slowing engineering velocity) or ignored (letting resource waste compound).
The new AI Regression Solver changes that calculus by producing a fix-forward pull request automatically. The in-house coding agent follows three steps:
- Gather context with tools: Identify the regressed functions, look up the root-cause pull request, and retrieve the exact files and lines changed.
- Apply domain expertise with skills: Use regression mitigation knowledge specific to the codebase, language, or regression type. Logging regressions, for instance, can be mitigated by increasing sampling.
- Create a resolution: Generate a new pull request and send it to the original author for review.
Offense: From Opportunity Description to Deployable Code
On the offensive side, an "efficiency opportunity" is a proposed conceptual code change believed to improve performance of existing code. Engineers previously had to investigate each opportunity manually before implementing it. Now they can view an opportunity and request an AI-generated pull request that implements it, with review and deployment taking minutes instead of hours.
The offensive pipeline mirrors the defensive AI Regression Solver:
- Gather context with tools: The agent retrieves opportunity metadata, documentation explaining the relevant optimization pattern, examples of how similar opportunities were resolved, the specific files and functions involved, and validation criteria for confirming the fix works.
- Apply domain expertise with skills: Use expert-engineer knowledge for the specific opportunity type, such as memoizing a function to reduce CPU usage, encoded as a skill.
- Create resolution: Generate a candidate fix with guardrails, verify syntax and style, confirm it addresses the intended issue, and surface the code in the engineer's editor ready for one-click application.
As with defense, the tools are shared: profiling data, documentation, and code search are all invoked through the same interfaces. Only the skills change.
Compounding Returns from a Shared Foundation
The unified architecture has proven to be a clean abstraction. Every agent, existing or new, has ready access to performance context through the same interfaces without reimplementing integrations. Within a year of the initial use cases for regressions and opportunities, the same foundation expanded to support conversational efficiency assistants, capacity planning agents, personalized opportunity recommendations, guided investigation workflows, and AI-assisted validation. Each new capability required few to no new data integrations, just composition of existing tools with new skills.
Measured Impact
The Capacity Efficiency program has recovered hundreds of megawatts of power, enough to power hundreds of thousands of American homes for a year. AI systems on both offense and defense contribute to that total, but the deeper shift is operational. Engineers who once spent mornings on defensive triage now review AI-generated analyses in minutes. Engineers working efficiency opportunities can request AI-assisted code instead of starting from scratch. The question "where do I even start?" has been replaced by reviewing and deploying high-impact fixes, enabling the program to scale megawatt delivery across growing product areas without proportionally scaling headcount.
