Why Meta Reworked Its Ad Scheduler

Meta’s ads serving platform handles more than 5 million requests per second on average at its entry point — over 400 billion per day across all monetized surfaces. At that scale, even a few milliseconds of tail latency degradation can hurt ad relevance and advertiser ROI. When a Linux kernel upgrade threatened to regress latency across the ads fleet, the company turned to sched_ext, the BPF-based extensible scheduling framework that upstreamed into kernel v6.12.

The Problem: General-Purpose Schedulers Don't Know the Workload

Meta previously ran its ads serving on the Linux kernel's general-purpose schedulers — CFS and, more recently, EEVDF. These balance threads across CPUs with no awareness of which work is important. But in the ads serving case, every thread's purpose and criticality are known in advance. The standard schedulers ignore that context.

While upgrading the fleet from kernel v6.4 to the stable v6.9 (which includes the EEVDF scheduler introduced in v6.6), Meta observed a latency regression that reduced the number of ads ranked per response. As a result, some ads hosts were pinned to the older v6.4 kernel, creating technical debt and operational fragmentation across the fleet.

sched_ext — developed in partnership with the authors of Google's ghOS — offered a path forward. It lets developers implement a custom scheduling policy as a BPF program, loaded at runtime, without rebuilding or reinstalling the kernel.

The Custom Policy: Soft Partitions With Domain Knowledge

With sched_ext, the kernel calls into the BPF scheduler through event-driven callbacks when threads wake up, get enqueued, need dispatch to an idle CPU, or when CPUs enter and leave idle states. The ads-specific policy encodes two key pieces of domain knowledge:

  • Soft CPU partitioning: CPUs are dynamically divided into two pools — one for latency-critical request-path threads, one for less sensitive work.
  • L3 cache locality: Related work is kept on the same CPUs over time, reducing expensive DRAM accesses by improving last-level cache hit rates.

The policy itself is packaged as a user-space binary that loads the BPF program. Rolling out a scheduler change means restarting the scheduler process — unloading the old policy, loading the new one — with no kernel rebuild.

Measured Results

The initial launch moved the largest ads serving server type from kernel v6.4 with CFS to kernel v6.9 with sched_ext. Backtest experiments showed:

  • +1.1% on weighted-ads-ranked, the metric for ads retrieved and ranked.
  • 3.28 megawatts of power savings across the fleet.
  • 28% reduction in service p99 latency on the ads retrieval path.

Two follow-on policy updates, shipped entirely as user-space changes, compounded those gains:

  • Additional 60% reduction in service p99 latency.
  • 18% reduction in timeout errors on the critical path.

The second wave of improvements arrived in days, not months, because no kernel release was required. That iteration speed turned sched_ext from a kernel-upgrade unblocker into a continuous optimization platform.

From Stopgap to Strategic Asset

What began as a targeted fix for a specific kernel regression has broader implications for Meta's infrastructure:

A decoupled optimization path. Upstream scheduling changes like the CFS-to-EEVDF transition can be disruptive. With sched_ext, Meta can maintain and refine custom schedulers alongside upstream evolution, keeping critical workloads optimized regardless of what changes in the base kernel.

Independent deployment. Scheduler improvements ship as BPF program updates in days rather than months. Ideas that previously required kernel patches and lengthy validation cycles — cache-aware placement, ROI-based executor routing, NUMA-aware steering — become tractable iterations.

A shared industry asset. Because sched_ext is upstream in Linux v6.12, any operator running a workload that doesn't fit the general-purpose scheduling model — hyperscaler, cloud provider, embedded systems team — can ship workload-specific policies without forking the kernel.

Future Direction

Meta sees room for further gains by giving the ads services more fine-grained control. The services hold context about which requests matter most; they could signal the scheduler when a thread begins working on an important request. The scheduler could then extend that thread's scheduling slice or keep it at the top of the run queue.