Why AV1 Makes Sense for Real-Time Video

Meta has been rolling out AV1 across its real-time communication (RTC) applications, including Messenger and WhatsApp, since initially introducing it on high-end devices in 2023. The codec is now enabled on a majority of mobile devices in those apps. The move is motivated by straightforward math: AV1 delivers the same visual quality as H.264/AVC while consuming significantly less bandwidth. In offline testing, Meta observed at least a 20% bitrate reduction with AV1 compared with H.264/AVC under standard product settings on low-end and mid-range devices, with even larger reductions possible when devices can handle more encoding complexity.

That efficiency matters because RTC products operate under tight latency constraints and unpredictable networks. Real-world video bitrates for RTC typically range from 10 kbps to 400 kbps, and maintaining quality below 100 kbps is a persistent challenge. In live comparisons using two Android phones running Messenger, AV1 kept video substantially clearer than H.264/AVC when both were limited to 100 kbps — the H.264 feed appeared noticeably blurry by comparison.

H.264/AVC (left) versus AV1 (right).

AV1 also offers meaningful advantages for screen content, which is a growing part of video calls. Text and other high-frequency computer-generated content are particularly hard for traditional encoders, and users are highly sensitive to blurry text. AV1’s main profile includes two coding tools designed for this: palette mode, which encodes screen frames by signaling color clusters instead of transform-domain coefficients, and intra-block copy, which enables block prediction within the same frame by exploiting repetitive patterns. Both tools substantially improve compression efficiency for screen-sharing scenarios.

The Real-World Hurdles of Live Encoding

Adopting AV1 for RTC is far more complex than using it for video on demand. Real-time systems must keep end-to-end latency below roughly 300 milliseconds; beyond that, participants begin noticing conversational delays. This constraint rules out techniques that improve quality but add delay, such as multi-pass encoding. Decoder-side buffering must also be kept minimal, and sudden bitrate spikes can cause video freezes that degrade the call experience.

Dynamic adaptation to network conditions creates additional complications. Fluctuations in bandwidth require the encoder to adjust resolution and frame rate, but switching resolutions typically demands a new key frame, which can produce a bitrate spike and temporary freezing. Packet loss may trigger retransmissions or force an additional key frame, both of which carry the same risk. Managing those trade-offs is central to maintaining uninterrupted, high-quality calls. Power efficiency is another key concern, since RTC clients perform real-time encoding and decoding simultaneously on devices where battery life is a primary constraint.

Selecting the Right Tools

The first and most critical technical decision is choosing the encoder and decoder. AV1’s advanced coding tools improve compression but add computational burden during encoding. In an offline experiment using an open-source AV1 encoder on a Pixel 8, power consumption during a video call rose 14% compared with H.264/AVC.

Meta addressed this by adopting an internal low-complexity encoder with power consumption similar to the H.264 baseline. The thinking behind the design is that a newer codec need not inherently demand a heavier encoder. Since modern codecs support a larger set of coding tools, a well-designed encoder can identify better quality-versus-complexity trade-offs — often referred to as presets. An ultra-low-complexity preset that matches H.264/AVC’s encoding complexity would allow AV1 to run on low-end devices. To that end, the low-complexity encoder includes both an optimized high-complexity preset and a new ultra-low-complexity preset. The system dynamically adjusts the encoder preset based on device capabilities, extending AV1 coverage to a significantly broader range of hardware.

Memory is another complication. AV1 encoding increases memory usage relative to H.264/AVC, which caused app crash regressions during mobile adoption and required additional engineering work.

For decoding, complexity remains significant on mobile devices even though decoders are generally less demanding than encoders. Initial A/B tests showed that some low-end devices could not decode in real time, resulting in video freezes and audio/video synchronization problems. Meta compared several open-source decoders and chose dav1d after testing, citing its superior power efficiency and reliability — experiments also showed an increase in talk time with that decoder.

Managing Binary Size

Embedding an AV1 encoder and decoder in a mobile app has a further cost: binary size. Using libAOM as a benchmark, AV1 support adds 1.7 MB to the application (600 kB compressed). For a company serving billions of users, that is a substantial regression. Larger binaries reduce update success rates, slow application startup, and negatively affect memory usage and crash metrics. A 600 kB increase can consume an entire year’s binary size budget for a large organization.

Meta pursued several strategies to keep binary growth in check:

  • A dynamic-download framework that delivered AV1 as a separate component was tried initially, but download failures from poor networks or device issues degraded the user experience.
  • Direct binary size optimizations followed. The quantization matrix (QM) tool, for example, accounts for roughly 10% of the encoder’s library size, and optimizations can halve that footprint. Contributions were also made to the dav1d project to reduce its size.

This work extends to the full pipeline. Removing unused tools from the library entirely is part of the strategy — eliminating QM frees 60 kB, for instance. At the application level, codec libraries can be shared across features such as video message transcoding, and built-in platform codec support is leveraged to avoid bundling additional libraries.

Not Every Device Can Handle AV1

Picking the right encoder and decoder is only half the battle. The harder question is which devices can actually run AV1 without hurting call quality. Determining that list for iOS was trivial given the small number of models. Android was a different story entirely, with thousands of variants to account for. Memory, release year, and OS version all failed as eligibility criteria in testing. Meta eventually landed on its in-house, ML-based device eligibility framework to build a reliable list of Android devices.

Scoring Devices with ML

Beyond simple model lists, the framework itself uses real-world telemetry to categorize device capabilities, rather than relying on lab benchmarks that don't reflect production conditions. The system ingests low-level performance metrics collected from the logging pipeline as model features. The model then outputs an rtc_score, a value that gauges overall AV1 performance on that device and informs codec and call setting decisions.

Figure 1: Our ML-based device eligibility framework.

During 2025, the model went through several iterations. The first, Model V1.1, launched in August 2025 and widened AV1 traffic across a growing set of devices. That added traffic created a larger, more representative dataset for AV1 specifically, which was then used to train Model V2. This version introduced a two-tier structure that separates higher-end from lower-end devices, acknowledging that a flagship and an entry-level phone have very different AV1 encoding headroom. Each iteration broadened AV1 enablement, and the design keeps improving as traffic volumes increase.

Encoding Latency Is the Real Bottleneck

Device eligibility identifies capable hardware, but A/B testing revealed a second problem: calls showed significant audio/video sync regressions because many devices simply couldn't encode or decode video in real time. Even a 2023 octa-core smartphone failed to keep up with encoding at 320×180@15fps. This was true for both H.264 and AV1, though more pronounced with AV1. The likely culprit is CPU frequency throttling during calls, which reduces effective processing power. Pure device-name-based enablement wasn't enough.

The solution came as three latency-aware mechanisms:

  • Adaptive encoder preset adjustment: The system maintains several presets from low to high complexity. A monitor watches live encoding latency and adjusts complexity up or down, keeping the device within real-time limits. Lower complexity when latency spikes, higher when there's headroom, maintaining quality where possible.
  • Local device encoding latency-aware codec switch: If reducing the preset still leaves latency too high, the client switches to H.264/AVC, which can be notably less demanding on the encoder for that content. Both codecs are negotiated at call setup, and the client continuously evaluates conditions to pick the right one. The preset and codec selection decisions are made jointly to avoid oscillation.
  • Peer device decoding latency-aware codec switch: AV1 decoding complexity matters as much as encoding. A high-end phone calling a low-end one can send AV1 just fine while the receiver can't decode it in real time. Each device therefore sends its video decoding latency back during the call, letting the sender fall back to H.264/AVC if the peer is struggling.

These mechanisms also incorporate other health signals, like battery level. A low-battery state can force a switch to H.264/AVC, helping maintain call quality and stretch call duration.

An Asymmetric Rollout Strategy

Once codec selection was dynamic, AV1 support could be extended to mid-range and lower-end Android devices. Many such devices can decode AV1 in real time even if they can't encode it. That makes an asymmetric design viable: mid-range devices send H.264/AVC but receive AV1 from higher-end peers, effectively expanding AV1 coverage across the Android ecosystem.

Figure 2: Asymmetric codec design.

Keeping AV1 Calls Stable Under Real-World Network Conditions

With AV1 enabled across a majority of mobile devices in Meta RTC applications, the focus shifts to maintaining call quality when networks misbehave. Two recurring problems in real-time communication are bandwidth fluctuations and packet loss. The former demands accurate rate control; the latter requires a robust error-resilience strategy.

Rate Control: Avoiding Overshoot and Undershoot

RTC calls run at constant bitrate (CBR), where even brief overshoots can cause congestion and playback freezes on the receiving end. A simple average bitrate check won't catch these spikes, so we evaluate CBR accuracy using the Video Buffering Verifier (VBV) delay — a leaky-bucket model that measures whether a stream can be smoothly decoded from a buffer.

Consider a scenario where the network allocates 100 kbps for video, and the encoder is asked to match that rate. It produces Frame N at 20 kbits, but 5 kbits of Frame N-1 remain in the buffer. Transmitting Frame N now takes 250 ms, well above our desired VBV delay of under 200 ms. This overshoot risks higher latency, congestion, and freezes — a clear case for precise rate control.

Figure 3: An example of VBV delay calculation.

Our rate-control optimizations focus on keeping the encoder from overshooting. The encoder tracks VBV buffer status and uses it to allocate bits; when an overshoot happens, it lowers the bitrate of subsequent frames until the delay is back under control. Many encoders let VBV delay run unchecked, which can snowball into network problems.

Key frames receive the same treatment. Encoders often boost quality on intra-only frames to keep them consistent with inter frames, but in RTC that creates unwanted bitrate spikes. We strictly cap key-frame bitrate and compensate with tighter rates on following frames.

RTC rate control also has to survive frequent changes:

  • Target bitrate shifts. Clients update the encoder rate often; the algorithm must hold VBV delay steady, especially on sharp downshifts.
  • Resolution changes. Senders may alter resolution mid-call. AV1's Reference Picture Resampling (RPR) handles resolution changes without a key frame, cutting bitrate spikes and reducing freezes.

Undershoot is just as harmful as overshoot. Early versions of our algorithm were conservative to avoid overshoot, but the resulting undershoot confused the congestion-control module, slowed bandwidth ramp-up, and ultimately lowered video quality. We revised the algorithm to balance both directions, and a stable bitrate — with minimal deviation — has noticeably improved call quality.

Error Resilience via Temporal Layers and LTR

Modern codecs rely on long dependency chains between frames, which are fragile under packet loss. When a packet is dropped, the receiver NACKs and waits a round trip; if retransmission fails, the chain breaks and the video freezes. Requesting a key frame is expensive — roughly 10x the size of a P-frame — and can congest the network further. We tuned two AV1 features to contain these failure modes.

Temporal Layers for Graceful Degradation

Temporal layers provide scalable frame rates: the base layer (TL0) is independently decodable, while enhancement layers add intermediate frames. If enhancement packets are lost, decoding continues off the base layer without stalling.

Figure 4: Two temporal layer structure.

This structure lets us prioritize protection. We apply FEC to base-layer data and treat enhancement-layer retransmissions based on round trip time — retransmit when RTT is low, skip when it's high, without breaking the decode flow.

The catch is efficiency. A temporal-layer structure is less compression-efficient than a tightly packed prediction chain, so keeping it on full-time degrades quality at a given bitrate. Since losses are intermittent, we enable TL adaptively: the sender watches network feedback, turns TL on when loss climbs, and shuts it off when conditions recover. This delivers resilience when needed and efficiency the rest of the time.

Long-Term Reference for Fast Recovery

Long-Term Reference (LTR) keeps reference frames in the encoder buffer longer than regular references. When a decoding chain breaks, a request pulls an LTR-predicted (LTRP) frame, which resynchronizes the stream from the last successfully decoded LTR — faster than a retransmission and smaller than a key frame.

Figure 5: LTR and LTRP in lossless and lossy scenarios.

LTR requires tight coupling with the network layer. The encoder periodically emits LTR frames and pins them in a bounded buffer of size 4, evicting the oldest pinned frame. From the network's perspective, an LTR frame looks like any other; it can't tell when to send an ACK. We solve this with an explicit LTR indicator — a binary flag in a proprietary RTP header extension that carries per-frame metadata. We also expose the frame_id through LTR bitstream syntax. ACK feedback travels in a separate RTP header extension, each ACK carrying the corresponding frame_id so the sender knows exactly which LTR was received. When handling an LTRP request, the encoder references the most recently ACKed LTR.

The network layer requests LTRP frames in two cases:

  • Reactive recovery. When the receiver freezes and sends an RPSI message requesting an LTRP.
  • Proactive protection. When the sender detects elevated packet loss and asks for periodic LTRP frames — redundant, but effective at cutting freezes.
Figure 6: AV1 encoder interaction with the network layer.

Either way, the encoder's response is the same: if an ACKed LTR is in the buffer, produce an LTRP frame; if not, assume resynchronization is needed and send a key frame.

LTR is more efficient than key frame requests or retransmissions, but it has a cost — LTRP frames reference an older LTR with weaker temporal correlation, making motion prediction less accurate. We mitigate this by leveraging an existing encoder behavior: the encoder already emits a periodic, slightly higher-quality frame to maintain overall quality. We simply mark that frame as LTR, so the reference stays high quality even as it ages.

What's Next for AV1 at Meta

Bringing AV1 to Meta RTC has been a multi-year project spanning codec selection, device eligibility, rate control, and error resilience. Pairing a low-complexity encoder with ML-driven device qualification and adaptive codec switching has put AV1 on most mobile devices, delivering meaningful quality gains — particularly on bandwidth-limited networks.

Group calls are the next frontier. Unlike 1:1 calls, group participants decode multiple streams, making AV1 coverage harder to expand. Software AV1 implementations help, but higher quality and new features will likely depend on hardware AV1 support. SoC vendors that invest in AV1 hardware across device tiers stand to enable better experiences with improved battery life and network efficiency — the clear direction for content and RTC providers alike.