Optimized shot-based encodes: Now Streaming!

Shot-Level Encoding Reaches Netflix’s Production Pipeline

Netflix’s video algorithms team has spent years refining how its encodes are structured, moving from per-title optimization to per-shot optimization to squeeze more perceptual quality out of every bit. The latest step in that evolution is the production deployment of the Dynamic Optimizer, a framework that analyzes an entire video across multiple resolutions and quality points to find the best compression trajectory for each shot. The result: same or better picture quality at dramatically lower bitrates, now live across a significant portion of Netflix’s catalog.

Adapting a Parallel Pipeline for a 900x Increase in Chunks

The core idea behind Dynamic Optimizer is straightforward—encode each individual shot with its own optimal parameters rather than applying a single setting across a multi-minute chunk. But retrofitting an existing parallel encoding pipeline to handle shot-level granularity was anything but simple.

Under Netflix’s older chunk-based model, a one-hour episode of Stranger Things was divided into roughly twenty 3-minute segments. With shot-based encoding, where the average shot lasts about four seconds, that same episode splits into around 900 separate encode units. That’s an increase of more than two orders of magnitude in the number of chunks to process. It also meant the analysis step—which requires encoding at multiple resolutions and quality levels to find the optimal trajectory—demanded an order of magnitude more compute.

The surge in chunks exposed a bottleneck in the messaging system that coordinates work across compute instances. Rather than overhauling that core infrastructure, the team adapted the pipeline with two engineering innovations: collation and checkpoints.

Collation groups consecutive shots back into larger chunks, preserving roughly the same 3-minute durations the pipeline was originally tuned for. Each compute instance handles the chunk, but encodes each shot within it independently with its own parameter set. This keeps resource allocation predictable while still delivering shot-level optimization.

Those independently encoded shots made checkpoints possible. Previously, if a borrowed compute instance was reclaimed mid-encode, the entire chunk had to be re-encoded from scratch. Now, each shot and its associated metadata are persisted to storage immediately upon completion. If an instance is lost, the retry resumes from the last completed shot rather than restarting the whole chunk—saving both time and computational cost.

Measured Gains: 30–50% Bitrate Reductions

Netflix compared the new optimized encodes against its existing baselines using bitrate-VMAF curves built from thousands of catalog titles. For each bitrate point, the team selected the highest-quality encode at or below that bitrate and averaged the VMAF scores across titles. The curves compare four encoding types: per-chunk encodes for downloads (AVCHi-Mobile and VP9-Mobile), per-title encodes for streaming (AVCMain), and the new shot-based optimized encodes (AVCHi-Opt and VP9-Opt).

Drawing a horizontal line at VMAF = 80—generally considered good quality—reveals how much the optimized encodes save in bandwidth:

Compared to AVCMain per-title encodes, the optimized streams need less than half the bits to hit the same quality. VP9-Opt goes further, achieving that quality at less than one third of AVCMain’s bitrate. Against the mobile-optimized encodes, the savings are 17% for AVCHi-Opt versus AVCHi-Mobile and 30% for VP9-Opt versus VP9-Mobile.

The quality gains at fixed bitrates are equally pronounced. At a typical cellular connection speed of 250 kbps, the optimized encodes deliver noticeably higher VMAF scores than the per-title baseline:

Visual comparisons bear this out. In a frame from an episode of Chef’s Table encoded at around 250 kbps, the AVCMain version scores VMAF 58 and shows visible softening in textures like bricks, trees, and water. The AVCHi-Opt frame jumps to VMAF 73, while VP9-Opt reaches VMAF 79 with the sharpest detail.

A similar pattern appears in the opening scene of 13 Reasons Why. Text at the top of the frame is barely legible in the AVCMain encode at VMAF 60. AVCHi-Opt improves that to VMAF 74, and VP9-Opt makes the text and edges crisp at VMAF 81.

Validating With A/B Tests Across Devices

Compression efficiency is one thing; the real question is whether members notice the difference. Before rolling out the new encodes, Netflix ran A/B tests across a broad range of devices and titles, comparing the optimized streams against the existing AVCMain experience as a control. These tests let the team fine-tune both the encoding algorithms and the adaptive streaming engine for each platform class.

Based on the A/B results, Netflix expects the following member-facing improvements:

• Lower quality drops and fewer rebuffers for viewers on low-bandwidth or unstable connections.
• Same-quality video at reduced bitrate for members with high-bandwidth connections.
• Higher video quality at identical bitrates for devices supporting VP9 streams.
• More viewing hours under cellular data caps, at the same or better quality per gigabyte.

The optimized encodes also apply to Netflix’s offline downloads, so members can store noticeably higher quality video in the same storage footprint.

Rollout Status

AVCHi-Opt encodes have been generated for the entire Netflix catalog over the past several months and are now streaming on iOS, Android, PS4, and Xbox One. VP9-Opt streams are available for a selection of popular content on certain Android devices, with broader device and browser testing still in progress. The shift to shot-based encoding in production is a practical illustration of how research in perceptual quality metrics—VMAF in particular—can translate directly into measurable improvements in the streaming experience.