VMAF v1: Good Is Not Good Enough

VMAF v1: What Changes Under the Hood

Netflix has open-sourced VMAF v1, a major revision of its Video Multimethod Assessment Fusion metric. The update targets known weaknesses in the previous version (v0): sensitivity to compression artifacts, handling of different viewing conditions, and blind spots for banding, chroma artifacts, and high-motion content.

VMAF combines quality-aware features and fuses them with a support-vector regressor trained on subjective data. Despite its accuracy and broad adoption, the core algorithm had room for improvement—especially as new codecs like AV2 emerge and use cases like live streaming and cloud gaming demand reliable quality prediction.

Compression Sensitivity Gains a New Component

Encoding pipelines introduce both compression and scaling artifacts. VMAF quantifies the tradeoff between the two to determine the optimal resolution for a given bitrate budget. In practice, v0 tended to favor higher resolutions at lower bitrates, preferring compression artifacts over scaling—which can be visually annoying. This was partly due to the Detail Loss Metric (DLM) feature, which penalizes contrast/detail loss but is less sensitive to distracting artifacts like blockiness.

VMAF v1 adds the AIM (additive impairments) component from the original ADM formulation, with minor modifications, to complement DLM. The two metrics are linearly combined to improve accuracy.

One Model, Many Viewing Conditions

Artifact visibility depends on viewing distance and canvas size: the same encoded video looks better on a smaller screen or from farther away. The standard VMAF model assumes a 1920×1080 display viewed from approximately 3× the screen height (3H), corresponding to roughly 60 pixels per degree.

For phone viewing—typically 4 to 5H—artifacts are less visible. VMAF v0 approximated this with a second-order polynomial mapping applied to the standard score, but that mapping is hard to generalize and can overpredict quality. VMAF v1 instead adjusts the elementary feature values based on normalized viewing distance. The same model can be trained and reused for phone viewing, 4K@3H, or a more discerning [email protected].

This is achieved by modulating the spatial contrast sensitivity function (CSF) used in DLM. The CSF describes human sensitivity to contrast across spatial frequencies and relates to distortion perceptibility. As viewing distance increases, more pixels fit into a degree of visual angle, lowering distortion visibility.

VMAF v1 uses an adapted version of Barten’s CSF model.

New Foundational Features

VMAF v1 addresses several blind spots in v0:

• Banding detection: Banding appears as staircase-like edges in areas that should be smooth. VMAF v1 integrates the Contrast Aware Multiscale Banding Index (CAMBI) as an elementary feature.
• Chroma artifacts: VMAF v0 only extracted luma-based features. V1 modifies SpEED-QA and applies it to chroma channels to capture artifacts from quantization and subsampling.
• No-enhancement gain (NEG): Previously a standalone mode, NEG is now enabled by default. It serves as a conservative quality metric that helps preserve creative intent, and is already used during codec development for AV2.

Motion Gets an Upper Bound and a Wider Window

VMAF v0’s motion feature has no upper bound, and the training data lacked coverage for high-motion sequences—leading to overprediction for very high-motion scenes. Conversely, motion differencing between consecutive frames caused underprediction for frame rates above 24 or 30 fps. V1 applies an empirically derived hard threshold to the motion feature and adds an option to measure motion over a larger temporal window. The wider window alone doesn't fully capture the perceptual impact of 60 fps content, but it reduces the underprediction seen in v0.

Available Models in VMAF v1

VMAF v1 ships with these models:

• Standard 1080p model: Calibrated for 1080p viewed at 3H, operating range [0, 100].
• Phone model: Normalized viewing distance set to 5H based on experimental data. DLM, AIM, and chroma calculations are adjusted to reflect reduced artifact visibility. Retains the [0, 100] range.
• 4K models: Two variants are released. The 1.5H variant is based on a discerning viewing condition and is conceptually similar to the v0 4K counterpart, operating on [0, 100]—the default choice for most users. The 3H variant is based on a consumer-like condition and operates on [0, 110], which quantifies the additional perceptual benefit of 4K over 1080p when both are viewed at 3H.

VMAF v1’s scores are calibrated to align with v0 through a score transform, preserving the meaning of the numbers while retaining the accuracy gains. The metric is evaluated across subjective datasets covering various codecs, content types, and use cases, with Spearman’s rank correlation coefficients reported against subjective data. Full results will be available in a future technical paper.

Benchmarking VMAF v1 Against v0

When comparing VMAF v1 with v0 across a wide variety of datasets, the newer version shows clear superiority in most cases. (Refer to the model outputs chart below.)

The most significant gains appear on large-scale test sets such as WATERLOO IVC 4K and the Netflix Screen Size Crowdsourcing dataset. VMAF v1 also pulls ahead on content featuring chroma and banding artifacts, and on tests that simulate phone viewing conditions. A small number of datasets show minor regressions, but these are negligible when set against the overall improvements.

For a point of reference on the measurement setup: datasets marked “4K” are evaluated with the [email protected] model, while standard datasets run at 1080p with the appropriate 1080p model. Where indicated, phone-based evaluation uses the dedicated 1080p phone model rather than the standard 1080p@3H variant.

What Changed Inside the Engine

Adding new features did not come at the cost of efficiency. The team deliberately reworked the algorithm to cut computational overhead relative to v0:

• VIF removed: Visual Information Fidelity was dropped from the core feature set. It was computationally expensive and, after updating the other features, no longer contributed meaningfully to accuracy.
• CAMBI optimizations: Both algorithmic and software-level improvements were introduced specifically for the banding detector.
• Lower-scale chroma analysis: The chroma feature is now computed at a reduced resolution, a change that has no measurable impact on prediction accuracy.

The combined effect is a metric that is not only more accurate but substantially faster. Processing speed and threading performance for each model at 1080p and 4K are shown below. The latest libvmaf release also brings markedly better multi-threading performance, which benefits v0 users as well. One caveat: content with heavy banding will incur extra CAMBI processing time, reducing the overall speedup.

Two Practices Worth Keeping

The core algorithm may have changed, but the recommended usage practices have not. Netflix still advises:

1. Match the resolution: Upscale the distorted video to match the source resolution before running VMAF. This ensures both compression and scaling artifacts are captured. Bicubic upsampling remains a solid general-purpose approximation.
2. Choose the right model context: Pick the model (1080p, 4K, or phone) that fits the actual viewing scenario, taking the intended viewing distance into account when interpreting the score.

What Still Needs Work

VMAF v1 is an improvement, not a finish line. The team has identified several areas for future development, including better handling of film grain, higher frame rate content, and perceptual encoder optimizations like adaptive quantization. Netflix plans to release a detailed technical paper on v1 and is also working on an HDR version that builds on the v1 refinements. In the meantime, the project remains open source, and the team is inviting feedback and contributions on edge cases.

This effort was a collaboration across many teams at Netflix, encompassing research, engineering, QA, and user-experience research.

References

The work draws on prior Netflix publications, subjective quality databases (WATERLOO IVC 4K, MCL-V, BVI-CC, and the NFLX Screen Size Crowdsourcing dataset), and academic papers spanning perceptual coding, contrast sensitivity, chroma distortion prediction, and efficient entropy differencing.