Dialogue Levels and Dynamic Range

Before examining how xHE-AAC handles loudness, it helps to define the phenomena being controlled. Consider the waveform of a program, shown below in Figure 1.

Figure 1. Example program waveform

To measure a program’s dynamic range, engineers break the waveform into short segments—such as half-second intervals—and compute the RMS level of each segment in dBFS. Plotting those measurements on a single vertical line reveals the spread, as shown in Figure 2. A campfire’s ambient sound might be 60 dB softer than an exploding car in an action scene. That difference between the quietest and loudest sounds is the program’s dynamic range.

Figure 2. Dynamic range of a program with some examples

Loudness is the subjective perception of sound pressure. While most directly correlated with sound pressure level, it is also influenced by the duration and spectral makeup of a sound. Research in cinematic and television content shows that dialogue level is the most important element in a viewer’s perception of program loudness. In Figure 2, the dialogue level is the bold black line.

Not every program shares the same dialogue level or dynamic range. Figure 3 illustrates the variety across different types of content.

Figure 3. Typical dynamic range and dialogue levels of a variety of content. Black lines indicate average dialogue level; red and yellow are used for louder/softer sounds.

An action film may place dialogue at -27 dBFS, leaving headroom for loud effects like explosions. A live concert, by contrast, has a relatively small dynamic range, with dialogue near the top of the mix. Other shows fall somewhere in between. Each title is mixed under its own set of conditions.

Switch from the action film to the live concert without any processing and you will likely be reaching for the volume control. Move to a drama and dialogue may become unintelligible until you turn the volume back up. This is the problem loudness management solves.

Loudness Management with MPEG-D DRC

Loudness management aims to play all titles at a consistent volume relative to each other. When working effectively, once you set a comfortable volume you never have to change it—even jumping from a movie to a documentary to a concert. Netflix specifically targets playing all dialogue at the same level, consistent with the ATSC A/85 and AES71 recommendations for broadcast and online video.

Before encoding, Netflix measures the loudness of all content. Using an anchor-based (dialogue) measurement as recommended by A/85, the measured dialogue level is stored in the MPEG-D DRC metadata in the xHE-AAC bitstream via the anchorLoudness metadata set. In the Figure 3 example, the action show would carry an anchorLoudness of -27 dBFS; the documentary, -20 dBFS.

On Android, Netflix controls the output level through KEY_AAC_DRC_TARGET_REFERENCE_LEVEL in the MediaFormat class. The decoder applies a gain equal to the difference between the target output level and the anchorLoudness metadata, normalizing all content so dialogue is consistently output at the same level. In Figure 4, the output level is -27 dBFS, and content with higher anchor loudness is attenuated accordingly.

Figure 4. Content from Figure 3, normalized to achieve consistent dialogue levels

Devices can set a target output level based on their capabilities and the listening environment. On a mobile device with small speakers, a higher output level such as -16 dBFS is often desirable, as shown in Figure 5.

Figure 5. Content from Figure 3, normalized to a higher output level, with peak limiting applied as needed (dark red)

Programs with low anchor loudness—notably the action and thriller titles—have to be amplified to reach that output level. Without protection, the loudest content would clip and introduce harmonic distortion. The decoder applies peak limiting to prevent this. The tradeoff is acceptable on some devices, but xHE-AAC also includes peak audio sample metadata that offers improved peak protection.

This decode-side gain approach uses metadata to normalize loudness with a minimal number of gain stages in the end-to-end chain, preserving audio quality. Devices keep the flexibility to adjust output level for their specific conditions. When listening conditions are optimal—such as in a home theater—Netflix can bypass loudness normalization entirely for a “pure” mode.

Dynamic Range Control via uniDRC()

Dynamic range control serves many creative and practical purposes in audio production. During playback, its goal is to optimize the dynamic range for the best listening experience on any device, in any environment. Netflix applies the uniDRC() payload metadata in xHE-AAC to perform dynamic range reduction when it benefits the member, based on device and environment.

Figure 2 (repeated). Dynamic range of a program with some examples

The example in Figure 2 has a dynamic range of 60 dB. In a high-end listening environment—over-ear headphones, home theater, or cinema—members can appreciate both the subtleties of a quiet scene and the impact of a loud action sequence. But many playback situations cannot accommodate such a wide dynamic range: low-fidelity earbuds, mobile device speakers, or playback in a noisy environment. When the dynamic range of the playback chain exceeds what the content needs, listeners miss details in the soundtrack or end up adjusting the volume frequently—turning up quiet sections and turning down loud ones. In extreme cases, dialogue becomes hard to follow even at maximum volume. In these scenarios, DRC compresses the content’s dynamic range to something more suitable, as in Figure 6.

Figure 6. The program from Figure 5, after dynamic range compression (gradient). Note that DRC affects loudest and softest parts, but not dialogue.

Reducing dynamic range without audible artefacts requires a sophisticated algorithm with sufficient lookahead. A good DRC algorithm leaves dialogue untouched and applies gentle gain only when sounds exceed or fall below comfortable listening levels. Because encoding offers ample processing power and lookahead, DRC parameters are computed at encode time and delivered to the decoder as metadata. The decoder then applies exactly the gains prescribed by the MPEG-D DRC metadata in the xHE-AAC bitstream.

Loudness and Dynamic Range Control on Android

MPEG-D DRC metadata, which is embedded in xHE-AAC streams, enables Android decoders to adapt audio output to the listening situation. Because the encoding conditions can't predict how a program will be heard, the standard defines several DRC profiles: Limited Playback Range (for small speakers), Clipping Protection, and Noisy Environment, among others. On Android, the profile is selected with KEY_AAC_DRC_EFFECT_TYPE on a MediaFormat.

There's an alternative control path: scaling the DRC gains directly. Android exposes this via KEY_AAC_DRC_ATTENUATION_FACTOR (for gain reduction) and KEY_AAC_DRC_BOOST_FACTOR (for gain increase) on the same format key.

Using Sample Peak Metadata for Limiting

The DRC metadata includes samplePeakLevel, which records the program's maximum sample level — effectively its headroom. A title with an anchorLoudness of, say, -6 dBFS has that much room before clipping.

When the decoder's target output level and the program's loudness together imply amplification — the situation for louder programs at elevated output targets — samplePeakLevel lets the decoder apply DRC-based peak limiting instead of its internal limiter. Since the encoder computed the DRC gains with a more sophisticated algorithm than a decoder's short-lookahead limiter, the result is higher fidelity.

Figure 7. Content from Figure 3, normalized to a higher output level, using DRC to prevent clipping as needed.

Dialogue Preservation in Noisy Environments

Loudness management plus DRC together handle compromised listening conditions. Consider a member in a noisy room where background sound masks everything below roughly -40 dBFS, even with the output level raised to -16 dBFS. In worse cases — loud noise combined with small device speakers — even dialogue can be inaudible.

DRC metadata addresses this directly. When engaged, it boosts the softest program details above the noise floor. Because loudness normalization has already set dialogue at -16 dBFS, DRC leaves it untouched while lifting quieter elements, providing the best possible experience in suboptimal conditions.

Figure 9. Content from Figure 8, with DRC applied to boost previously-inaudible details.

Seamless Bitrate Switching

Adaptive video bitrate has long been standard at Netflix, but audio was fixed-rate due to codec limitations. After bringing adaptive bitrate audio to TVs in 2019, xHE-AAC now enables the same on Android mobile devices, thanks to its native support for seamless bitrate switching. Streams deliver studio-quality audio when bandwidth allows and reduce rebuffering under congestion.

Deployment and Testing

Before rollout, Netflix ran a full A/B test. Content was encoded with Fraunhofer IIS's xHE-AAC encoder, packaged with MP4Box, and compared against the existing HE-AAC streams on Android devices running Android 9 and newer. The Android xHE-AAC decoder used default KEY_AAC_DRC_TARGET_REFERENCE_LEVEL and KEY_AAC_DRC_EFFECT_TYPE values.

Testing covered the three primary audio sinks: built-in speakers, wired headphones/earbuds, and Bluetooth devices. With default loudness and DRC settings, xHE-AAC showed improved member engagement on Android mobile. Three metrics stood out: time-weighted device volume level, volume change interactions, and audio sink changes.

Volume Level

Press enter or click to view image in full size

Figure 10. Time-weighted volume level distribution for built-in speakers. (Cell 2: xHE-AAC)

The plot shows volume level (ranging from 0, mute, to 1,000,000, max) by percentile of members. For the built-in speaker sink, roughly 30% of xHE-AAC listeners kept volume below 0.5M versus about 15% for the control. Lower time-weighted volume is expected — the content is 11 dB louder — and notably, fewer members max out the volume. That matters because listeners at maximum volume may still be unsatisfied with output level; fewer such listeners indicates reduced dissatisfaction.

Volume Change Rate

Press enter or click to view image in full size

Figure 11. Difference in total volume change interactions (Cell 2: xHE-AAC)

"Riding the volume" — turning down loud passages and boosting quiet ones — is common with high dynamic range content. Volume change interactions dropped noticeably with xHE-AAC, a sign that DRC is managing intra-show level differences well. The effect was even stronger for high-dynamic-range titles.

Switching Audio Sinks

Most mobile members use built-in speakers. Moving to headphones often signals that the built-in output is inadequate — perhaps dialogue is too quiet. In the test, members switched away from built-in speakers 7% less often with xHE-AAC overall, and 16% less for high-dynamic-range content.

Conclusion

The findings from this Android deployment should extend to other platforms supporting xHE-AAC. The goal is a consistent experience across listening environments: immersive audio without reaching for the volume control or earbuds.