xHE-AAC: Meta's New Standard for Audio at Scale
Meta serves an enormous range of media content to billions of users, from short-form uploads like Reels to premium VOD and live streams. To ensure consistent, high-quality audio across these diverse use cases, the company has adopted the xHE-AAC audio codec, developed substantially by the Fraunhofer Institute for Integrated Circuits IIS. The codec is now live on Facebook and Instagram, powering audio for features including Reels and Stories.
Solving the Loudness Problem
With hundreds of millions of uploads daily, Meta receives audio tracks with loudness levels spanning from silence to full scale. When users play videos sequentially, these variations can force constant volume adjustments, creating listener fatigue.
xHE-AAC addresses this through integrated loudness management. Rather than baking in a fixed target level and dynamic range compression (DRC) profile during encoding, the codec delegates loudness processing to the client via metadata. This allows the original audio to remain untouched during encoding while clients apply context-appropriate loudness and dynamic range adjustments for the specific playback environment. The system averages loudness across sessions to a consistent target level, preserving creator intent without sacrificing uniformity.
Enabling Seamless Adaptive Bitrate Audio
Mobile users expect uninterrupted, high-quality audio, but variable connection quality makes this a challenge. Meta has historically produced multiple audio qualities to match bandwidth conditions, yet audio ABR was impractical because switching lanes during playback compromised quality. The company previously applied ABR algorithms almost exclusively to video.
xHE-AAC changes this with immediate playout frames (IPFs), which contain all the data needed to start playing a new audio lane independently of other frames. By inserting an IPF at the start of each DASH segment and aligning segment durations across lanes, playback can switch between audio qualities seamlessly. This delivers the highest quality audio the network can support while avoiding playback stalls. After launching audio ABR on Facebook for Android, the company observed a reduction in sessions with playback stalls.
Deployment Details
Meta generates xHE-AAC bitstreams using an encoder SDK from Fraunhofer IIS, then prepares the resulting files for DASH streaming with shaka-packager. The encoder operates in two-pass mode: the first pass measures the input loudness envelope and average program loudness, and the second pass performs audio compression. This approach also enables loudness range control (LRAC) DRC, which mitigates the pumping artifacts common with single-pass DRC algorithms.
To prepare an xHE-AAC audio adaptation set for ABR delivery, the team inserts IPFs at constant intervals, keeps parameters such as sample rate and channel configuration constant, and assigns unique stream identifiers to each lane in the set.
Playback and Availability
At playback time, the client configures a target loudness level and DRC effect type based on listening context, ranging from headphones to device speakers and varying ambient noise levels. Because loudness metadata is embedded in the bitstream, a single xHE-AAC stream can serve a wide array of consumption scenarios. When bandwidth is constrained, audio ABR automatically switches to a lower quality lane, and conversely switches up when bandwidth is plentiful, all without disrupting the session.
xHE-AAC audio is currently available on Facebook for iOS and Android, and on select Instagram surfaces including Reels and Stories. Users need iOS 13+ or Android 9+ and the latest app versions to experience it.
