Containers, streams, and codecs: a primer
Delivering media on the web is rarely a matter of uploading a single file straight from your camera. Raw files—often .mov or .mp4 if you're using a modern mirrorless camera—are typically intended for recording and post-production, not playback in a browser. Their size alone can be prohibitive for streaming, especially at 4K resolution on mobile devices. To serve video effectively, you need to understand a few underlying concepts and likely produce multiple optimized files.
Containers and codecs
Media on the web is structured in layers. The file you see on disk is a container, identified by an extension such as .mp4, .webm, or .ogg. Inside, the container holds one or more streams—typically an audio stream and a video stream, though other types like captions and data can exist. The audio and video streams themselves are compressed with a codec, which is a coder/decoder and compression format for the data.
The distinction between container and codec is important because the same container format can hold content encoded with a variety of codecs. For instance, a single WebM file may have a video stream compressed with one codec and an audio stream compressed with another, all wrapped together in the same structural frame:
Typically, the files you'll encounter contain just one audio stream and one video stream. Files in the WebM container can be significantly smaller than other formats, which makes them an appealing choice for mobile streaming. However, not all codecs enjoy universal browser support. As of the time of this writing, Safari does not support WebM for embedded video, though there is partial support if the VP8 or VP9 codec is used in a WebRTC context. In practice, you should plan to provide a fallback video format for maximum compatibility.
A quick guide to common codec formats
WebM may not be the only format you need to account for. Many containers support multiple codecs, and your best bet is to match the file format to the browsers you're targeting. MDN maintains practical lists of what is usable on the web for both video and audio codecs. The table below shows the currently preferred file types and the codecs they may use, with an indicator for the recommended video codec:
Following the file type links will show you the browsers that support each format. Codec support can be a moving target, so it is a good idea to verify your choices against current compatibility data when building a media delivery pipeline.
Bitrate and resolution
Two other characteristics shape how a stream performs in the wild. Bitrate is the maximum number of bits used to encode one second of a stream; more bits per second generally means higher detail and fidelity. Resolution describes the amount of information in a single frame of video, given by the number of logical pixels in each dimension. These concepts are covered in more depth in the companion articles on Bitrate and Resolution.
With a grasp of these fundamentals, you can move on to the practical side of examining your own media characteristics with command-line tools like Shaka Packager and FFmpeg, as outlined in the next article on media application basics.
