Encrypted Media Extensions: a common API for protected playback

Encrypted Media Extensions (EME) is a browser API that lets web applications play encrypted audio and video through content protection systems. EME extends the HTMLMediaElement specification — hence the name — and browser support for it is optional; a browser that doesn't implement EME simply won't play encrypted media.

EME is designed so the same application and encrypted files work in any browser, regardless of the underlying protection system. The standardized APIs and flow make the former possible; the concept of Common Encryption makes the latter possible.

EME does not define a content protection or Digital Rights Management system. Instead, it defines a common API for discovering, selecting, and interacting with such systems as well as simpler content encryption. The only system browsers must implement for EME compliance is Clear Key, which serves as a common baseline. EME leaves functions like authentication and authorization to the page author: content protection system-specific messaging is mediated by the page rather than assuming out-of-band communication between the encryption system and a license or other server.

EME components

EME implementations rely on several external components:

  • Key System: A content protection (DRM) mechanism. Apart from Clear Key, EME doesn't define Key Systems.
  • Content Decryption Module (CDM): A client-side software or hardware mechanism for playback of encrypted media. EME provides an interface for applications to interact with whatever CDMs are available.
  • License (Key) server: Interacts with a CDM to supply decryption keys. Negotiations with the license server are the application's responsibility.
  • Packaging service: Encodes and encrypts media for distribution.

An application using EME talks to a license server to obtain keys, but user identity and authentication aren't part of EME. Key retrieval happens after optional user authentication. Services like Netflix handle authentication themselves within their web application.

How EME works

When an application attempts to play media with one or more encrypted streams, the browser recognizes the encryption — from header metadata such as ISO BMFF's protection scheme information box or WebM's ContentEncryption element — and fires an encrypted event carrying metadata (initData) about the encryption.

The recommended flow is to negotiate a MediaKeys configuration before selecting a format and codec. The CDM may support only a subset of what the browser supports for unencrypted content. If the application waits for the encrypted event and MediaKeys turns out to be incompatible with the chosen format/codec, switching mid-playback may be impossible. Use MediaKeysSystemAccess.getConfiguration() to learn the negotiated configuration. The only reason to wait for the encrypted event is if you can't know whether content is encrypted in advance — unlikely in practice.

From the encrypted event:

  1. If no MediaKeys object is associated with the media element, select an available Key System using navigator.requestMediaKeySystemAccess(). A MediaKeySystemAccess object lets you create a MediaKeys object for that Key System. Each MediaKeys object represents a CDM instance and provides access for creating key sessions. Fetching the license server URL is handled independently by the application. Ideally, MediaKeys initialization happens before the first encrypted event.
  2. Call setMediaKeys() to associate the MediaKeys object with the HTMLMediaElement, so its keys can be used during decoding.
  3. Create a MediaKeySession via createSession() on MediaKeys. This session represents the lifetime of a license and its keys.
  4. Pass the media data from the encrypted handler to the CDM by calling generateRequest() on the session.
  5. The CDM fires a message event — a request to acquire a key from a license server.
  6. The application receives this message and sends the request to the license server, e.g., via XHR.
  7. The application passes the response to the CDM via the session's update() method.
  8. The CDM decrypts the media using keys from the license, indexed by Key ID. Media playback resumes.

Multiple messages may pass between the CDM and license server. All of this communication is opaque to browser and application alike — only the CDM and license server understand the messages, although the app layer can see the message type. The license request proves the CDM's validity and includes a key for encrypting the content keys in the resulting license.

Getting a key from a license server

In typical commercial use, a packaging service encrypts and encodes content. A web client obtains a key — contained within a license — from a license server to enable decryption and playback. The following code, adapted from the spec examples, demonstrates choosing a key system and fetching a key.

    var video = document.querySelector('video');

    var config = [{initDataTypes: ['webm'],
      videoCapabilities: [{contentType: 'video/webm; codecs="vp09.00.10.08"'}]}];

    if (!video.mediaKeys) {
      navigator.requestMediaKeySystemAccess('org.w3.clearkey',
          config).then(
        function(keySystemAccess) {
          var promise = keySystemAccess.createMediaKeys();
          promise.catch(
            console.error.bind(console, 'Unable to create MediaKeys')
          );
          promise.then(
            function(createdMediaKeys) {
              return video.setMediaKeys(createdMediaKeys);
            }
          ).catch(
            console.error.bind(console, 'Unable to set MediaKeys')
          );
          promise.then(
            function(createdMediaKeys) {
              var initData = new Uint8Array([...]);
              var keySession = createdMediaKeys.createSession();
              keySession.addEventListener('message', handleMessage,
                  false);
              return keySession.generateRequest('webm', initData);
            }
          ).catch(
            console.error.bind(console,
              'Unable to create or initialize key session')
          );
        }
      );
    }

    function handleMessage(event) {
      var keySession = event.target;
      var license = new Uint8Array([...]);
      keySession.update(license).catch(
        console.error.bind(console, 'update() failed')
      );
    }

What happens inside a CDM

EME itself doesn't decrypt media; it only provides the API for an application to talk to CDMs. What CDMs actually do is not defined by the spec — a CDM may handle decoding as well as decryption. From least to most robust, possible CDM functionality includes:

  • Decryption only, with playback through the normal media pipeline, e.g., via a <video> element.
  • Decryption and decoding, passing video frames to the browser for rendering.
  • Decryption and decoding with direct rendering in hardware, such as the GPU.

The EME spec doesn't dictate how a CDM gets onto a system. Options include bundling it with the browser, distributing it separately, building it into the OS, putting it in firmware, or embedding it in hardware. In all cases, the browser vets and exposes the CDM. EME doesn't mandate any particular Key System; among current browsers, Chrome supports Widevine and IE11 supports PlayReady.

Common Encryption

Common Encryption enables content providers to encrypt and package content once per container/codec and use it across various Key Systems, CDMs, and clients — anything supporting Common Encryption. A video packaged with PlayReady could, for example, play back in a browser using a Widevine CDM with a key from a Widevine license server. This contrasts with legacy vertical solutions that required a single, tightly coupled client stack.

Common Encryption (CENC) is an ISO standard defining a protection scheme for ISO BMFF; WebM follows a similar concept.

Clear Key

Every browser supporting EME must implement Clear Key, even though EME doesn't define DRM. With Clear Key, media is encrypted with a key and played back by simply providing that key. It can be built into the browser — no separate decryption module required.

Clear Key is fully interoperable across all EME-supporting browsers and is useful for testing EME implementations and applications without needing to request a content key from a license server. There's a working Clear Key example at simpl.info/ck, and the code walkthrough below parallels the license-server flow described above, minus the license server.

// Define a key: hardcoded in this example
// – this corresponds to the key used for encryption
var KEY = new Uint8Array([
  0xeb, 0xdd, 0x62, 0xf1, 0x68, 0x14, 0xd2, 0x7b, 0x68, 0xef, 0x12, 0x2a, 0xfc,
  0xe4, 0xae, 0x3c,
]);

var config = [
  {
    initDataTypes: ['webm'],
    videoCapabilities: [
      {
        contentType: 'video/webm; codecs="vp8"',
      },
    ],
  },
];

var video = document.querySelector('video');
video.addEventListener('encrypted', handleEncrypted, false);

navigator
  .requestMediaKeySystemAccess('org.w3.clearkey', config)
  .then(function (keySystemAccess) {
    return keySystemAccess.createMediaKeys();
  })
  .then(function (createdMediaKeys) {
    return video.setMediaKeys(createdMediaKeys);
  })
  .catch(function (error) {
    console.error('Failed to set up MediaKeys', error);
  });

function handleEncrypted(event) {
  var session = video.mediaKeys.createSession();
  session.addEventListener('message', handleMessage, false);
  session
    .generateRequest(event.initDataType, event.initData)
    .catch(function (error) {
      console.error('Failed to generate a license request', error);
    });
}

function handleMessage(event) {
  // If you had a license server, you would make an asynchronous XMLHttpRequest
  // with event.message as the body.  The response from the server, as a
  // Uint8Array, would then be passed to session.update().
  // Instead, we will generate the license synchronously on the client, using
  // the hard-coded KEY at the top.
  var license = generateLicense(event.message);

  var session = event.target;
  session.update(license).catch(function (error) {
    console.error('Failed to update the session', error);
  });
}

// Convert Uint8Array into base64 using base64url alphabet, without padding.
function toBase64(u8arr) {
  return btoa(String.fromCharCode.apply(null, u8arr))
    .replace(/\+/g, '-')
    .replace(/\//g, '_')
    .replace(/=*$/, '');
}

// This takes the place of a license server.
// kids is an array of base64-encoded key IDs
// keys is an array of base64-encoded keys
function generateLicense(message) {
  // Parse the clearkey license request.
  var request = JSON.parse(new TextDecoder().decode(message));
  // We only know one key, so there should only be one key ID.
  // A real license server could easily serve multiple keys.
  console.assert(request.kids.length === 1);

  var keyObj = {
    kty: 'oct',
    alg: 'A128KW',
    kid: request.kids[0],
    k: toBase64(KEY),
  };
  return new TextEncoder().encode(
    JSON.stringify({
      keys: [keyObj],
    }),
  );
}

To test Clear Key code, you need an encrypted video. For WebM, encrypting a video for Clear Key is done per the webm_crypt instructions; commercial services are available for ISO BMFF/MP4, with other solutions in development.

MSE: JavaScript-Driven Streaming

The HTMLMediaElement handles playback simply when given a src URL. Media Source Extensions (MSE) extends this by letting JavaScript build streams from chunks of video, enabling adaptive streaming and time shifting.

<video src="foo.webm"></video>

MSE matters for EME because commercial providers must adapt delivery to network conditions. Netflix, for instance, dynamically changes stream bitrate as conditions change. EME works with media streams from an MSE implementation just as it would with media from a src attribute.

After creating a SourceBuffer, an entire movie can be "streamed" to a video element by appending chunks via appendBuffer():

var sourceBuffer = mediaSource.addSourceBuffer(
  'video/webm; codecs="vorbis,vp8"',
);

reader.onload = function (e) {
  sourceBuffer.appendBuffer(new Uint8Array(e.target.result));
  if (i === NUM_CHUNKS - 1) {
    mediaSource.endOfStream();
  } else {
    if (video.paused) {
      // start playing after first chunk is appended
      video.play();
    }
    readChunk_(++i);
  }
};

A working example is available at simpl.info/mse, where a WebM video is split into five chunks using the File APIs; in production, chunks would be retrieved via AJAX.

DASH: Adaptive Delivery Over HTTP

Multi-device web usage means changeable connectivity. Dynamic Adaptive Streaming over HTTP (DASH, or MPEG-DASH) is designed for the best possible delivery under flaky conditions, for both streaming and download. Alternatives like Apple's HTTP Live Streaming (HLS) and Microsoft's Smooth Streaming exist, but DASH is the only HTTP-based adaptive bitrate method built on an open standard. YouTube already uses it.

DASH describes itself directly:

  • Dynamic: responds to changing conditions.
  • Adaptive: adapts to provide an appropriate audio or video bitrate.
  • Streaming: allows for streaming as well as download.
  • HTTP: enables content delivery with the advantages of HTTP, without the disadvantages of a traditional streaming server.

With MSE-based DASH, a client parses a manifest, downloads segments at an appropriate bitrate, and feeds them to a hungry video element over existing HTTP infrastructure.

In practice, media is encoded several times at different bitrates, with each encoding called a Representation. These are split into Media Segments. The client plays a programme by requesting segments in order from a representation over HTTP. Representations containing equivalent content can be grouped into Adaptation Sets. A client wishing to change bitrate picks an alternative from the current adaptation set. Content is encoded to make switching easy. Each representation generally also has an Initialization Segment—think of it as a header with encoding and frame size info—which the client must obtain before consuming media segments.

To summarize:

  1. Media is encoded at different bitrates.
  2. The different bitrate files are made available from an HTTP server.
  3. A client web app chooses which bitrate to retrieve and play back with DASH.

During segmentation, an XML manifest called a Media Presentation Description (MPD) is built programmatically. It describes Adaptation Sets and Representations with durations and URLs:

    <MPD xmlns="urn:mpeg:DASH:schema:MPD:2011" mediaPresentationDuration="PT0H3M1.63S" minBufferTime="PT1.5S" profiles="urn:mpeg:dash:profile:isoff-on-demand:2011"
    type="static">
      <Period duration="PT0H3M1.63S" start="PT0S">
        <AdaptationSet>
          <ContentComponent contentType="video" id="1" />
          <Representation bandwidth="4190760" codecs="avc1.640028" height="1080" id="1" mimeType="video/mp4" width="1920">
            <BaseURL>car-20120827-89.mp4</BaseURL>
            <SegmentBase indexRange="674-1149">
              <Initialization range="0-673" />
            </SegmentBase>
          </Representation>
          <Representation bandwidth="2073921" codecs="avc1.4d401f" height="720" id="2" mimeType="video/mp4" width="1280">
            <BaseURL>car-20120827-88.mp4</BaseURL>
            <SegmentBase indexRange="708-1183">
              <Initialization range="0-707" />
            </SegmentBase>
          </Representation>

          …

        </AdaptationSet>
      </Period>
    </MPD>

This XML comes from the .mpd file used by the YouTube DASH demo player.

While an MPD could in theory serve as a video's src, browser vendors instead leave DASH support to JavaScript libraries using MSE, such as dash.js. Implementing DASH in JavaScript lets the adaptation algorithm evolve without browser updates, and also allows experimentation with alternative manifest formats. Google's Shaka Player implements a DASH client with EME support.

Mozilla Developer Network provides instructions for using WebM tools and FFmpeg to segment video and build an MPD.

The Outlook for Protected Content

Streaming paid video and audio over the web is growing rapidly, and over 85% of mobile and desktop browsers now support <video> and <audio>. As browser vendors curb support for the plugin APIs most media plugins rely on, native support for protected content distribution becomes increasingly significant for the major content providers delivering to tablets, consoles, connected TVs, and set-top boxes.

Specs and standards

Articles, demos and tools