The plugin-free RTC stack

WebRTC brings real-time communication to the browser with three core JavaScript APIs: MediaStream (commonly called getUserMedia), RTCPeerConnection, and RTCDataChannel. All three are supported on desktop and mobile by Chrome, Safari, Firefox, Edge, and Opera, and are defined in the WebRTC and getUserMedia specifications.

MediaStream acquires data streams, such as from a user's camera and microphone. RTCPeerConnection handles audio and video calling with built-in encryption and bandwidth management. RTCDataChannel enables peer-to-peer transport for arbitrary data. To see the APIs in action, the WebRTC samples collection provides over 20 demos, and the WebRTC codelab walks through building a complete video-chat and file-sharing app, including a simple signaling server.

Why WebRTC was built

Historically, real-time communication was corporate and complex, requiring expensive licensed codecs and in-house audio and video infrastructure. Services like Skype, Facebook, and Google Hangouts relied on proprietary plugins or native apps, which are difficult to deploy, debug, test, and maintain, and often hard to persuade users to install in the first place.

Google's acquisition of GIPS, a company that developed core RTC components such as codecs and echo cancellation, provided the technology base. Google open sourced that work and engaged with the IETF and W3C to standardize it. In May 2011, Ericsson built the first implementation of WebRTC. The project's guiding principles were that its APIs should be open source, free, standardized, and built directly into browsers, with better efficiency than existing technologies.

Today, WebRTC is used in production by apps like Google Meet and has also been integrated into WebKitGTK+ and Qt native apps.

What a WebRTC app must do

Getting a WebRTC application working means coordinating several tasks:

  • Acquire streaming audio, video, or other data.
  • Gather network information such as IP addresses and ports, and exchange it with other clients (called peers) to establish a connection, even through NATs and firewalls.
  • Coordinate signaling to report errors and to initiate or close sessions.
  • Exchange details about media and client capabilities, such as resolution and codecs.
  • Stream audio, video, or data back and forth.

If you want to try it immediately, open appr.tc, click Join to enter a chat room and grant webcam access, then open the displayed URL in a different tab or on another computer. For a simpler single-page demonstration of peer connectivity, see the WebRTC samples Peer connection, which relies on adapter.js, a JavaScript shim maintained by Google and the WebRTC community to smooth over browser differences and spec changes.

If you run into issues, the WebRTC Troubleshooter can help isolate whether the problem is with your machine and WebRTC setup.

The MediaStream API

The MediaStream API represents synchronized streams of media. A stream captured from a camera and microphone, for example, contains both video and audio tracks that stay in sync. Note that MediaStreamTrack is unrelated to the HTML <track> element, which serves an entirely different purpose.

The quickest way to understand the API is to experiment with a live example. Open the WebRTC samples getUserMedia demo in your browser, then open the console and inspect the stream variable, which is in global scope.

Every MediaStream has an input—often a stream from getUserMedia()—and an output, which can be attached to a video element or passed to an RTCPeerConnection. The getUserMedia() method accepts a MediaStreamConstraints parameter and returns a Promise that resolves to a MediaStream.

Each stream has a unique label (for example, 'Xk7EuLhsuHKbnjLWkW4yYGNJJ8ONsgwHBvLQ'). The methods getAudioTracks() and getVideoTracks() return an array of MediaStreamTrack objects. In the basic getUserMedia sample with a webcam, stream.getAudioTracks() returns an empty array, while stream.getVideoTracks() returns a single track with a kind of 'video' and a label such as 'FaceTime HD Camera (Built-in)'. Each track represents one or more channels of audio or video. Real-world apps might capture from front and rear cameras, a microphone, and a screen-sharing session simultaneously.

To display a stream, set the srcObject attribute on a video element. The older approach—using URL.createObjectURL() to set src—is deprecated. getUserMedia can also serve as an input node for the Web Audio API:

// Cope with browser differences.
let audioContext;
if (typeof AudioContext === 'function') {
  audioContext = new AudioContext();
} else if (typeof webkitAudioContext === 'function') {
  audioContext = new webkitAudioContext(); // eslint-disable-line new-cap
} else {
  console.log('Sorry! Web Audio not supported.');
}

// Create a filter node.
var filterNode = audioContext.createBiquadFilter();
// See https://dvcs.w3.org/hg/audio/raw-file/tip/webaudio/specification.html#BiquadFilterNode-section
filterNode.type = 'highpass';
// Cutoff frequency. For highpass, audio is attenuated below this frequency.
filterNode.frequency.value = 10000;

// Create a gain node to change audio volume.
var gainNode = audioContext.createGain();
// Default is 1 (no change). Less than 1 means audio is attenuated
// and vice versa.
gainNode.gain.value = 0.5;

navigator.mediaDevices.getUserMedia({audio: true}, (stream) => {
  // Create an AudioNode from the stream.
  const mediaStreamSource =
    audioContext.createMediaStreamSource(stream);
  mediaStreamSource.connect(filterNode);
  filterNode.connect(gainNode);
  // Connect the gain node to the destination. For example, play the sound.
  gainNode.connect(audioContext.destination);
});

Chromium-based apps and extensions can incorporate getUserMedia by adding audioCapture and/or videoCapture permissions to the manifest. Permission is then requested once at installation, and the user is not prompted again for camera or microphone access.

For regular web pages, permission is granted once per getUserMedia() call via an Allow button in the browser's infobar. Chrome deprecated HTTP access for getUserMedia() at the end of 2015, classifying it as a powerful feature that requires a secure origin.

The API is designed to be extensible beyond cameras and microphones. The intention is that any streaming data source—stored files, sensors, or arbitrary input—could eventually generate a MediaStream.

getUserMedia() becomes far more interesting when combined with other JavaScript APIs. Notable examples include Webcam Toy, a WebGL-based photobooth; FaceKat, a face-tracking game built with headtrackr.js; and ASCII Camera, which converts video to ASCII art via the Canvas API.

ASCII image generated by idevelop.ro/ascii-camera
gUM ASCII art!

Constraints

Constraints allow you to request specific values for video resolution, aspect ratio, facing mode (front or back camera), frame rate, height, and width. The applyConstraints() method can be used to update these after the stream has started. If you request a value that is not available—for example, a resolution the device doesn't support—the operation fails with a DOMException or OverconstrainedError. The WebRTC sample "getUserMedia: select resolution" demonstrates both valid and invalid constraint combinations.

Screen and tab capture

Chrome apps can share a live video of a single tab or the entire desktop using the chrome.tabCapture and chrome.desktopCapture APIs. It is also possible to use screen capture as a MediaStream source through the experimental chromeMediaSource constraint. Screen capture requires HTTPS and in some cases is enabled by a command-line flag, so it should only be used for development.

Signaling: Session control, network, and media information

WebRTC uses RTCPeerConnection to stream data directly between browsers, also called peers. However, before streaming can begin, the peers must exchange control messages and configuration details in a process called signaling. The WebRTC specification deliberately leaves signaling undefined—it is not part of the RTCPeerConnection API. Developers can choose any messaging protocol—SIP, XMPP, or something custom—over any duplex channel. The appr.tc example uses XHR and the Channel API; Google's codelab uses Socket.io on a Node server.

Signaling exchanges three types of information:

  • Session control messages, which initialize or close communication and report errors.
  • Network configuration: your computer's IP address and port as seen from the outside.
  • Media capabilities: which codecs and resolutions the browsers can handle.

All this exchange must complete successfully before peer-to-peer streaming begins. The following code sample from the W3C WebRTC spec illustrates the process between two hypothetical peers, Alice and Bob, assuming a createSignalingChannel() helper is already in place. Note that RTCPeerConnection is prefixed on Chrome and Opera.

// handles JSON.stringify/parse
const signaling = new SignalingChannel();
const constraints = {audio: true, video: true};
const configuration = {iceServers: [{urls: 'stun:stun.example.org'}]};
const pc = new RTCPeerConnection(configuration);

// Send any ice candidates to the other peer.
pc.onicecandidate = ({candidate}) => signaling.send({candidate});

// Let the "negotiationneeded" event trigger offer generation.
pc.onnegotiationneeded = async () => {
  try {
    await pc.setLocalDescription(await pc.createOffer());
    // Send the offer to the other peer.
    signaling.send({desc: pc.localDescription});
  } catch (err) {
    console.error(err);
  }
};

// Once remote track media arrives, show it in remote video element.
pc.ontrack = (event) => {
  // Don't set srcObject again if it is already set.
  if (remoteView.srcObject) return;
  remoteView.srcObject = event.streams[0];
};

// Call start() to initiate.
async function start() {
  try {
    // Get local stream, show it in self-view, and add it to be sent.
    const stream =
      await navigator.mediaDevices.getUserMedia(constraints);
    stream.getTracks().forEach((track) =>
      pc.addTrack(track, stream));
    selfView.srcObject = stream;
  } catch (err) {
    console.error(err);
  }
}

signaling.onmessage = async ({desc, candidate}) => {
  try {
    if (desc) {
      // If you get an offer, you need to reply with an answer.
      if (desc.type === 'offer') {
        await pc.setRemoteDescription(desc);
        const stream =
          await navigator.mediaDevices.getUserMedia(constraints);
        stream.getTracks().forEach((track) =>
          pc.addTrack(track, stream));
        await pc.setLocalDescription(await pc.createAnswer());
        signaling.send({desc: pc.localDescription});
      } else if (desc.type === 'answer') {
        await pc.setRemoteDescription(desc);
      } else {
        console.log('Unsupported SDP type.');
      }
    } else if (candidate) {
      await pc.addIceCandidate(candidate);
    }
  } catch (err) {
    console.error(err);
  }
};

First, Alice and Bob exchange network information. The phrase finding candidates refers to the process of discovering network interfaces and ports using the ICE framework:

  1. Alice creates an RTCPeerConnection with an onicecandidate handler, which fires as network candidates become available.
  2. Alice sends serialized candidate data to Bob over their chosen signaling channel.
  3. Bob calls addIceCandidate to add each candidate to his remote peer description.

Peers must also agree on audio and video configuration, such as resolution and codec choices. This is handled through an offer and answer exchange using the Session Description Protocol (SDP):

  1. Alice calls createOffer() to generate an RTCSessionDescription—her local session description.
  2. Alice sets it as the local description with setLocalDescription() and sends it to Bob through the signaling channel. Candidates are not gathered until this call is made, per the JSEP IETF draft.
  3. Bob receives the description and sets it as the remote description with setRemoteDescription().
  4. Bob calls createAnswer(), passing the remote description, to generate a compatible local session. He receives an RTCSessionDescription in the callback, sets it as his local description, and sends it back to Alice.
  5. Alice sets Bob's answer as the remote description with setRemoteDescription().
  6. At this point, streaming can begin.

RTCSessionDescription objects conform to SDP. A serialized SDP object typically looks like this:

v=0
o=- 3883943731 1 IN IP4 127.0.0.1
s=
t=0 0
a=group:BUNDLE audio video
m=audio 1 RTP/SAVPF 103 104 0 8 106 105 13 126

// ...

a=ssrc:2223794119 label:H4fjnMzxy3dPIgQ7HxuCTLb4wLLLeRHnFxh810

The exchange of network and media information can happen in parallel, but neither is complete until the other is done; only then can direct streaming commence. This offer/answer architecture is known as the JavaScript Session Establishment Protocol, or JSEP.

JSEP architecture diagram
JSEP architecture

Once signaling has finished, data flows directly between peers via RTCPeerConnection—or, if a direct connection fails, through an intermediate relay server. Streaming is entirely the job of RTCPeerConnection.

Inside RTCPeerConnection

RTCPeerConnection is the WebRTC component that manages stable, efficient streaming data between peers. On the surface, the JavaScript API appears simple — create a connection, add tracks, set local and remote descriptions. Everything else is abstracted away:

WebRTC architecture diagram
WebRTC architecture (from webrtc.org)

What the API hides is substantial. The codecs and protocols underneath do heavy lifting to keep real-time communication viable even over unreliable networks:

  • Packet-loss concealment
  • Echo cancellation
  • Bandwidth adaptivity
  • Dynamic jitter buffering
  • Automatic gain control
  • Noise reduction and suppression
  • Image-cleaning

A connection without servers

The WebRTC samples Peer connection demonstrates how RTCPeerConnection works in its simplest form. Both caller (pc1) and callee (pc2) exist on the same page, so they exchange data and signaling messages directly — no intermediary mechanism required.

On the caller side, a new RTCPeerConnection is created and tracks from getUserMedia() are added. The code then creates an offer, sets it as pc1's local description, and directly sets it as pc2's remote description:

  1. Create pc1 and add the local stream's tracks: js pc1 = new RTCPeerConnection(servers); localStream.getTracks().forEach((track) => { pc1.addTrack(track, localStream); });
  2. Create an offer and exchange descriptions between peers: js pc1.setLocalDescription(desc).then(() => { onSetLocalSuccess(pc1); }, onSetSessionDescriptionError ); trace('pc2 setRemoteDescription start'); pc2.setRemoteDescription(desc).then(() => { onSetRemoteSuccess(pc2); }, onSetSessionDescriptionError );

The callee side creates pc2 and, when the incoming stream arrives via ontrack, displays it in a video element:

js pc2 = new RTCPeerConnection(servers); pc2.ontrack = gotRemoteStream; function gotRemoteStream(e){ vid2.srcObject = e.stream; }

When servers are required

Real-world WebRTC needs servers to handle the steps that peers cannot perform alone:

  • User discovery and name exchange
  • Signaling — network, media format, and resolution data
  • NAT and firewall traversal
  • Relay when direct peer-to-peer connections fail

NAT traversal is handled by the STUN protocol and its relay extension, TURN, both used within the ICE framework. ICE first tries to connect peers directly via UDP with the lowest possible latency. STUN servers have a single job here: to reveal a peer's public address and port behind a NAT.

Finding connection candidates
Finding connection candidates

If UDP fails, ICE tries TCP. If a direct connection still can't be established — typical with enterprise firewalls and NATs — ICE falls back to a TURN relay server. The phrase finding candidates describes the process of discovering network interfaces and ports. A detailed walkthrough of STUN and TURN server implementations is available in the backend services guide, and WebRTC engineer Justin Uberti covers ICE, STUN, and TURN in his 2013 Google I/O presentation.

WebRTC data pathways
WebRTC data pathways

A practical video chat example

The appr.tc demo provides a functional video chat client complete with signaling and STUN-based NAT traversal. It uses adapter.js, a shim that insulates apps from spec changes and prefix differences. The code logs extensively — check the browser console to observe the order of operations.

Beyond one-to-one

WebRTC implementations currently support only one-to-one communication directly, but more complex topologies are possible. Multiple peers can interconnect directly, or through a Multipoint Control Unit (MCU) — a server that handles large numbers of participants, performs selective stream forwarding, and can mix or record audio and video.

Multipoint Control Unit topology diagram
Multipoint Control Unit topology example

Gateway servers extend reach beyond browsers. In May 2012, Doubango Telecom open sourced sipml5, a SIP client built with WebRTC and WebSocket enabling video calls between browsers and iOS/Android apps. At Google I/O, Tethr and Tropo demonstrated a portable disaster communication framework using an OpenBTS cell, connecting feature phones and computers via WebRTC.

Tethr/Tropo demo at Google I/O 2012
Tethr/Tropo: Disaster communications in a briefcase

Exchanging arbitrary data with RTCDataChannel

WebRTC extends beyond audio and video. The RTCDataChannel API provides peer-to-peer exchange of arbitrary data with low latency and high throughput. Typical applications include:

  • Gaming
  • Remote desktop tools
  • Real-time text chat
  • File transfer
  • Decentralized networks

The API's design maximizes the capabilities of RTCPeerConnection:

  • Leverages RTCPeerConnection session setup
  • Multiple simultaneous channels with prioritization
  • Both reliable and unreliable delivery semantics
  • Built-in security via DTLS and congestion control
  • Works with or without accompanying audio or video

Its syntax deliberately tracks WebSocket: a send() method and a message event:

const localConnection = new RTCPeerConnection(servers);
const remoteConnection = new RTCPeerConnection(servers);
const sendChannel =
  localConnection.createDataChannel('sendDataChannel');

// ...

remoteConnection.ondatachannel = (event) => {
  receiveChannel = event.channel;
  receiveChannel.onmessage = onReceiveMessage;
  receiveChannel.onopen = onReceiveChannelStateChange;
  receiveChannel.onclose = onReceiveChannelStateChange;
};

function onReceiveMessage(event) {
  document.querySelector("textarea#send").value = event.data;
}

document.querySelector("button#send").onclick = () => {
  var data = document.querySelector("textarea#send").value;
  sendChannel.send(data);
};

Since communication goes directly between browsers, RTCDataChannel can outperform WebSocket, even when a TURN relay is needed after hole-punching fails against firewalls and NATs.

The API is available in Chrome, Safari, Firefox, Opera, and Samsung Internet. The Cube Slam game uses it to synchronize game state, while Sharefest enabled file sharing and peerCDN demonstrated peer-to-peer content distribution. The IETF maintains the protocol draft spec.

Security by design

Real-time communication introduces specific security risks:

  • Unencrypted media or data intercepted in transit
  • Unauthorized recording and distribution of audio or video
  • Malware hidden in plugins or apps

WebRTC's architecture addresses these concerns directly:

  • Secure protocols are mandatory, including DTLS and SRTP
  • Encryption applies to all components, including signaling
  • No plugin architecture — components run inside the browser sandbox and receive automatic updates with the browser
  • Camera and microphone access require explicit user consent, with visible indicators during use

The IETF's proposed security architecture provides a more detailed treatment.

Developer tooling

  • Live session statistics are available at:
    • about://webrtc-internals in Chrome
    • opera://webrtc-internals in Opera
    • about:webrtc in Firefox
chrome://webrtc-internals page
chrome://webrtc-internals screenshot

References and further reading

For a broader technical grounding in WebRTC, the following resources are widely used by the community:

Standards and protocol specifications

The WebRTC ecosystem is defined by a set of evolving W3C and IETF documents. Key specifications include:

Browser support landscape

Support for the main WebRTC APIs has matured across browsers, though the specific versions and levels of stability vary.

MediaStream and getUserMedia API support

These APIs are broadly available. Chrome has supported them since desktop 18.0.1008 and Chrome for Android 29. Opera joined with version 18 (and Opera for Android 20), while early support also existed in Opera 12 and Opera Mobile 12 based on the Presto engine. Firefox has included support since version 17. On the Microsoft side, Edge supports it from version 16. Safari has supported it since iOS 11.2 and macOS 11.1. UC Browser for Android (11.8+) and Samsung Internet (4+) also offer support.

RTCPeerConnection API support

The RTCPeerConnection API is enabled by default in Chrome since desktop 20 and Chrome for Android 29. It is on by default in Opera 18+ and Opera for Android 20+, as well as in Firefox 22+. Microsoft Edge supports it from version 16. Safari's support begins with iOS 11.2 and macOS 11.1. Samsung Internet also includes it from version 4.

RTCDataChannel API support

Support for RTCDataChannel has evolved separately. Chrome 25 offered an experimental version; Chrome 26 and higher (and Chrome for Android 29+) provide a more stable implementation with Firefox interoperability. Opera's stable version arrived with Opera 18 and higher (Opera for Android 20+), also interopable with Firefox. Firefox has supported the API by default since version 22.

For up-to-date, cross-browser compatibility details, consult caniuse.com and Chrome Platform Status. Native RTCPeerConnection APIs for non-browser applications are documented on webrtc.org.