Build WebRTC app backend services

Signaling: The Coordination Layer WebRTC Leaves to You

WebRTC handles the peer-to-peer transport of audio, video, and data, but it deliberately says nothing about how two clients find each other and agree on how to talk. That coordination job—called signaling—is yours to build. Clients must exchange session-control messages, error reports, media metadata (codecs, bandwidth, media types), security keying material, and network addresses as seen from outside any NAT or firewall.

This isn't an oversight. The WebRTC standards bodies chose not to define a signaling protocol to avoid duplicating existing technologies and to keep browsers from having to maintain signaling state. If a page reloads and drops in-progress signaling data, that state can be recovered from a server instead. This design is outlined in the JavaScript Session Establishment Protocol (JSEP):

JSEP's architecture also avoids a browser having to save state, that is, to function as a signaling state machine. This would be problematic if, for example, signaling data was lost each time a page was reloaded. Instead, signaling state can be saved on a server.

The Offer/Answer Dance and ICE Candidates

Under JSEP, peers exchange offers and answers—media metadata in Session Description Protocol (SDP) format:

v=0
o=- 7614219274584779017 2 IN IP4 127.0.0.1
s=-
t=0 0
a=group:BUNDLE audio video
a=msid-semantic: WMS
m=audio 1 RTP/SAVPF 111 103 104 0 8 107 106 105 13 126
c=IN IP4 0.0.0.0
a=rtcp:1 IN IP4 0.0.0.0
a=ice-ufrag:W2TGCZw2NZHuwlnf
a=ice-pwd:xdQEccP40E+P0L5qTyzDgfmW
a=extmap:1 urn:ietf:params:rtp-hdrext:ssrc-audio-level
a=mid:audio
a=rtcp-mux
a=crypto:1 AES_CM_128_HMAC_SHA1_80 inline:9c1AHz27dZ9xPI91YNfSlI67/EMkjHHIHORiClQe
a=rtpmap:111 opus/48000/2
...

That SDP text is not sacrosanct. WebRTC lets you edit the SDP before setting it as the local or remote description—the preferAudioCodec() function in appr.tc, for example, adjusts the default codec and bitrate this way. Manipulating SDP in JavaScript is awkward, and there's ongoing discussion about switching to JSON in future versions, but SDP has its advantages and is the current standard.

The RTCPeerConnection API drives the mechanism. It has two jobs: determine local media capabilities (the metadata behind the offer/answer), and discover potential network addresses (candidates) that the remote peer might use to reach it. Both jobs require a signaling channel to communicate results to the other side.

The full flow, from Alice to Eve:

Alice creates an RTCPeerConnection object.
Alice creates an offer (an SDP session description) with createOffer().
Alice calls setLocalDescription() with that offer.
Alice stringifies the offer and sends it to Eve over the signaling mechanism.
Eve calls setRemoteDescription() with Alice's offer so her RTCPeerConnection knows Alice's setup.
Eve calls createAnswer(); the success callback receives her local session description.
Eve calls setLocalDescription() with her answer.
Eve sends the stringified answer back to Alice.
Alice calls setRemoteDescription() with Eve's answer.

Then network discovery: ICE (Interactive Connectivity Establishment) finds network interfaces and ports, firing the onicecandidate handler. Each candidate must be sent to the remote peer and added there with addIceCandidate(). JSEP supports ICE Candidate Trickling, so the caller can incrementally send candidates after the initial offer, letting the callee start acting without waiting for the full set.

// handles JSON.stringify/parse
const signaling = new SignalingChannel();
const constraints = {audio: true, video: true};
const configuration = {iceServers: [{urls: 'stun:stun.example.org'}]};
const pc = new RTCPeerConnection(configuration);

// Send any ice candidates to the other peer.
pc.onicecandidate = ({candidate}) => signaling.send({candidate});

// Let the "negotiationneeded" event trigger offer generation.
pc.onnegotiationneeded = async () => {
  try {
    await pc.setLocalDescription(await pc.createOffer());
    // send the offer to the other peer
    signaling.send({desc: pc.localDescription});
  } catch (err) {
    console.error(err);
  }
};

// After remote track media arrives, show it in remote video element.
pc.ontrack = (event) => {
  // Don't set srcObject again if it is already set.
  if (remoteView.srcObject) return;
  remoteView.srcObject = event.streams[0];
};

// Call start() to initiate.
async function start() {
  try {
    // Get local stream, show it in self-view, and add it to be sent.
    const stream =
      await navigator.mediaDevices.getUserMedia(constraints);
    stream.getTracks().forEach((track) =>
      pc.addTrack(track, stream));
    selfView.srcObject = stream;
  } catch (err) {
    console.error(err);
  }
}

signaling.onmessage = async ({desc, candidate}) => {
  try {
    if (desc) {
      // If you get an offer, you need to reply with an answer.
      if (desc.type === 'offer') {
        await pc.setRemoteDescription(desc);
        const stream =
          await navigator.mediaDevices.getUserMedia(constraints);
        stream.getTracks().forEach((track) =>
          pc.addTrack(track, stream));
        await pc.setLocalDescription(await pc.createAnswer());
        signaling.send({desc: pc.localDescription});
      } else if (desc.type === 'answer') {
        await pc.setRemoteDescription(desc);
      } else {
        console.log('Unsupported SDP type.');
      }
    } else if (candidate) {
      await pc.addIceCandidate(candidate);
    }
  } catch (err) {
    console.error(err);
  }
};

That W3C example presumes a SignalingChannel exists—the snippet shows the offer/answer and candidate exchanges but not how the channel itself is implemented. For a working reference, the console log at simpl.info RTCPeerConnection traces the whole exchange in a single-page video chat. In Chrome or Opera, about://webrtc-internals (or opera://webrtc-internals) provides a complete dump of signaling messages and stats.

Peer Discovery: The "Who Do I Call?" Problem

Even with signaling in place, clients still need to know the other party exists. Telephony solves this with numbers and directories; WebRTC apps need identity, presence, and a way to initiate a session. The mechanism is not defined by WebRTC: it can be as simple as sharing a URL by email or IM. Video chat services like Talky, tawk.to, and Browser Meeting all operate this way—you invite someone by sending them a link. Developer Chris Ball even built a serverless-webrtc experiment where call metadata is exchanged through whatever messaging service the two parties prefer, including deliberately non-technical ones.

Signaling versus the request-response model

WebRTC does not define a signaling protocol, so the service you build must act as a relay between clients. The fundamental constraint is that a browser cannot initiate a connection to another browser directly; a server must mediate. The good news is that the signaling workload is modest: an appr.tc video session exchanges about 30-45 messages totaling roughly 10KB. The service must also retain only minimal session state, such as which clients are connected. This makes bandwidth, CPU, and memory demands low on a per-client basis. The real engineering challenge is pushing messages from server to client, which contradicts the HTTP request-response model.

Delivering server-to-client messages

Several techniques exist for server push:

EventSource (server-sent events) allows a server to push data over HTTP to a client. It is one-way, but you can pair it with XHR requests to build a bidirectional signaling service: the caller sends a message to the server via XHR, and the server pushes it to the callee via EventSource.
WebSocket provides true full-duplex communication. Implementing a signaling service with raw WebSocket or EventSource has the advantage of being achievable on common web frameworks for PHP, Python, or Ruby. Every browser that supports WebRTC supports WebSocket. You should use TLS for all connections to prevent interception of unencrypted messages and to reduce issues with proxy traversal. Ilya Grigorik's High Performance Browser Networking explains the proxy issues in detail.
Ajax polling is a fallback where clients repeatedly query the server. It works, but generates superfluous network requests, especially after a session is established, when peers still need to check for session changes or teardown. The WebRTC Book reference app uses this approach with polling frequency optimizations.

Designing for scale

A single signaling server can handle modest traffic, but a popular app needs servers that support high concurrency. Several suitable architectures exist, each with trade-offs:

XMPP was developed for instant messaging and can be used for signaling. Server implementations include ejabberd and Openfire. JavaScript clients like Strophe.js rely on BOSH to emulate bidirectional streaming, but BOSH often scales less efficiently than WebSocket.
Message queue libraries such as ZeroMQ (used inside TokBox's Rumour service) and OpenMQ provide high-throughput messaging. NullMQ applies ZeroMQ concepts to browsers by using the STOMP protocol atop WebSocket.
Commercial cloud-messaging platforms such as Pusher and Kaazing typically use WebSocket with long-polling fallback.
Commercial WebRTC platforms such as vLine offer signaling as part of a managed service.

Example service: Socket.io on Node

The following example shows a signaling service written with Socket.io on Node. Socket.io is a convenient choice because it wraps WebSocket with fallbacks (AJAX long polling, multipart streaming, Forever Iframe, JSONP polling) and introduces a room concept directly suited to call groups. The sample is meant for a small number of users, not production load. It omits WebRTC entirely so you can see how the signaling layer works. As clients join a room and send messages, the console log shows the activity.

Client index.html:

<!DOCTYPE html>
<html>
  <head>
    <title>WebRTC client</title>
  </head>
  <body>
    <script src='/socket.io/socket.io.js'></script>
    <script src='js/main.js'></script>
  </body>
</html>

Client logic in main.js:

const isInitiator;

room = prompt('Enter room name:');

const socket = io.connect();

if (room !== '') {
  console.log('Joining room ' + room);
  socket.emit('create or join', room);
}

socket.on('full', (room) => {
  console.log('Room ' + room + ' is full');
});

socket.on('empty', (room) => {
  isInitiator = true;
  console.log('Room ' + room + ' is empty');
});

socket.on('join', (room) => {
  console.log('Making request to join room ' + room);
  console.log('You are the initiator!');
});

socket.on('log', (array) => {
  console.log.apply(console, array);
});

Server app:

const static = require('node-static');
const http = require('http');
const file = new(static.Server)();
const app = http.createServer(function (req, res) {
  file.serve(req, res);
}).listen(2013);

const io = require('socket.io').listen(app);

io.sockets.on('connection', (socket) => {

  // Convenience function to log server messages to the client
  function log(){
    const array = ['>>> Message from server: '];
    for (const i = 0; i < arguments.length; i++) {
      array.push(arguments[i]);
    }
      socket.emit('log', array);
  }

  socket.on('message', (message) => {
    log('Got message:', message);
    // For a real app, would be room only (not broadcast)
    socket.broadcast.emit('message', message);
  });

  socket.on('create or join', (room) => {
    const numClients = io.sockets.clients(room).length;

    log('Room ' + room + ' has ' + numClients + ' client(s)');
    log('Request to create or join room ' + room);

    if (numClients === 0){
      socket.join(room);
      socket.emit('created', room);
    } else if (numClients === 1) {
      io.sockets.in(room).emit('join', room);
      socket.join(room);
      socket.emit('joined', room);
    } else { // max two clients
      socket.emit('full', room);
    }
    socket.emit('emit(): client ' + socket.id +
      ' joined room ' + room);
    socket.broadcast.emit('broadcast(): client ' + socket.id +
      ' joined room ' + room);

  });

});

To run the app locally, you need Node.js, Socket.IO, and node-static. From the app directory, install dependencies with:

npm install socket.io
npm install node-static

Then launch the server:

node server.js

Open localhost:2013 in a browser, then repeat in a new tab. Open the browser console (in Chrome and Opera, use Ctrl+Shift+J or Command+Option+J on Mac) to observe the message exchange. Any production backend must furnish capabilities similar to this example.

Signaling pitfalls and shortcuts

When integrating signaling with WebRTC, keep these points in mind:

RTCPeerConnection does not begin gathering ICE candidates until setLocalDescription() is called. This behavior is mandated by the JSEP IETF draft.
Use Trickle ICE: call addIceCandidate() immediately as each candidate arrives rather than waiting for the full set.

Instead of building your own, you can adopt prebuilt signaling servers that integrate with WebRTC client libraries:

webRTC.io was an early abstraction layer for WebRTC.
Signalmaster is designed for the SimpleWebRTC JavaScript client.

Commercial platforms such as vLine, OpenTok, and Asterisk include hosted signaling. Also of historical note, Ericsson published an early Apache/PHP-based signaling server. Though now largely obsolete, its source is worth studying if you plan a similar design.

Securing the signaling channel

WebRTC mandates encryption for its own components. Because signaling is not standardized, its security is entirely your responsibility. A compromised signaling service can terminate sessions, redirect calls, or inject and alter content. The primary safeguard is to use secure transport — HTTPS and WSS (TLS) — so messages cannot be intercepted in plain text. Additionally, ensure your server does not broadcast messages in a way other callers sharing the same server can access.

Working with NATs and Firewalls

WebRTC signaling runs through an intermediary server, but once a session is established, RTCPeerConnection tries to stream media and data directly between peers. Ideally, every endpoint would have a unique address to exchange, but real-world networking is rarely that simple.

Peers behind NATs and firewalls — The real world

Most devices sit behind one or more NAT layers, and many also contend with antivirus software blocking certain ports, corporate proxies, and firewalls. A home WiFi router often combines firewall and NAT in one device. WebRTC handles these complications through the ICE framework, but the app must provide ICE server URLs to RTCPeerConnection.

ICE attempts all possible connection paths in parallel and selects the most efficient one that works. It first tries the host address from the device's OS and network card. If that fails, as it will behind a NAT, ICE obtains an external address from a STUN server. If a direct path still isn't possible, traffic gets relayed through a TURN server. In short, STUN discovers your public address; TURN relays traffic when peer-to-peer fails. Every TURN server supports STUN — a TURN server is essentially a STUN server with relaying built in.

Apps supply STUN and TURN server URLs via the iceServers configuration object passed to the RTCPeerConnection constructor. For appr.tc, that looks like this:

{
  'iceServers': [
    {
      'urls': 'stun:stun.l.google.com:19302'
    },
    {
      'urls': 'turn:192.158.29.39:3478?transport=udp',
      'credential': 'JZEOEt2V3Qb0y27GRntt2u2PAYA=',
      'username': '28224511:1379330808'
    },
    {
      'urls': 'turn:192.158.29.39:3478?transport=tcp',
      'credential': 'JZEOEt2V3Qb0y27GRntt2u2PAYA=',
      'username': '28224511:1379330808'
    }
  ]
}

From there, ICE handles the path-finding automatically, coordinating with STUN and TURN servers as needed.

STUN Servers

NATs assign devices an internal IP address that is useless externally. Without a public address, peers have nothing to connect to. STUN servers, which live on the public internet, solve this by inspecting the IP:port of an incoming request and returning that address to the sender. The WebRTC app learns its own public-facing address and passes it to another peer via signaling, establishing a direct link.

STUN servers are stateless and lightweight, so even modest hardware can handle large request volumes. Most WebRTC calls connect successfully using STUN alone, though calls between peers behind complex NATs or firewalls may need more help.

Peer to peer connection using a STUN server — Using STUN servers to get public IP:port addresses

TURN Servers

RTCPeerConnection first attempts UDP, then TCP. If both fail, it falls back to a TURN server, which relays the stream between endpoints. TURN only handles audio, video, and data — never signaling traffic.

TURN servers have public addresses, so peers behind firewalls and proxies can reach them. They consume substantial bandwidth because every byte of the stream passes through them.

Deploying STUN and TURN Servers

For testing, Google runs a public STUN server at stun.l.google.com:19302, which appr.tc uses. Production setups can use the rfc5766-turn-server, whose source code is available on GitHub along with installation guides and an AWS VM image. Another option is restund, available as source code and for AWS. Setup involves:

Open firewall ports as needed: tcp=443, udp/tcp=3478.
Create four instances, one per public IP, using Standard Ubuntu 12.06.
Allow ANY from ANY in the local firewall config.
Install build tools: sudo apt-get install make and sudo apt-get install gcc.
Install libre from creytiv.com/re.html.
Fetch and unpack restund from creytiv.com/restund.html.
Apply the auth patch with patch -p1 < restund-auth.patch.
Run make and sudo make install for both libre and restund.
Adapt restund.conf, replace IPs, ensure the shared secret matches, copy to /etc.
Copy restund/etc/restund to /etc/init.d/.
Set LD_LIBRARY_PATH, copy the config, and point it at the correct internal IP.
Run restund and test with the stund client: ./client IP:port.

Multi-Party Architectures

Many use cases — group video conferences or large public events — require more than one-to-one connections. A WebRTC app can create multiple RTCPeerConnection instances so every endpoint links to every other in a mesh. Apps like talky.io use this and it works fine for small groups, though CPU and bandwidth usage grow quickly beyond a handful of peers, especially on mobile.

Mesh: small N-way call — Full mesh topology: Everyone connected to everyone

Alternative topologies include a star config, where one endpoint distributes streams to all others, or running a WebRTC endpoint on a server for custom redistribution. Since a MediaStream from one RTCPeerConnection can feed another, apps can implement flexible call-routing by choosing which peer to connect to. The WebRTC samples relay and multiple connections pages demonstrate this.

Multipoint Control Units

For large deployments, a Multipoint Control Unit (MCU) is a better option. An MCU is a server-side bridge that distributes media among many participants. It handles different resolutions, codecs, and frame rates, can transcode, forward streams selectively, and mix or record audio and video. Multiparty calls introduce issues like displaying multiple video inputs and mixing audio sources. Cloud platforms such as vLine attempt to optimize routing as well.

MCU hardware can be purchased, but open source software options also exist. Licode (formerly Lynckia) provides an open source WebRTC MCU, and OpenTok offers Mantis.

Interoperating with SIP, Telephony, and Messaging

WebRTC's standardized nature allows browsers to communicate with devices using other protocols. For VoIP and video-conferencing systems, that often means SIP. A WebRTC app needs a proxy gateway to translate signaling between WebRTC and SIP. Once the session is up, SRTP traffic flows peer-to-peer without the gateway.

The PSTN requires a gateway for calls between WebRTC and analog phones. Likewise, communicating with Jingle endpoints (IM clients) requires an XMPP intermediary. Jingle — Google's XMPP extension for voice and video — became the basis for libjingle, the C++ library underlying current WebRTC implementations.

Several libraries and services build on this interoperability:

jsSIP: JavaScript SIP library
Phono: open source JavaScript phone API plugin
Twilio: voice and messaging platform