A Distributed Take on Live Video Ingest
Cloudflare’s Stream Live service, generally available since 2022, is built less like a traditional centralized media platform and more like an extension of Cloudflare’s own edge network. The design goal is straightforward: keep the point where a broadcaster’s content enters the network close to that broadcaster, and keep the point where viewers receive content close to those viewers. That distributed posture drives almost every architectural choice in the system.
For ingest, broadcasters connect to an Anycast IP address advertised from every Cloudflare data center in more than 330 cities. Whether they use a custom ingest domain or the default live.cloudflare.com, the connection lands on the nearest available server. The protocol used—RTMPS, SRT, or WHIP—determines how frames are packaged, but the path into the network is the same.
That path runs through a Spectrum application first. Spectrum provides TLS termination for RTMPS, absorbs DDoS traffic, and solves a particular problem for SRT: many SRT clients don't support the protocol's Stream ID feature and assume that a unique port identifies their connection. Spectrum exposes many listening ports that map to specific broadcasters, wraps each connection in Simple Proxy Protocol, and forwards everything to a single ingest service port. This matters because the ingest service spends meaningful CPU per listening port, while Spectrum spends effectively none.
State and the "Volatile" Gap
The ingest service, written in Go, receives content, forwards it to live outputs, and records it for playback. It acts on configuration and state held in Durable Objects—one Durable Object instance per live input. Those objects store both the customer configuration (everything set when creating or modifying a live input) and the canonical ongoing state of each broadcast.
State is split into two categories. Volatile state is content the ingest has received but not yet acknowledged to the Durable Object. Committed state is content the Durable Object has indexed and confirmed. The committed state acts as the sync point whenever a new ingest service instance must take over a broadcast.
This split exists because broadcasters can disconnect and reconnect at any time—a network blip on the broadcaster's side is normal. With centralized ingest and state storage, resuming is simple: every connection hits the same server. With Anycast, a reconnect can land on a completely different server in a different city. The Durable Object ensures that whichever server receives the reconnect has the data and knowledge to continue the broadcast seamlessly.
Cleaning the Stream and Storing It
Inside the ingest service, the relay is implemented as two coupled loops communicating over a Go channel—one for sending, one for receiving. Incoming packets are normalized based on protocol; for example, converting delimiter-based annex B H.264 NALUs to length-prefixed avcc format, or vice versa, depending on what the downstream consumer expects.
When Live Playback and Recording is enabled, the service sanitizes packets further. Timestamps are issued monotonically, which resolves many encoder quirks, and severely misaligned audio/video blocks—common at the start of broadcasts—are dropped. The cleaned packets are packaged into fragmented MP4s on keyframe boundaries, creating what Cloudflare calls "original segments."
Those segments are the canonical copy of everything the customer uploaded. They're stored in R2 and reused when a live stream transitions to on-demand. This tight coupling is why live playback and recording can't be decoupled: serving live viewers and supporting later replay are built on the same original segment store, with the only extra cost being state management overhead.
Playlists as an Index
Most Stream Live viewing today happens through segmented delivery using HLS or MPEG-DASH. Both formats rely on playlists that act as an index. A primary, or multi-variant, playlist describes available renditions—1080p video, 720p, different bitrates, codecs—and points to each rendition's specific stream playlist. The stream playlist lists the individual segments that make up the timeline and can be appended to over time for live content, whereas an on-demand playlist includes a flag indicating no more content is coming.
Cloudflare generates both playlist formats from a single internal representation of tracks, renditions, and muxings. Since HLS and DASH contain nearly the same data arranged differently, that single representation is rendered into either newline-delimited M3U8 or XML-based MPD as needed. For live broadcasts, viewers re-fetch the playlist every 1–10 seconds.
#EXTM3U
#EXT-X-VERSION:6
#EXT-X-INDEPENDENT-SEGMENTS
#EXT-X-MEDIA:TYPE=AUDIO,GROUP-ID="group_audio",NAME="original",LANGUAGE="en-0a76e0ad",DEFAULT=YES,AUTOSELECT=YES,URI="stream_2.m3u8" <- Audio track description + URL path
...
#EXT-X-STREAM-INF:RESOLUTION=426x240,CODECS="avc1.42c015,mp4a.40.2",BANDWIDTH=149084,AVERAGE-BANDWIDTH=145135,SCORE=1.0,FRAME-RATE=30.000,AUDIO="group_audio" <- description of variant contents
stream_1.m3u8 <- URL path to fetch variantEncoding on Demand
Playlist and segment requests from viewers are handled by the delivery-worker, a Cloudflare Worker that has been in production since 2017. It handles all Stream content requests, renders playlists by transforming Durable Object broadcast state, and performs delivery logic.
When a client requests encoded media content described in the playlist, the delivery worker sends a subrequest to OTFE, an on-the-fly encoder running across the Cloudflare network. The request specifies the format—for example, the video stream of segment 89282 at 1280x720 using AVC with a bitrate cap. OTFE then encodes the original segment into the requested configuration.
This on-the-fly model is the opposite of always-encoding, which is standard elsewhere. If no one is watching a particular quality level, no encoding happens for it. That saves power, CPU, RAM, network, and cache space. The tradeoff is difficulty in two areas. Media-correctness requires each segment to have precise start times and durations, otherwise playback stutters or audio/video desyncs; most encoders are not designed for per-segment encoding, so output requires fine-tuning and spec-level adjustments. Performance is handled by aggressive prefetching: when a viewer requests Segment N, work starts on Segment N+1, and because the logic runs as a Worker, that behavior can be adjusted easily.
Network as the Multiplier
The overall flow is supported by tiered caching and request coalescing. The critical optimization is coalescing: for any number of viewers requesting the same encoded content, only one request reaches the encoder origin. This means multiple viewers are served without redundant encoding passes.
The entire platform, from the ingest relay to the delivery worker and on-demand encoder, runs on Cloudflare's own network infrastructure—the same products any customer can use. That self-hosting posture is what makes the distributed architecture, with edge-based ingest and edge-near delivery, possible in the first place.
