Streamline: custom video pipelines with Cloudflare Stream and Workers

A production media pipeline needs more than a request handler. Video streams run for minutes or hours, so the processing environment must be long-lived, predictable in memory and CPU, and able to run compiled, specialized code. Starting, feeding, inspecting and stopping a pipeline should not require holding one request open for the whole session.

Cloudflare's primitives map onto those needs: Containers provide the long-running runtime for media processing, Durable Objects handle orchestration, and Workers are used for control signaling and monitoring. Streamline is a developer playground built on that combination. Its media engine runs in a Container and is controlled by a Worker; processing continues even if the controlling Worker disconnects. The components are modular, so the media engine could later be swapped for dedicated encoding products.

Every Streamline deployment has two halves. The Media Engine performs media input/output and processing. The controlling Application creates, configures, observes and stops media sessions.

The Media Engine is itself split in two:

• A Controller — a Go control harness exposing an HTTP server. It receives requests and translates them into operations for the engine.
• A Processor that does the actual work. The current implementation is FFmpeg, but that is an internal detail rather than part of the user-facing API.

Hosted in a Container, the engine handles all input, output and processing. It can pull RTMPS playback from one Stream Live input and publish RTMPS to another. It can pull a Cloudflare Stream HLS manifest and its segments to use a hosted video as input. It can accept video supplied by the controlling application, such as a webcam. Preview video can be published over an outbound WebSocket to a Durable Object relay, which an application connects to through its own WebSocket.

The Application

The application is built with Workers and can be a full-stack browser app, an agent, or an embedded system. It contains a UI (client logic, identity and access policy, and — in the browser case used as this post's example — a browser interface) and an Orchestrator, implemented as a Durable Object, which coordinates the session, the Container lifecycle and the preview relay.

Local development skips the Durable Object entirely: the container is a local Docker instance, there is a single user, no authorization is required, and preview connects directly to a WebSocket on localhost. Deployed to Cloudflare, an authorized user or agent visits the Worker to start a session, which spins up a Streamline container if needed, manages its lifecycle, exposes an API for video manipulation operations, and routes inputs and outputs to and from Cloudflare Stream.

Container lifecycle and session management

A session is long-running by design. After the controlling Worker starts it, that application can disconnect and reconnect while the Container keeps processing until explicitly stopped. A maximum duration is enforced so a session always shuts down eventually, even with no external control. While a session runs, the container instance is unavailable to other applications.

Automatic sleeping would normally stop an idle container, but a running pipeline must survive with no incoming requests. Streamline overrides the onActivityExpired() callback: if the expiry time has not been reached, activity is renewed; otherwise the container is destroyed.

async onActivityExpired() {
  await this.withControlLock(async () => {
    const session = await this.getRelaySessionLocked()

    if (session?.expiresAt) {
      this.renewActivityTimeout()
      return
    }

    await this.destroy()
  })
}

The API surface

The Go harness's HTTP server and the Container's Durable Object together form the low-level interface. Streamline layers an abstraction over it so the system stays agnostic to who or what controls a session and to backend details. Two packages are exported:

• @cloudflare/streamline/client — a high-level, session-based API.
• @cloudflare/streamline/ — the Durable Object base class associated with the container, which routes API requests, implements the preview relay server, and provides hooks for security and access policy.

In a remote deployment the controlling Worker imports @streamline/cloudflare and defines a concrete subclass of the exposed Durable Object for application-specific logic and storage. In local mode, with no Durable Object, the frontend supplies a thin adapter that preserves the session-based API but talks directly to the local Docker instance, without access controls.

const streamline = createStreamline({ baseUrl: 'https://media.example' })
const session = await streamline.sessions.create()
const result = await session.start(config)

// At this point the pipeline is running, unless failure occurred.
console.log(result)

Here config is the JSON object describing the processing pipeline.

Defining a pipeline

session.start() constructs and runs a pipeline from a single JSON configuration argument covering inputs, operations and output.

A typical pipeline takes an RTMP broadcast as input (for example a Stream Live input receiving an inbound livestream), applies an overlay image with transparency, and sends the result to an RTMP destination such as another Stream Live input for recording or broadcast — producing a modified livestream in real time.

const session = await streamline.sessions.create()

const result = await session.start({
  input: { type: 'rtmp', profile: 'primary-input' },
  pipeline: [
    {
      op: 'overlay',
      params: { image: '/app/assets/cf-logo.png', position: 'top-right' },
    },
    {
      op: 'encode',
      params: {
        codec: 'h264',
        preset: 'fast',
        bitrate: '1500k',
        resolution: '1280x720',
        fps: 30,
      },
    },
  ],
  output: { mode: 'rtmp', profile: 'primary-output' },
})

Hosted video via HLS

Input can also arrive as HLS. A pipeline can ingest a video hosted on Cloudflare Stream, read its embedded closed-caption subtitles and render them as burned-in text, then output over RTMP to a Stream Live input for broadcasting or recording the modified version.

const streamVideoId = 'your-cloudflare-stream-video-id'
const session = await streamline.sessions.create()

await session.start({
  input: {
    type: 'hls',
    url: `https://videodelivery.net/${streamVideoId}/manifest/video.m3u8`,
  },
  pipeline: [
    { op: 'subtitle', params: { source: 'auto' } },
    {
      op: 'encode',
      params: {
        codec: 'h264',
        preset: 'fast',
        bitrate: '1500k',
        resolution: '1280x720',
        fps: 30,
      },
    },
  ],
  output: { mode: 'rtmp', profile: 'default' },
})

Direct ingest and animated overlays

Sending video straight to Streamline is useful for quick pipeline previews — from a webcam, say — and for agents or embedded devices: factory cameras feeding AI analysis, or several camera feeds combined into a composite view.

A pipeline can expect input from the Worker application and emit preview video over a WebSocket. The example applies two filters and an annotation: a PNG overlay that can be updated while processing runs, enabling animated graphics.

const session = await streamline.sessions.create()
const sessionId = session.id
if (!sessionId) throw new Error('Session creation returned no session ID')

const viewer = await openViewer('https://media.example', sessionId)

await session.start({
  input: { type: 'webcam' },
  pipeline: [
    { op: 'filter', params: { preset: 'brightness', amount: 0.1 } },
    { op: 'filter', params: { preset: 'flip' } },
    { op: 'overlay', params: { image: 'annotation', position: 'full' } },
    {
      op: 'encode',
      params: {
        codec: 'h264',
        preset: 'veryfast',
        bitrate: '1500k',
        resolution: '1280x720',
        fps: 30,
        gop: 60,
      },
    },
  ],
  output: { mode: 'websocket', format: 'fmp4' },
})

Starting the pipeline sends no media yet; session.ingest() is what feeds it. A browser application can receive chunks from the webcam and forward them this way.

const stream = await navigator.mediaDevices.getUserMedia({ video: true, audio: true })
const recorder = new MediaRecorder(stream, { mimeType: 'video/webm;codecs=vp8,opus' })
let uploadTail = Promise.resolve()

recorder.addEventListener('dataavailable', (event) => {
  if (event.data.size === 0) return
  uploadTail = uploadTail
    .then(() => session.ingest(event.data))
    .catch((error) => reportUploadFailure(error))
})

recorder.start(250)

The annotation is updated with session.annotation(), for example by snapshotting a canvas and sending it. Running this on an animation loop is possible, though the practical update rate depends on PNG overlay size, available bandwidth and processing power.

function canvasPng(canvas: HTMLCanvasElement): Promise<Blob> {
  return new Promise((resolve, reject) => {
    canvas.toBlob((blob) => {
      if (blob) resolve(blob)
      else reject(new Error('Canvas could not produce a PNG'))
    }, 'image/png')
  })
}

const png = await canvasPng(overlayCanvas)
await session.annotation(png)

Preview over WebSocket

Setting output: { mode: 'websocket' } produces preview video. For low latency, the container publishes fMP4 fragments to the Durable Object, which forwards them to an output relay reachable at /relay/view relative to the application origin. The application connects a WebSocket there and receives video as it becomes available.

function openViewer(origin: string, sessionId: string): Promise<WebSocket> {
  const url = new URL('/relay/view', origin)
  url.protocol = url.protocol === 'https:' ? 'wss:' : 'ws:'
  url.searchParams.set('session_id', sessionId)

  return new Promise((resolve, reject) => {
    const socket = new WebSocket(url)
    socket.binaryType = 'arraybuffer'
    socket.addEventListener('open', () => resolve(socket), { once: true })
    socket.addEventListener('error', () => reject(new Error('Preview relay failed')), { once: true })
  })
}

const sessionId = session.id
if (!sessionId) throw new Error('Session creation returned no session ID')

// Connect before session.start(), or the relay rejects the publisher.
const viewer = await openViewer('https://media.example', sessionId)

viewer.addEventListener('message', (event) => {
  if (typeof event.data === 'string') {
    if (event.data === '{"type":"eos"}') mediaSource.endOfStream()
    return
  }
  sourceBuffer.appendBuffer(new Uint8Array(event.data as ArrayBuffer))
})

In production a MediaSource player must queue fragments while SourceBuffer.updating is true. During local development the browser or other controlling application simply opens a WebSocket connection directly on the local container.

Supported operations

pipeline is an array of operations supported by the underlying media engine. Operation order is currently fixed by the engine — the order within the array carries no meaning.

Security

Security is part of the design rather than an add-on. Only authorized users may create a session or take control of an existing one; sessions are isolated from each other; Stream RTMPS input and output keys are treated as secrets never leaked to the controlling application; and resource use is bounded.

The owner deployment stays private through Workers' Access integration. The configured owner identity and other allowed users can edit shared profiles and start a session while the singleton is idle. The Worker verifies the Access session before accepting control requests and binds the active session to the verified principal. Only one session runs at a time, and a different principal cannot stop or replace the active one.

Stream Live input keys live in Worker secrets or as write-only shared overrides in Durable Object storage; the settings API never returns them and they are never placed in browser storage. The controlling application refers to RTMPS input and output by named profile, and the Worker resolves that profile before contacting the container.

Preview uses two credentials with distinct purposes: a Cloudflare Access service token authenticating the container workload to the publisher endpoint, and a random per-session capability authorizing publishing only for the currently active relay. The service token is injected by the container's outbound Worker and never enters container memory. Initially the deployment uses a temporary path-specific Access Bypass while the per-session capability remains enforced; after deployment and a successful smoke test, Service Auth replaces Bypass.

This owner deployment is intentionally private and singleton-routed; it is not the security model for a public multi-user service.

Open source release and hosted playground

Streamline ships as open source alongside a public playground deployment. The container can be run locally or deployed into your own account, and it exports the Worker API for a control application to consume.

An example Worker application with an Astro web frontend exercises overlays, subtitle decoding, filters and picture-in-picture. Its probe functionality surfaces performance metrics and system tracing, which helps when debugging the system during new feature work. The example runs on a local Astro server or behind Cloudflare Access, so access to a Streamline instance can be restricted.

Both repositories live on Cloudflare's GitHub:

• https://github.com/cloudflare/streamline — media engine and package exports for applications.
• https://github.com/cloudflare/streamline-demo — example application Worker, Astro frontend, deployment profiles and Access tooling.

The published playground deployment of the example application is also something users can deploy themselves. It relies on its own Access configuration, one container identity per verified user, one active session per user, global admission control, concurrency, media and session limits, and prevents one user from replacing another user's session.

The public playground is available at:

• https://playground.streamline-video.workers.dev

Limits and next steps

Streamline is one demonstration of pairing managed services such as Stream with lower-level primitives to assemble highly customizable media pipelines. In this iteration, media processing runs on Container CPU, which becomes a bottleneck at higher qualities or frame rates.

Future work the team wants to explore with the developer community includes computer vision pipelines, hardware-accelerated media processing, realtime experiences over next-generation protocols like WebRTC and MoQ, and eventually video encoding and decoding primitives native to Workers.

The hosted demo shows what the tooling can do today; the open source codebases show how to deploy Streamline into your own account and build your own experiences on it.