Captions, Without the Captioning Work
Cloudflare Stream now offers AI-generated captions for on-demand videos and live stream recordings, available to all Stream customers at no additional cost. The feature removes the need for third-party transcription services, specialized workflows, or manual transcription—a process that can be impractical for large video libraries and historically required dedicated teams or external vendors.
The entire captioning workflow is integrated directly into the Stream platform. Whether a video was uploaded yesterday or years ago, captions can be added from the Cloudflare Dashboard or via API request in the same unified interface used to manage other video settings. Privacy is handled entirely within Cloudflare's ecosystem; the company states it does not use customer content for model training, and content is not shared with external entities during transcription.
Starting with the Beta
The beta launched June 20th, 2024, and is available to all Stream customers as well as Professional and Business plan subscribers (which include 100 minutes of video storage). To get started:
- Upload a video to Stream through the Dashboard or via API.
- Open the "Captions" tab on the video and click "Add Captions."
- Choose the language and select "Generate captions with AI."
- Save the captions and wait a few minutes.
Once ready, captions appear in the captions manager and are automatically offered in the Stream player. HLS and DASH manifests are updated as well, so third-party players that support text tracks can display them.
Both on-demand videos and live stream recordings are supported regardless of when they were created. During beta, the feature generates English captions only, and videos must be shorter than two hours. Transcription quality is highest for content with clear speech and minimal background noise. While testing showed strong results across content types, the company advises users to verify caption accuracy for their specific use cases, as results may occasionally vary.
Built on Workers AI
The Stream engineering team built the feature using Workers AI, accessing the Whisper open-source Automatic Speech Recognition model through a single API call. This approach offloaded the significant infrastructure burden typically associated with AI deployments—configuring GPUs, managing workload distribution, minimizing latency, and maintaining availability. Workers AI handles the underlying scaling automatically, transforming the task into roughly 30 lines of Worker code for transcribing audio files.
Engineering for Scale
Processing a high volume of videos quickly required addressing Whisper's input requirements. Video content varies widely in format, duration, and audio quality, so the team wrote pre-processing logic to normalize audio before inference. Whisper performs optimally on 30-second audio clips; shorter clips lack contextual meaning for sentences, while longer clips require increasingly large models to maintain coherence. Stream leverages its existing video segmentation infrastructure, concatenating smaller playback segments into 30-second batches before sending them to Workers AI.
Processing speed comes from aggressive parallelization. The team concurrently creates audio batches and sends transcription requests, taking full advantage of Workers AI's scalability. This introduces a sequencing challenge: responses may arrive out of order, meaning the transcription for the second half of a video might complete before the first half. To handle this, the team tracks audio batch ordering and sorts incoming responses before assembling the final WebVTT caption file, ensuring timestamps align correctly with the video.
What's Next
The feature is now in open beta for all subscribers and Pro and Business plan customers. Documentation covers tutorials and current limitations, and Cloudflare indicates the roadmap includes support for additional languages and longer videos.
