A container platform built for Cloudflare’s network
Cloudflare has been running a new container platform in production for some time now. It currently powers Workers AI, Workers Builds, Remote Browsing Isolation, and the Browser Rendering API. If you have used any of those products, you have already run containers on Cloudflare’s network — without having to manage them yourself.
The platform exists because many of Cloudflare’s newer products need to run untrusted code outside of a v8 isolate. The requirement was to do so in a way that fits the core promise of the Workers platform: no need to think about regions or data centers, no need to be a distributed systems engineer, and no compromise on speed.
There was no off-the-shelf container platform that could meet those requirements. So Cloudflare built its own — from scheduling and IP address management to image caching and startup time optimization.
Scheduling across the network, not regions
Cloudflare’s architecture is built around the idea that the network is the computer. Traditional clouds expect developers to choose a region, a data center, or an availability zone. Cloudflare runs in 330+ cities across 120+ countries, and wants workloads to be able to run anywhere on that network — and to move when it makes sense.
Previously, Cloudflare ran every service via systemd on every server, or “metal,” in its fleet. That worked when the number of services was small — but it cannot scale to thousands of different compute-heavy workloads. Running Llama 3.1 8B on every metal just to support Workers AI would leave no GPU capacity for other models.
Instead, Cloudflare built a global scheduler on its own developer platform. The control plane schedules a container to a specific Cloudflare location, and then a local scheduler within that location decides which metal should run it. The global scheduler is built with Workers, Durable Objects, and KV. The local schedulers monitor compute capacity and expose that information to the global scheduler, allowing dynamic placement based on capacity and hardware availability, including different GPU types.
Why global scheduling matters
With a traditional regional model, the developer must specify where each workload runs. The platform cannot move it, even if moving it would improve performance. That creates a problem for the platform provider too: adding a new location only pays off if developers manually migrate workloads to it.
Global scheduling changes the contract. Because the workload is not tied to a specific location, Cloudflare can add capacity and use it immediately. It can move workloads to locations that are closer to users, improve time to first token for AI inference, and absorb spikes in demand more effectively. In South America, for example, Cloudflare has compute in 19 cities, while other clouds have one region in one city. Running anywhere means better performance and availability — but only if the platform handles the placement.
GPU workloads: solving three problems
Workers AI provides GPU-backed inference, and the container platform has to handle three major challenges that GPU workloads present: limited GPU memory, container runtime limitations, and very large model images.
First, GPU memory is the scarcest resource. Different models need different amounts of GPU memory, and different GPUs have different amounts. The global scheduler knows which locations have blocks of GPU memory available and delegates placement on a specific metal to the local scheduler. Large models are placed on machines with sufficient GPU memory, while smaller models are moved to other machines in the same location. This maximizes the number of locations where AI models can run, and keeps utilization high.
Second, not all container runtimes support GPUs. Cloudflare’s platform is runtime agnostic — it supports gVisor, Firecracker microVMs, and traditional VMs with QEMU. The team is also evaluating cloud-hypervisor, which is based on rust-vmm and offers:
- GPU passthrough support using VFIO
- vhost-user-net support for high-throughput networking between the host and VM
- vhost-user-blk support for flexible network-based storage
- A smaller codebase than QEMU, written in a memory-safe language
gVisor is used for workloads that need GPU support. Its main component is an application kernel called Sentry, written in Go, which runs in userspace and intercepts application system calls. This approach has a lower resource footprint than a VM because it doesn’t require virtualized hardware or a separate kernel — but it trades away some application compatibility and adds per-system-call overhead. To handle GPUs, gVisor uses nvproxy, which intercepts ioctls destined for the GPU and proxies a subset to the GPU kernel module.
Firecracker microVMs do not currently support GPUs. Cloudflare built the platform with runtime flexibility from day one precisely to handle these differences, without asking developers to choose a runtime themselves.
Squeezing more out of every image pull
The images behind today's AI inference workloads are heavy — 15 GB and up once specialized libraries and GPU drivers are included. That creates a real operational problem. Scheduling a fresh container in Tokyo and naively pulling its image from object storage in Los Angeles means latency that wrecks the whole point of scaling in new locations at short notice.
The requirements were straightforward: very large images must move quickly, the registry must not be a single point of failure, and teams shouldn't have to operate registry infrastructure. Cloudflare's R2 storage handles the global distribution, and the Cache — with Tiered Cache to follow — drives hit rates. On top of that, the team open-sourced serverless-registry, a container registry built on Workers that deploys in about five minutes and runs in every Cloudflare location.
The visible bottleneck turned out to be docker push. Docker relies on gzip for layer compression, so the team swapped in Zstandard (zstd), which compresses faster and produces smaller payloads. A custom CLI tool now handles build, chunking, and push internally instead of docker build and docker push, splitting layers into 500 MB chunks that stay under Workers body size limits. The result: 30 GB GPU images pull in four minutes rather than eight, with the tool slated for open sourcing.
Letting anycast decide where containers should live
Cloudflare's network runs on anycast, letting a single IP route to the nearest data center. Unimog, the Layer 4 load balancer, sends traffic to any metal that's online and has capacity. That works because nearly every service runs in every location.
GPU-backed container workloads break that assumption. If a workflow only lives in 20 locations, Unimog can't know where to send a request. A developer bringing their own load balancer just shifts the problem to scheduling decisions that migrate, scale up, and scale down across regions. The platform needed to preserve the core deal: deploy an app, get one IP, and let the network handle balancing by load, health, and latency.
The solution is a sidecar to Unimog called the Global State Router. An eBPF program intercepts packets bound for virtual IPs at the network interface. The router keeps a mapping from anycast IPs to candidate container destinations, constantly refreshed by health, readiness, distance, and latency, then forwards packets at Layer 4 to the best fit. On the receiving server, another instance of the router catches the packet and delivers it to the local container.
For developers, the workflow collapses to pointing Cloudflare at a container image, declaring constraints and health checks, and getting back a single usable IP. The platform handles the placement decisions in the background.
Proximity as a feature
Remote Browser Isolation is the workload that demanded the platform in the first place. Chromium runs in containers on Cloudflare's network; users only receive rendered output. Since every mouse move and keystroke travels between user and browser, distance translates directly into perceived lag. A user in Santiago hitting a browser in Buenos Aires adds roughly 21 ms, São Paulo adds 48 ms, Bogota adds 67 ms, and Raleigh adds 128 ms — a spread that changes the feel of every interaction.
Keeping containers near eyeballs doesn't just serve browser isolation. WebRTC video, multiplayer games, ad serving, and financial transactions all improve when the compute is geographically close, and the container platform generalizes that capability beyond browser workloads.
Banking the nightly idle cycles
Most web traffic trails human waking hours. Between midnight and 5 AM local time, far fewer eyeballs are making requests, leaving many cores across Cloudflare's network unused. Containers can fill that gap for background workloads that don't need geographic proximity — nothing is waiting on their output, so they can run wherever the night is deepest.
These jobs, however, often execute untrusted code that can't run in a v8 isolate. Cron Triggers already make a best-effort attempt to use off-peak compute, but other workloads need stronger sandboxing. Workers Builds — announced in open beta on Builder Day 2024 — fits the bill: it compiles a Worker from a git repository after every merged pull request, running on the container platform with otherwise idle compute. The service schedules the job without knowing anything about network capacity:
scheduling_policy: "off-peak"
Fast starts without cutting isolation corners
An off-peak job still carries expectations about startup speed, which poses a conflict: each build needs a fresh container for strong isolation, yet cold starts in a Firecracker VM are costly. Prewarming solves that by having servers fetch the required image before they become eligible for an off-peak job. Once a build finishes, the container is discarded and the next job gets a fresh one based on the prewarmed image.
The numbers are stark. Without prewarming, pulling Workers Build images takes roughly 75 seconds. With it, a new container spins up in under 10 seconds, with more headroom ahead via pre-booting images and Firecracker snapshotting, which can restore a VM in under 200 ms.
Bringing Containers and Workers Closer Together
Our engineering teams are increasingly asking for tighter coupling between the container platform and Workers. We’re listening and starting to explore what a unified experience might look like. The project Key Transparency, already running on our containers platform, demonstrates the potential use case.
Key Transparency audits public key changes for WhatsApp message encryption. The architecture splits between Workers for most logic and long-running, compute-heavy processes that make more sense as containers. Today, deploying a container next to Workers requires careful region selection, manual scaling decisions, IP exposure and updates, and a bespoke auth layer between Worker and container. We want to remove those hurdles.
We’re still sketching API designs, but envision a pattern where a Worker handles ingress and business logic, a Durable Object binds to a container, and the cloud platform loads the image on the right hardware and scales automatically.
Configuration starts in the Worker's wrangler.toml (or via Terraform):
[[container-app]]
image = "./key-transparency/verifier/Dockerfile"
name = "verifier"
[durable_objects]
bindings = { name = "VERIFIER", class_name = "Verifier", container = "verifier" } }
From inside the Worker, the Durable Object’s RPC method is called directly:
fetch(request, env, ctx) {
const id = new URL(request.url).searchParams.get('id')
const durableObjectId = env.VERIFIER.idFromName(request.params.id);
await env.VERIFIER.get(durableObjectId).runVerification()
//...
}
Within the Durable Object, the code can boot the container, set its configuration, mount buckets as filesystems, and issue HTTP requests to it:
class Verifier extends DurableObject {
constructor(state, env) {
this.ctx.blockConcurrency(async () => {
// starts the container
await this.ctx.container.start();
// configures the container before accepting traffic
const config = await this.state.storage.get("verifierConfig");
await this.ctx.container.fetch("/set-config", { method: "PUT", body: config});
})
}
async runVerification(updateId) {
// downloads & mounts latest updates from R2
const latestPublicKeyUpdates = await this.env.R2.get(`public-key-updates/${updateId}`);
await this.ctx.container.mount(`/updates/${updateId}`, latestPublicKeyUpdates);
// starts verification via HTTP call
return await this.ctx.container.fetch(`/verifier/${updateId}`);
}
}
The heavy lifting is abstracted away.
No placement or scaling logic is needed, no manual service discovery or custom authorization. Binding to other services like KV and R2 works declaratively. The platform routes and authenticates requests and scales the container out as the number of bound IDs grows.
These integrations are early-stage, but the direction is already promising.
Early Access and Next Steps
We aren't ready to open the container platform to everyone yet, but after launching several GA products on it internally, we are inviting a limited number of engineering teams to start building ahead of wider availability in 2025. We're also hiring more engineers for this effort.
If you have a project in mind that doesn't fit within the current Workers or Developer Platform constraints, tell us here what you'd like to build and why it needs more than what's available today.
