Message queues for the Workers Developer Platform

Cloudflare has opened the beta for Queues, a message queue service built into the Workers Developer Platform. Queues joins Workers and R2 as part of Cloudflare's application services, giving developers a way to decouple components of distributed applications without managing infrastructure. The service runs on Cloudflare's network, so there are no regions to select or capacity to estimate, and no data egress fees.

Message queues are a standard pattern in cloud applications. An ecommerce checkout flow, for example, can place an order into a queue that a fulfillment service processes later. This decoupling lets each service operate independently, which simplifies deployments and makes individual components easier to reason about and maintain. Queues also help batch and buffer calls to downstream services and APIs, which can reduce load and cost.

Build applications of any size on Cloudflare with the Queues open beta

Enabling the Queues open beta

To get started, open the Cloudflare dashboard, go to the Workers section, and select Queues from the navigation menu. Click Enable Queues Beta, review the order, and proceed to payment details. If you are not already on a Workers Paid Plan, one will be added automatically. After completing the purchase, you can return to the Queues page and create your first queue with the Create Queue button.

BLOG-1452 Embedded Image - PJNlvN

Each account can currently create up to ten queues, a limit Cloudflare intends to raise before general availability.

Building a log aggregator with Workers, Queues, and R2

The Wrangler CLI manages queues alongside Workers and R2 resources. A practical example is a log sink: a Worker that receives HTTP requests with log entries, places them in a queue, and then writes batches to R2 storage.

After installing and authenticating Wrangler, create the queue and bucket resources:

wrangler queues create log-sink
wrangler r2 bucket create log-sink

You can verify both resources with wrangler queues list and wrangler r2 bucket list.

Next, initialize a new Worker application with wrangler init. When prompted, create a package.json, use TypeScript, and add a fetch handler at src/index.ts. The wrangler.toml file defines producer and consumer bindings:

name = "queues-open-beta"
main = "src/index.ts"
compatibility_date = "2022-11-03"
 
 
[[queues.producers]]
 queue = "log-sink"
 binding = "BUFFER"
 
[[queues.consumers]]
 queue = "log-sink"
 max_batch_size = 100
 max_batch_timeout = 30
 
[[r2_buckets]]
 bucket_name = "log-sink"
 binding = "LOG_BUCKET"

The [[queues.producers]] binding gives the Worker access to the log-sink queue via env.BUFFER. The [[queues.consumers]] binding registers this Worker to receive batches from the queue. When the queue accumulates a batch, the Workers runtime invokes the queue() handler with the batch as an argument. The R2 bucket bindingmakes the storage available as env.LOG_BUCKET.

The worker code in src/index.ts has a fetch() handler that takes the request body, parses it as JSON, sends each log entry to the queue with await env.BUFFER.send(log), and returns an HTTP 200 response. The queue() handler receives a batch of messages, concatenates the log entries into a string buffer, and writes the result to the R2 bucket with a timestamp as the filename.

export interface Env {
 BUFFER: Queue;
 LOG_BUCKET: R2Bucket;
}
 
export default {
 async fetch(request: Request, env: Environment): Promise<Response> {
   let log = await request.json();
   await env.BUFFER.send(log);
   return new Response("Success!");
 },
 async queue(batch: MessageBatch<Error>, env: Environment): Promise<void> {
   const logBatch = JSON.stringify(batch.messages);
   await env.LOG_BUCKET.put(`logs/${Date.now()}.log.json`, logBatch);
 },
};

Publish the application with wrangler publish. The output confirms both bindings: the Worker is a producer for the log-sink queue and also its consumer. Send HTTP POST requests with JSON log entries to the Worker's URL using curl or another API client. The aggregated logs appear in the R2 bucket as JSON arrays, organized under the logs prefix. The entire application, including configuration, is fewer than 45 lines.

A real-world use case: UUID.rocks

UUID.rocks uses Queues to solve a practical problem. The service generates unique UUIDv4 identifiers and wanted to verify uniqueness across all IDs it produces — roughly 80,000 requests per day. Writing each ID directly to R2 would be inefficient and costly, so the team introduced a queue between UUID generation and storage.

Each time a UUID is requested, a Worker places the value into a queue. Once enough messages accumulate, the buffered batch is written to R2 as JSON objects. This reduces the number of R2 writes and makes the data easier to process later.

The uuid-queue application is a single Worker with three event handlers:

  1. A fetch() handler receives the generated UUID as JSON and sends it to the queue.
  2. A queue() handler writes batches of messages to R2 in CSV format.
  3. A scheduled handler combines the previous hour's batches into a single file.

The source code and deployment instructions are available on GitHub.

What’s under the hood

Queues is built by composing other Cloudflare services, primarily Workers and Durable Objects. That composition solved two hard problems quickly: securely invoking a user’s consumer Worker from Cloudflare’s own service, and maintaining strong consistency at scale.

Worker invocation without HTTP

Before mid-2022, invoking one Worker from another meant making an HTTP call from inside your script. That required knowing the downstream endpoint at deploy time, added latency by sending the request through the Cloudflare network a second time, and left authentication and authorization up to you.

Service Worker Bindings, which reached general availability in May 2022, changed that. A Worker can hold a binding to another Worker in the same account and invoke it directly, avoiding the extra network hop and the need to build your own auth scheme. The trade-off is that the target Worker must be known at compile time—a model Cloudflare describes as “static dispatch.”

Dynamic dispatch arrived with Workers for Platforms. After a closed beta, that product entered general availability in September 2022. It lets SaaS and platform providers accept user-uploaded scripts and run them safely, dispatching to scripts that were never known at compile time. Queues uses that same runtime dispatch to invoke your consumer Worker the moment a message or batch is ready.

Durable storage via Durable Objects

Messages must be persisted to disk in multiple locations before Queues acknowledges receipt. Rather than build that distributed storage layer from scratch, the team reused Durable Objects, which had reached general availability about a year earlier.

Durable Objects are uniquely named class instances that run on a single thread, process messages in order, and expose a strongly consistent key-value storage API. Offloading the durability problem to that existing service is what made it possible to get Queues ready for open beta quickly.

The path from beta to GA

The open beta is designed to let usage shape the roadmap. The stated goal for general availability is unlimited throughput with 100 percent durability. Planned features include FIFO message ordering and API compatibility layers to simplify migrations from other queue services. Which of those lands first depends on what beta users ask for.