Building a scheduling system with Workers and Durable Objects

In this post we're going to show you how to build a scalable service that will schedule HTTP requests on a specific schedule or as one-off at a specific time

AJ

Adam JanisCloudflare

·August 5, 2022

Building a scheduling system with Workers and Durable Objects

Scheduling HTTP requests on Cloudflare Workers

Building a reliable scheduling system is harder than it looks. The scheduler and its storage backend have to be designed for scale from day one, or you'll run into limits as soon as you start adding real traffic. Most scheduling services use a central database and a cron-like process to scan for due jobs — a design that gets complicated once you need redundancy and horizontal scaling.

Cloudflare Workers, Durable Objects, and Alarms turn out to be a natural fit for this kind of workload. Durable Objects each come with their own isolated storage and an alarm scheduler, both automatically replicated and failed over by the platform. That means you get a scheduling primitive that is reliable by default, without needing to operate your own queue or database.

Common use cases include webhook delivery, sending reminder emails a week after signup, or invoicing. In this walkthrough, we'll build a scalable service that can fire HTTP requests either once at a specified time or on a repeating interval, using one Durable Object per scheduled request.

How the pieces fit together

Wrangler — the CLI for developing and publishing Workers
Cloudflare Workers — the runtime that executes our code at the edge
Durable Objects and Alarms — per-request storage and timers

The application has two parts: a scheduling API and a Durable Object class. The API accepts a scheduled request, stores its metadata (URL, headers, body, timing), and creates a Durable Object. Each Durable Object holds one scheduled request in its own storage and sets an alarm as the trigger that wakes it up when it's time to fire.

Because each request lives in its own Durable Object, the system scales horizontally as far as you need — there's no shared bottleneck. The Cloudflare network manages data replication and guarantees the alarm fires when the time comes.

Setting up the project

Start by scaffolding a new project with:

wrangler init -y durable-objects-requests-scheduler

Then define the TypeScript types that describe the scheduled request. Doing this upfront keeps the rest of the implementation cleaner.

src/types.ts

export interface Env {
    DO_REQUEST: DurableObjectNamespace
}
 
export interface ScheduledRequest {
  url: string // URL of the request
  triggerAt?: number // optional, unix timestamp in milliseconds, defaults to `new Date()`
  requestInit?: RequestInit // optional, includes method, headers, body
}

The Durable Object class

The Durable Object that handles one scheduled request is just a few lines. In the object's constructor logic, it accepts the request payload, persists it, and sets the alarm. Later, when the worker runtime wakes the object, the alarm() method reads the stored data, performs the HTTP request, and — if the schedule is repeating — sets another alarm for the next interval.

src/request-durable-object.ts

import { ScheduledRequest } from "./types";
 
export class RequestDurableObject {
  id: string|DurableObjectId
  storage: DurableObjectStorage
 
  constructor(state:DurableObjectState) {
    this.storage = state.storage
    this.id = state.id
  }
 
    async fetch(request:Request) {
    // read scheduled request from request body
    const scheduledRequest:ScheduledRequest = await request.json()
     
    // save scheduled request data to Durable Object storage, set the alarm, and return Durable Object id
    this.storage.put("request", scheduledRequest)
    this.storage.setAlarm(scheduledRequest.triggerAt || new Date())
    return new Response(JSON.stringify({
      id: this.id.toString()
    }), {
      headers: {
        "content-type": "application/json"
      }
    })
  }
 
  async alarm() {
    // read the scheduled request from Durable Object storage
    const scheduledRequest:ScheduledRequest|undefined = await this.storage.get("request")
 
    // call fetch on scheduled request URL with optional requestInit
    if (scheduledRequest) {
      await fetch(scheduledRequest.url, scheduledRequest.requestInit ? webhook.requestInit : undefined)
 
      // cleanup scheduled request once done
      this.storage.deleteAll()
    }
  }
}

With the class written, you need to tell Wrangler about it. Add a Durable Object binding in wrangler.toml, pointing at the module and exported class name.

wrangler.toml

name = "durable-objects-request-scheduler"
main = "src/index.ts"
compatibility_date = "2022-08-02"
 
# added Durable Objects configuration
[durable_objects]
bindings = [
  { name = "DO_REQUEST", class_name = "RequestDurableObject" },
]
 
[[migrations]]
tag = "v1"
new_classes = ["RequestDurableObject"]

The scheduling API

The API itself is a Worker that accepts JSON payloads describing a scheduled request. It exposes a single endpoint:

POST /:scheduledRequestId? — create a new scheduled request, or update an existing one by passing its ID as the URL path.

Any other HTTP method returns 405 - Method Not Allowed. The payload must match the schema defined in types.ts: a required url, optional headers and body, and exactly one of triggerAt (a Unix timestamp) or triggerEverySeconds. If neither timing field is provided, the request fires immediately.

src/index.ts

import { Env } from "./types"
export { RequestDurableObject } from "./request-durable-object"
 
export default {
    async fetch(
        request: Request,
        env: Env
    ): Promise<Response> {
        if (request.method !== "POST") {
            return new Response("Method Not Allowed", {status: 405})
        }
 
        // parse the URL and get Durable Object ID from the URL
        const url = new URL(request.url)
        const idFromUrl = url.pathname.slice(1)
 
        // construct the Durable Object ID, use the ID from pathname or create a new unique id
        const doId = idFromUrl ? env.DO_REQUEST.idFromString(idFromUrl) : env.DO_REQUEST.newUniqueId()
 
        // get the Durable Object stub for our Durable Object instance
        const stub = env.DO_REQUEST.get(doId)
 
        // pass the request to Durable Object instance
        return stub.fetch(request)
    },
}

One limitation worth noting: the API doesn't keep a registry of created requests. If you need to list scheduled or completed webhooks, you'd either save each Durable Object ID in your own backend at creation time, or build a separate registry using Workers storage.

Testing locally

Wrangler can run the entire stack locally, which is handy for iterating without deploying. The dev server listens on localhost:8787 and doesn't require an internet connection.

wrangler dev --local

With the server running, POST a JSON payload to schedule your first request:

{
	"url":  "https://example.com",
	"triggerEverySeconds": 30,
}

This example sends a GET to https://example.com every 30 seconds, as configured by triggerEverySeconds. The logs show the generated Durable Object ID and the recurring alarm.

To change the schedule later — say, double the interval — send another POST with the same request ID in the path and the updated payload. The system will overwrite the stored metadata and reset the alarm.

Deploying

Once you're satisfied with the local test, push it live:

wrangler publish

Wrangler bundles the Worker, wires up the Durable Object bindings, and publishes to your workers.dev subdomain, printing the deployment URL. The whole service, from a clean project to a scalable scheduler running on the edge, takes only a few files.

The full source is available in the Workers templates repository, or you can try an interactive version in your browser via the StackBlitz template.

AJ

Written by

Adam Janis

Cloudflare

Writes for Cloudflare. Part of the Tech Report engineering index.

All articles by Adam Janis →