Cloudflare Images: A Rust-Based Image Pipeline on Workers

Cloudflare Images is built on a foundation of reusable Rust libraries and Cloudflare Workers. This approach has allowed the team to share core image processing code across multiple products and rapidly iterate on new features. The architecture combines a dedicated image processing service at the edge with a serverless delivery layer.

Shared Rust Libraries Across Products

The core of the system is Cloudflare Image Resizing, a Rust-based web server that handles image transformations. It receives HTTP requests with resizing options, fetches the original image from the origin, applies the requested operations, and returns an optimized response. By structuring the code as reusable crates, the image processing and compression logic can be embedded in other services.

This reuse extends beyond Cloudflare Images. The Polish service, originally a Golang tool that relied on command-line utilities like jpegtran and pngcrush, now wraps the core Image Resizing library in a command-line executable. This unification ensures consistent handling of metadata and color profiles across both services.

Cloudflare Images itself is a collection of Rust crates. It adds a custom storage back-end, named size presets (variants), and URL signing. The Image Resizing service uses these shared libraries to parse the Images URL scheme, access the storage layer, and look up variant definitions.

How Image Processing Is Triggered

Image Resizing runs on every server in Cloudflare's Anycast network. Its tight integration with the Cloudflare cache means it only handles requests on a cache miss; the closest data center serves the eyeball request.

Developers can use the service in two ways. The default is a declarative URL scheme that specifies dimensions and other options directly in the path. For more control, a JavaScript API is available in Cloudflare Workers, enabling programmatic handling for each request.

Inside the Cloudflare Images Architecture

Cloudflare Images comprises four main components:

  • Core service: Manages the public API for image assets.
  • Image Resizing service: Handles transformations and caching.
  • Delivery Worker: Parses image URLs and passes parameters to the resizing service.
  • Storage: Provides access to original image assets.

To support Cloudflare Images, the Image Resizing service was extended with access to Cloudflare storage for originals, variant definitions, and signed URL support.

Delivery is handled through the managed imagedelivery.net domain, which uses Tiered Caching to maximize cache hit ratios. This setup provides image hosting without requiring customers to configure their own domain. A Cloudflare Worker powers this delivery by parsing URLs and forwarding the appropriate options to the processing service.

Data Storage Strategy

Cloudflare Images distributes its data across the network based on access patterns:

  • Image metadata is stored in core data centers.
  • Variant definitions are distributed to edge locations via Quicksilver, Cloudflare's key-value store.
  • Original images are kept in core data centers.
  • Optimized images are cached at the edge, close to users.

Variant definitions are managed by the core service, which reads and updates them in Quicksilver. Image metadata itself is not stored in Quicksilver, as it would grow linearly with the number of images hosted. Instead, the image URL contains the account ID and image ID, along with a flag indicating whether access verification is required. The trade-off is that changing an image's access status changes its URL. Limiting variants per account keeps the data volume on edge disks manageable.

Cache Invalidation for Variants

Transformed images are cached based on their variant configuration. To keep these up to date, Cloudflare purges the cache under two conditions:

  • When an image's access permission changes.
  • When a variant definition is updated.

For access changes, the cache is purged by the image URL. When a variant is updated, a purge-by-tag request is issued using the account-id/variant-name tag, which the Image Resizing service attaches to all transformed images.

Signing URLs for Access Control

Restricted access relies on URL signatures using an SHA-256 HMAC with an optional expiration. The signing process is as follows:

  1. Take the URL path and query string, with the path starting with /.
  2. Compute the HMAC of the path with the query string using the signing key configured in the Dashboard.
  3. For expiring URLs, append ?exp= followed by the Unix timestamp to the URL.
  4. Add a ? or & before the signature, depending on whether a query string is already present.
  5. Append sig= and the hex-encoded HMAC (64 characters).

For example, with a secret of this is a secret, the signature for /hello/world is 6293f9144b4e9adc83416d1b059abcac750bf05b2c5c99ea72fd47cc9c2ace34, producing the URL https://imagedelivery.net/hello/world?sig=6293f9144b4e9adc83416d1b059abcac750bf05b2c5c99ea72fd47cc9c2ace34.

Direct Uploads via Workers KV

Cloudflare Images supports direct creator uploads, similar to Cloudflare Stream, for scenarios where end users upload content to a web or mobile app. This removes the need for those users to hold API tokens. The upload flow leverages Workers KV to store account details with an expiration date. A Worker handles the upload URL, validates the request against the KV store, and grants upload access only on a successful lookup.

Potential Future Enhancements

The current architecture opens several avenues for product evolution.

Resizing Hints at Upload Time

Presently, no transformation occurs during upload, allowing the image to be served globally immediately. A possible future feature is accepting resizing hints at upload. This would not necessarily trigger immediate processing for every upload but could serve as a signal to prepare important variants. For instance, this signal could be used to generate AVIF versions for high-priority assets.

Custom Domain Support

Serving images from the managed imagedelivery.net domain is a strong default, but it introduces client-side TLS negotiation overhead. For customers already on Cloudflare, serving images from their own domain could reduce latency. This feature can be supported by the current design without major architectural overhauls, providing a clear path for websites proxied through Cloudflare.