D1 Database Goes GA with a Route to Global Reads

Cloudflare has announced the general availability of D1, its serverless relational database built on SQLite. D1 is designed to give Workers applications a familiar SQL interface backed by Cloudflare's infrastructure, with no provisioning, built-in disaster recovery via Time Travel, and a usage-based pricing model that starts with a generous free tier.

The GA release, which follows an alpha in late 2022 and an open beta in October 2023, includes larger databases, bulk data import and export, and new query debugging tools. But the more significant news is the roadmap for making D1 a truly global database, which will center on asynchronous read replication and a new Sessions API.

The data locality problem

Workers already scale globally by executing serverless code in the data center closest to each incoming request. Cloudflare's Smart Placement can even move a Worker closer to a centralized backend service. But the underlying data for many applications is stored in a single location, so every global request still has to make a round trip to that one database.

For D1, the path to solving this is asynchronous read replication. In any database system, a read replica is a separate, read-only copy of the primary database that is updated asynchronously and therefore may lag behind the primary by some amount of time. Replicas can improve throughput by distributing load and lower query latency by sitting closer to users. Synchronous replication, where clusters stop to wait for the slowest replica, is deliberately avoided for global scale.

Consistency at scale

The complication with multiple replicas is consistency. A single D1 database provides snapshot isolation, the consistency model SQLite itself provides. D1 achieves this by relying on Durable Objects to guarantee that there is at most one active copy of the database, so routing every HTTP request to that one copy preserves a simple, easy-to-reason-about model.

That approach doesn't survive adding replicas. There is no reliable way to route a generic HTTP request to the same replica every time, and hopping between replicas that lag the primary by different amounts breaks the consistency model. An application could issue a write, then a subsequent read on a different replica, and appear to have travelled backwards in time. At best, that provides read committed consistency—a valid model, but one that can produce race conditions that are difficult to reason about.

Rather than try to achieve sticky routing, Cloudflare plans to use a Lamport timestamp, attached to every database query, to guarantee sequential consistency. This model provides a total ordering of writes and key properties like "read my own writes." Any replica can service any query, but it delays the response until it has caught up with the primary to the point of the timestamp carried by the query. Writes would still go to the primary to maintain the total write order, while reads could go to any replica without compromising the consistency guarantees within a logical session.

Sessions: an explicit API for sequential consistency

This design will be exposed through a new Sessions API, which encapsulates all the queries that make up one logical unit of application work—for instance, everything coming from a particular web browser or mobile app. D1's implementation of Sessions uses commit tokens, which identify a particular committed query within a session, to enforce that queries are sequentially ordered even if the session switches between replicas between awaits.

Because the Sessions API changes the consistency model, developers must explicitly opt in. Existing D1 API calls will not be affected them and will continue to provide snapshot isolation as today. Only queries made through a Session will be eligible for replica routing. For sessions that need to span multiple HTTP requests, the commit token from the last query can be passed back into a new db.withSession() call to continue the same consistency context.

Cloudflare emphasizes that this read replication will not require replica configuration or carry extra usage or storage costs. The platform will monitor traffic patterns and automatically create replicas closer to application users, matching the serverless model that expects developers to ignore infrastructure management. The design is still in active development, and Cloudflare is soliciting feedback on the proposal in the #d1 channel on Discord.

What's new in GA

Alongside the replication roadmap, the GA release brings several production-oriented improvements.

Larger database limits

D1 databases can now scale to 10 GB, and the Workers Paid plan supports up to 50,000 databases. Cloudflare notes that new beta-era databases are processing 40x more requests than the alpha-era ones were.

Bulk import and export

The wrangler d1 execute command has been improved to make large imports atomic—a job either completes or leaves the database untouched—and the command now defaults to a local-first execution to protect remote production data. Export support has also been added: wrangler d1 export can output the full schema and data, schema only, or data only to a SQL file. These facilities support tasks like migration testing, seeding local development instances, and manual backups for custom compliance needs.

Query insights

A new experimental command, wrangler d1 insights, gives developers a way to analyze query performance. The underlying metrics are additionally available through a GraphQL API.

ORM support

The Prisma ORM (version 5.12.0 and later) now supports Workers and D1 directly, joining Drizzle ORM and other community projects as options for developers.

Getting started with D1 and its current feature set is available through the standard Cloudflare developer documentation.