Vector Search Across Cloudflare’s Network
Vectorize is a globally distributed vector database built on Cloudflare’s Developer Platform. It provides similarity search over embeddings—dense numerical representations of text, images, audio, and other data—through Cloudflare Workers bindings and the REST API. The service runs in every Cloudflare data center on the same infrastructure that powers Workers, with each query processed by a Rust-based DB Service on a server selected to balance load within that data center. Data is read from R2 object storage, routed through Cloudflare’s Cache to reduce I/O latency.
The implementation draws on several platform components: Workers for compute, R2 for durable storage, Queues and Durable Objects for coordinating writes, and the container platform for running service components.
Why Vectors Break Traditional Indexes
Conventional databases rely on indexes like B-trees or binary search trees, which assume data follows a one-dimensional linear ordering. Strings, numbers, and booleans are cheap to compare under that assumption. Vectors are high-dimensional and do not fit this model well; a linear ordering of vectors does not preserve proximity in high-dimensional space, a problem commonly called the curse of dimensionality. Distance computations between vectors are also too expensive to run naively across large collections during query time.
Handling a Query
A similarity search conceptually involves three steps: measuring the distance from the query vector to every stored vector, sorting by that score, and returning the top K results. That brute-force approach becomes prohibitively costly for indexes holding millions of vectors. To make search practical at scale, Vectorize employs two techniques to prune the search space and reduce per-vector computation costs.
Indexing with IVF
Vectorize uses IVF (Inverted File Index) to group similar vectors into clusters, each defined by its centroid—the high-dimensional point minimizing total distance to all vectors in the cluster. Each centroid receives a numeric ID, and vectors are stored in files named after their nearest centroid. At query time, only the files corresponding to centroids nearest the query vector are examined, skipping the rest of the index entirely. The clustering and centroid assignment are performed during index building; the data layout reflects these groupings on storage.
Compression via Product Quantization
Vector dimensions are 32-bit floats, and Vectorize supports vectors up to 1536 dimensions—roughly 6 KB per vector in uncompressed form. For an index storing millions of vectors, that amounts to gigabytes of data to fetch and hold in memory per query if uncompressed data is used throughout. To control CPU, memory, and I/O demands, Vectorize applies Product Quantization (PQ), a dimensionality reduction technique that compresses vectors into a much smaller representation while preserving the information needed for effective distance comparison. The original uncompressed vectors remain available through the API; the compressed forms are used to accelerate the search itself.
Accuracy Refinement
Both pruning and compression reduce search efficiency but introduce approximation. The initial candidate set produced by the approximate search achieves roughly 80% accuracy against the true nearest neighbors. To recover precision, Vectorize performs a refinement pass on the top approximate matches using the original, uncompressed vector data. This re-ranking step pushes effective accuracy above 95%, and the improved results are returned to the caller.
Consistency and Versioned Snapshots
Queries in Vectorize always read from a consistent, immutable snapshot. Writes are processed asynchronously and strictly ordered by arrival. Index updates never modify files in place; instead the system reads an old version, applies the changes, and writes the resulting file as a new object in R2, with each IVF and metadata file carrying its own version number. A manifest file lists all files, including their versions, that make up a particular snapshot of the index. The current version of the index is referenced by a root manifest written to a deterministic R2 bucket location.
When the write process finishes producing all new index files, it commits by replacing the root manifest with a copy of the new manifest. R2’s atomic PUT semantics make this commit safe. Services on the network fetch the updated root manifest on subsequent reads. Files whose content was untouched are simply referenced again from the previous version, avoiding redundant writes.
Maintaining past manifest and index files means Vectorize effectively retains versioned snapshots of the index. This creates a direct foundation for supporting point-in-time recovery, modeled on D1’s Time Travel feature. Because commits happen asynchronously after a write request is acknowledged, Vectorize is eventually consistent: a write is visible to queries only after the async process finishes and updates the root manifest. The team accepted this consistency model because transactional use cases—where two users might race for the same resource and one should fail immediately—are not typical workloads for a vector database. Eventual consistency is the trade-off that keeps queries fast, high-throughput, and inexpensive as indexes grow toward millions of vectors.
From WAL blocks to visible writes
Vectorize’s write path is built around a write-ahead log (WAL) implemented with SQLite in Durable Objects. Every update payload gets an ID, is written to R2, and that ID is handed to the WAL Durable Object, which persists it as a lightweight "block" — essentially a pointer to the actual data.
Durable Objects provide strong transactional guarantees and horizontal scalability, but individual DOs have limited memory and compute. A single index update can involve reading and writing thousands of files in R2, which exceeds what one DO can handle. The WAL therefore acts as a coordinator: it delegates heavy lifting to dedicated compute instances (called "Executors") while using its transactional properties to enforce strong consistency across the steps. Each Vectorize index gets its own WAL DO instance.
Executors run from a shared pool of compute resources and consume work through a producer-consumer pattern via Cloudflare Queues. When an executor picks up a request, it calls an API on the WAL to be assigned to that write. The WAL guarantees that only one executor is ever assigned to a given write.
As the executor works, it writes index files and an updated manifest to R2, but these are not yet visible. The final step is a commit call back to the WAL, passing the updated manifest. The WAL then overwrites the root manifest — the pivot point for atomic updates. Once that write completes, the change becomes visible to the database service and appears in queries.
The system was designed with failure modes in mind from the start. If an executor stalls, the WAL assigns the work to a new executor. If the original executor later recovers and attempts to commit, the coordinator rejects it — any index files or manifests written from that stale version cannot overwrite those from the committed version.
Batching for throughput
The most time-intensive part of a write is reading and writing many files from R2. Even with concurrent I/O, the latency of network operations dwarfs the cost of updating thousands of vectors within a single file. The efficient approach is to maximize the number of vectors processed in one execution, so the WAL batches discrete updates.
When the WAL is ready to request work, it pulls a chunk of blocks off the WAL, preserving their sequence, and writes a new "batch" record into the SQLite table. This record ties together the sequence of blocks, the index version, and the ID of the assigned executor.
Batch size is adaptive. The WAL counts the number of updates represented by each block and fills batches up to 200,000 vectors at once (a limit determined through internal testing), with a cap of 1,000 blocks per batch. This throughput has allowed loading millions of vectors into an index with upserts of 5,000 vectors at a time. The WAL doesn’t pause to accumulate writes; it starts processing as soon as a write arrives. Since it processes one batch at a time, writes that arrive during processing naturally accumulate for the next batch.
Retraining and generations
The WAL also coordinates index retraining, which periodically updates the IVF centroid mapping to reflect the current vectors and maintain search accuracy. Retraining produces a completely new index — all files are updated and vectors are reshuffled. Each index carries a second version stamp, called the generation, to distinguish retrained indexes.
Training runs from a fixed snapshot and may take a few minutes, during which writes to the current generation continue uninterrupted. The trained index will therefore be out of date by the time training completes. When the trainer signals the WAL, the WAL enters a mode where it records writes but stops making them visible on the current index. It then catches the retrained index up with all updates that arrived since training started, and only after it has caught up does it switch over. This prevents the new index from appearing to "jump back in time." Subsequent writes apply to the new generation.
The batch record models this cleanly. Because each batch associates an index version with a range of WAL blocks, multiple batches can span the same sequence of blocks as long as they belong to different generations. A single WAL block can be associated with many batches across generations — the batches effectively form a second WAL layered over the WAL blocks.
Filtering metadata at query time
Vectorize supports metadata filters on similarity queries, letting a query restrict the vector search to a subset of the index — for example, finding the best matches for color: "blue" and category: "robe".
A naive implementation would scan all metadata for each predicate, intersect the resulting sets, and then score every vector in the intersection. That approach doesn’t scale to millions of vectors and prevents use of the IVF index, potentially requiring proximity scores on a very large filtered set.
Instead, Vectorize indexes each filterable metadata property using a Chunked Sorted List Index. For each property, this maintains a sorted list of all distinct values, with each value mapped to the set of vector IDs having that value. This allows binary search in O(log n) time. The sorted list is divided into chunks sized to a target KB weight, keeping index state fetches efficient.
A lightweight chunk descriptor list in the index manifest tracks the chunks and their lower/upper values. This descriptor list can be binary searched to find the chunk that would contain a queried metadata value. Vectorize fetches that chunk from index data and binary searches it to retrieve the matching vector ID set.
For queries with multiple predicates, Vectorize identifies the matching vector set for each predicate, then intersects the sets in memory to determine the final matched set.
Joining metadata results with the vector index
Once the filtered vector set is determined, Vectorize needs to find the most similar vectors within it. Since the matched set contains each vector’s ID and IVF centroid number, Vectorize can count how many matching vectors fall in each centroid. It then determines which and how many top-ranked centroids (ranked by proximity to the query vector) need to be scanned to satisfy the requested number of matches.
The vector search then proceeds through the IVF index, considering only vectors in the filtered set. Because the search space is pruned by metadata filters, filtered queries can often be faster than their unfiltered equivalents.
Latency and throughput in Vectorize
Vector database performance comes down to two metrics: latency and throughput. Latency measures how fast an individual query is processed, typically in milliseconds, and represents the experience an end user perceives. Throughput measures how many queries an index can handle concurrently, expressed in requests per second, and determines whether an application can serve thousands of simultaneous users.
Vectorize is engineered for strong throughput while keeping query latency low. Techniques that shape its performance include snapshot versioning, binary index formats, fragmented data storage, and SIMD-powered computations.
How query latency is minimized
Since Vectorize is a distributed database that persists index state on blob storage, query latency is largely governed by how quickly index data can be fetched. The system leans on Cloudflare’s cache infrastructure and per-server RAM caches to reduce fetch times.
Snapshot versioning helps here: because each index version is immutable once created (see "Eventual consistency and snapshot versioning"), the data is highly cacheable, improving the effectiveness of Cloudflare's caching layers.
To keep index payloads small, Vectorize uses Product Quantization (see "Compressing vectors with PQ") alongside a compact binary file format designed for runtime efficiency—fetched files require no parsing before use. Index data is split into fragments that limit how much must be read per query, and auxiliary indexes guide the engine directly to relevant fragments, cutting down on overfetching.
Vector proximity calculations are accelerated with SIMD CPU instructions. Searches run in two passes to balance latency against result accuracy (as discussed in "Approximate nearest neighbor search and result accuracy refining"). When called through a Worker binding, queries execute on the server handling the Worker request, keeping the database close to the end user and minimizing network overhead.
Scaling throughput horizontally
Vectorize operates in every Cloudflare data center across thousands of servers. Snapshot versioning allows each server to serve the same index without state contention, so throughput scales horizontally with demand. A Vectorize index can elastically accommodate traffic spikes generated by high-volume Worker applications.
A 25x expansion in index capacity
The upgraded Vectorize supports a maximum of 5 million vectors, a 25x increase over the beta-stage cap of 200,000 vectors. This expanded capacity is a byproduct of all the work described above: performance gains in query execution, throughput, and storage go hand in hand.
The 5 million cap may still constrain some use cases, and the team acknowledges this feedback. The limit is a natural outgrowth of building a new globally distributed stateful service, where fast iteration and a path to general availability matter for production readiness.
Builders can use Vectorize as a primary vector store by deploying a single index or sharding data across multiple indexes. Anyone who finds the cap limiting is encouraged to reach out to the team with details about their use case; the team will review what can be done to adapt the service.
Getting started with Vectorize
Vectorize is available to all developers on the free plan, with setup instructions provided in the developer documentation.
For a complete example, see the semantic search tutorial pairing Workers AI with Vectorize for document search runs entirely on Cloudflare. Another tutorial shows how to integrate OpenAI and Vectorize to give an LLM additional context and improve response accuracy.
Questions and ideas are welcome in the #vectorize and #workers-ai channels on the Developer Discord.
