Separating the write path from the read path

Classic search architectures keep indexing and query execution on the same machines and scale by adding replicas. That coupling has real costs: every replica duplicates the CPU and memory spent on indexing, and write traffic can steal resources from query traffic. An alternative design separates the two workloads entirely—indexing machines build data structures, while dedicated search machines serve queries from binary files synchronized through cloud storage.

How search indexes are built

Search engines store flattened records rather than relational tables. A product in an eCommerce catalog that would span several linked tables in a database becomes a single JSON object in a search index. Because each record is self-contained, the index can be partitioned into shards, with every shard holding a subset of the records. This independence is what allows read and write operations to be parallelized across machines.

Indexing is CPU-intensive. Before a record can be queried, the engine must tokenize text, handle language-specific challenges such as word segmentation for Chinese, update inverted lists, and often precompute ranking components. Semantic indexing adds neural embedding computation on top of that. The CPU required for this processing scales with the write workload, so it must be distributable to avoid a bottleneck.

The primary/replica model and its limits

Most databases and search engines scale reads by maintaining several copies of the data. A shard has one primary copy that accepts writes, and the log of write operations is replayed on each replica. Scaling writes means splitting data into more shards, while scaling reads means adding replicas. In an architecture that colocates reads and writes, adding a replica duplicates the indexing CPU and memory along with the data—even when only query capacity is short.

This inefficiency becomes more pronounced as the replication factor grows to handle read spikes. Indexing work can also interfere with query latency on a shared machine. Algolia's earlier designs mitigated this by running indexing and queries as separate processes and using Linux Nice Values to prioritize the search process. That helps, but it does not remove the fundamental duplication of indexing resources per replica.

The primary/replica model also makes scaling slow. Adding a replica requires copying data from an existing machine and replaying the log, a process that can take hours while placing extra load on production servers. Resharding, to scale writes, requires moving data between machines as well. As a result, capacity must be provisioned well ahead of demand.

Replicating binary files instead of operations

The alternative is to replicate data at the file level. Instead of sending write operations from a primary to replicas, the indexing machine commits binary data structures to disk and replicas fetch those files. This avoids duplicating indexing CPU, but it introduces latency because a complete shard can be large.

Search engines typically manage shards with generational file structures similar to a log-structured merge (LSM) tree. New writes go to a small level-0 file, which is periodically merged into level-1 to eliminate duplicates and keep queries efficient—a query over an LSM tree must search and merge results from all levels. Each merge rewrites the files that hold the shard data.

The cost is visible in the amount of data transferred to replicas. If a 10 GB shard receives 1,000 updates per minute, the engine ships the small level-0 deltas to replicas continuously. When level-0 reaches the merge threshold at around 1 GB and is merged into the bigger level, replicas must download the entire 10 GB file. The two dominant factors in this behavior are shard size—which caps the volume transferred in a full merge—and the number of LSM levels, which determines how often a full merge occurs.

Fast file replication over cloud infrastructure

Network bandwidth available in modern cloud environments changes the feasibility of file-based replication. With interface speeds up to 100 Gbps, large transfers can finish in seconds, making it possible to separate indexing machines from search machines without sacrificing indexing latency.

The architecture works per shard and can be parallelized across N shards. An indexing VM computes each new file in memory, writes it to the local SSD as a cache, and uploads it to cloud storage via multipart upload, splitting large files into segments that are pushed concurrently. When the upload is finalized, the search VM uses multipart download to fetch all files and switches to the new data version once they are resident in memory. Search VMs with less memory than the data size store a copy on local disk as well. On hardware failure, a replacement indexing VM launches and pulls the previous data copy from cloud storage.

This setup decouples read and write scaling:

  • Query capacity is scaled by adding or removing search VMs, without duplicating indexing work.
  • Write capacity is scaled by changing the number of indexing VMs or their core counts—replacing one machine building twenty 1 GB shards with two machines building ten each is a sub-minute operation.
  • Resharding an index can run as an asynchronous process without pausing ingest.
  • Merge effects stay under a minute when shards are capped at a few gigabytes; with smaller shards such as 1 GB, near-real-time indexing is achievable.

Cloud storage provides its own replication for indexing high availability, while search high availability comes from spreading search VMs across availability zones. Region-level data replication in the storage layer also allows cross-region search deployments by placing search VMs in the desired regions.

Independent scaling through architectural separation

The new architecture separates Read and Write workflows into distinct planes, enabling each to scale independently without coordinated over-provisioning. This gives a straightforward horizontal-scaling story: if writes increase, you grow the indexing cluster; if reads increase, you grow the search cluster. Dynamic scaling replaces static capacity planning.

System administrators no longer need to anticipate traffic surges or index-growth spikes far in advance. Since read and write resources are isolated, the previous strategy of massively over-sizing a shared cluster to absorb worst-case conditions becomes unnecessary.

Broader applicability beyond inverted indexes

The design is not specific to inverted-list indexing. Because the gains come from strong independence between shards, the same pattern applies to other data structures and workloads:

  • Vector search engines: Shards can similarly own document embeddings and return partial results that the orchestrator merges.
  • Any architecture with strong shard independence: Systems that can partition both data and computation benefit from the same isolation of read and write paths.

Independent operations is the core property. When shards do not depend on shared mutable state for reads or writes, each plane can be tuned, scaled, and deployed on its own cadence.