Selecting a model for semantic search at Dropbox scale

Why Dropbox moved beyond keyword search

Dropbox’s Nautilus search engine, introduced in 2018, relies on keyword matching. That approach has inherent limits: users must remember exact file names or specific terms, and queries only work when they match document language. A search for “employment contract” won’t surface a “job agreement,” and cross-lingual retrieval isn’t possible.

To address these gaps, Dropbox explored classic techniques like stemming and query expansion, but ultimately chose semantic search. Instead of matching literal terms, semantic search maps queries and documents into a shared vector space, capturing meaning and intent. This enables retrieval of related content even when wording differs, and supports cross-lingual search—users can query in one language and get results in another.

Rolled out internally in early 2024, then as an experiment for Pro and Essential users in May 2024, semantic search drove a nearly 17% reduction in empty searches (zero-results rate) and a 2% lift in search session success (qualified click-through rate). It became generally available to Pro and Essential users in August 2024, with Business rollout planned for early 2025. Dropbox evaluated models in-house using pre-trained embeddings only, consistent with their AI Principles—no training on user data.

Semantic search and vector search fundamentals

Semantic search refers to the full pipeline—from query input to result output—while vector search is the narrower step of retrieving items by vector similarity. Dropbox’s first iteration supports text files only; other file types are planned for later.

At its core, semantic search relies on embeddings: dense numerical vectors that capture document features across hundreds or thousands of dimensions. During training, models refine these dimensions to encode patterns like word associations, syntax, or document structure. Because individual dimensions often abstract away literal wording, related items cluster together in vector space even when their surface forms differ.

Vector search then compares query embeddings against stored document embeddings using nearest-neighbor algorithms. This similarity-based approach can surface results that match intent—say, returning instructional PDFs, training videos, and diagrams for a query like “guides and resources”—even when no keyword overlap exists.

Model requirements and evaluation strategy

Choosing the right document embedding model was critical for Dropbox’s scale: indexing over a trillion documents requires significant computational investment, and query embedding computation sits directly in the retrieval critical path. Model speed affects both user-facing latency and indexing throughput. Embedding quality matters equally—vectors must be similar enough between queries and relevant documents, yet discriminative enough to filter out irrelevant ones.

Dropbox turned to the Massive Text Embedding Benchmark (MTEB), an open-source benchmark covering eight evaluation tasks across 56 datasets, including multilingual ones. The team adapted MTEB in several ways:

• Infrastructure integration: adapters were added so models running in Dropbox’s in-house inference services could be evaluated alongside inline models, and datasets residing in Dropbox infrastructure could be streamed.
• Multiple embeddings per document: MTEB assumes one embedding per document, but production documents vary widely in size. Dropbox implemented chunking strategies—configurable overlap between chunks—plus summarization to respect model input limits. The resulting embeddings could be used directly or in aggregate.
• Storage and precision optimizations: full-precision embeddings were reduced via lower-bit floating point (half, quarter), fixed-point formats of varying bit depth, and dimensionality reduction through Gaussian random projections.
• Filename handling: public MTEB datasets contain unnamed documents, but Dropbox documents have user-chosen names that matter for retrieval. Filename embeddings were incorporated when applicable.

Building custom evaluation datasets

Public MTEB re-ranking and retrieval datasets were primarily English-only, which doesn’t reflect Dropbox’s multilingual corpus. To close that distribution gap, Dropbox constructed custom MTEB-compatible datasets using Kubeflow pipelines, pulling anonymized query-document pairs from search logs. Strict access controls apply, and datasets are deleted after 30 days per retention policy.

The pipeline also generated multilingual datasets in Spanish, French, German, Japanese, and Korean—at the time, public retrieval datasets had no such coverage. (The MIRACL dataset has since filled that gap.)

Model selection results

Dropbox evaluated 11 models, including four multilingual options, and selected multilingual-e5-large as the top performer. It excelled on Dropbox’s custom datasets and was the best multilingual model on the MTEB public leaderboard across various tasks at the time of benchmarking.

Results on custom datasets for multilingual models used two embeddings per document: one for title with path, one for the first content chunk. Metrics reported are mean reciprocal rank (MRR) and mean average precision (MAP), where higher is better.

The MTEB integration wasn’t limited to vector search. By connecting MTEB with Dropbox’s infrastructure and adding parameters for broader evaluation, other initiatives—including Dropbox Dash and AI-powered file summaries and conversational features—can now evaluate document embedding models for their own applications.

Production tradeoffs: storage and compute

Deploying multilingual-e5-large at Dropbox required balancing search quality against real infrastructure constraints. With storage as the first limiting factor, the team capped vector-search metadata at 4KB per document, which left room to experiment with the number of embeddings per document, their dimensionality, and their numerical precision.

Compression tests showed that reducing precision to 8-bit per channel kept quality loss minimal while cutting each embedding to about 1KB. Dimensionality reduction, on the other hand, had a noticeable negative effect on quality. Since two embeddings per document fit within the storage budget, the team chose to keep the full embedding dimension and preserve data integrity.

The final quantization scheme is a variant of standard 8-bit encoding. Each embedding is first scaled so its maximum channel magnitude equals 1.0. That scalar is stored separately as a 32-bit float (4 bytes), and the scaled embedding—which now lies in [-1, 1]—is remapped to 8-bit signed integer range and rounded. This approach minimized cosine similarity error across query-document pairs in internal evaluation sets.

On the compute side, the main considerations were:

• The maximum number of characters per document
• Limits on the number of document chunks
• Balancing chunk sizes to preserve contextual relevance without excessive processing
• Whether to embed the file path, the content, or both

The winning strategy was a dual-embedding design: one vector for the file path (including path and filename) and another for the document content truncated to 512 tokens. This does not capture full documents, but focusing on the opening 512 tokens kept processing demands low while delivering strong search relevance with only two embeddings per document.

What vector search changes for users

Vector search moves beyond keyword matching to understand the intent behind queries and the meaning of content. That makes finding files faster, and it also eliminates language barriers—queries and documents can be in different languages and still match accurately. For a product like Dropbox, that is a meaningful step forward.

The path to production involved deliberate engineering choices. Using multilingual-e5-large with two embeddings per document reflects a considered balance between search quality, latency, and operational cost. But the broader objective is not just speed. The real measure of value is how effectively users can find what they need, and that is an ongoing effort. The team continues to iterate on search improvements and expects to share more as the feature evolves.

Acknowledgements for this work (in alphabetical order): Aditya Jayaraman, Alex Yin, Jongmin Baek, Kedar Rudre, Marta Mendez, Matt Barta, Mingye Xia, Morgan Zerby, Muye Gao, Prasang Upadhyaya, Sarah Andrabi, Yidi Zhang, Zhangfan Dong.