Why Dropbox Dash needed its own context engine

Work content is scattered across dozens of apps and locked behind individual permissions. The average user has more tabs and SaaS accounts open than they can meaningfully track. LLMs are great at general knowledge, but they have no visibility into proprietary, walled-garden content. Dash exists to close that gap by pulling in data from third-party apps and making it searchable through a single interface.

The groundwork for that product is a pipeline we call the context engine. It starts with connectors—custom crawlers built for each third-party service. Every app brings its own rate limits, API quirks, and ACL systems, so this is largely bespoke integration work.

From there, we do heavy content understanding. Incoming files are normalized to formats like markdown, then enriched: extracting titles, metadata, links, and generating embeddings. Plain text documents are simple. Images need CLIP-style models at minimum, but complex visuals require true multimodal understanding. PDFs mix text and figures. Audio requires transcription. Video is harder still—a scene with no dialogue can't be found through transcription alone, so we apply multimodal models to extract per-scene understanding and store that.

All of this—raw chunks, embeddings, contextual graph representations—flows into our data stores. We run a lexical index built on BM25, alongside dense vectors in a vector store. That combination enables hybrid retrieval; BM25 alone has proven surprisingly effective as a workhorse. Retrieved results get multiple ranking passes for personalization and ACL enforcement before they reach the user.

Context is only part of the story. We model how pieces of information relate to each other, which is what makes cross-app intelligence useful. Meetings connect to documents, attendees, transcripts, and prior notes. Those relationships are the foundation for the knowledge graph work below.

Index-based versus federated retrieval

The core architecture question was: process everything on the fly, or pre-process at ingestion? Federated retrieval is easier to start. There are no storage costs, data is generally fresh, and you can keep plugging in new connectors. But it has serious weaknesses. You depend on the speed and quality of every upstream API or MCP server. You only get access to the content available to you personally—not company-wide connectors. And processing happens at query time: merging results, re-ranking, and reasoning over them consumes expensive tokens.

Index-based retrieval flips that trade. Because ingestion happens ahead of time, you can enrich content and build data sets that don't exist anywhere natively. You get company-wide access, offline ranking experiments, and fast query response. But the cost is upfront engineering: custom connectors for each source, freshness management under rate limits, and real hosting expenses. And you face storage architecture decisions—vector database, BM25, hybrid, or full graph RAG. The last of those is the path we took.

Containing MCP's context bloat

MCP generated a lot of excitement because it seemed to eliminate the need for custom APIs. In practice, MCP tool definitions consume precious context-window space. We aim to keep Dash requests around 100,000 tokens—including tool results. Tool definitions fill that quickly, and retrieval results flood it even faster. The net effect is context rot: degraded chat and agent performance. Latency suffers too. Simple MCP-based queries can take up to 45 seconds; an index query returns content in seconds.

We've taken several practical steps to keep MCP usable at scale:

  • Wrap the index in one super tool instead of exposing five to ten separate retrieval tools.
  • Use knowledge graph modeling to return only the most query-relevant information, cutting token overhead.
  • Store tool results locally rather than putting them into the LLM context window.
  • For complex agentic queries, route to narrow sub-agents via a classifier, so each agent has a focused tool set.

Knowledge graphs without the graph database

Knowledge graphs become genuinely valuable when you model relationships across apps. A calendar invite has attachments, minutes, a transcript, attendees, maybe a linked Jira project. Every connected app has its own notion of “people,” so we derive canonical IDs for each person. That identity resolution improves both profile views and the quality of retrieval itself.

Consider a query for past context engineering talks from a specific person. A people model in the graph lets the system understand which "Jason" is meant without multiple retrieval rounds. Scoring those results with normalized discounted cumulative gain showed clear gains from this people-based approach.

The architecture behind all this is nonstandard. We don't store the graph in a graph database—we tried. Query latency and hybrid retrieval patterns were both problematic. Instead, we stage graph construction asynchronously, build out relationships, and package them into knowledge bundles. These aren't raw graphs but compact contexts—effectively summaries of the graph, something like an enriched embedding. Those bundles flow through the exact same index pipeline as any other content: they get chunked and embedded for both lexical and semantic retrieval.

When the User Is a Model

Classic search systems have a built-in feedback loop: humans click on results they like, and miss the ones they don’t. Dash’s retrieval pipeline doesn’t have that luxury. Results are fetched for a large language model, not a person, so Dropbox engineers had to find another way to measure whether retrieval quality actually improved.

Their answer is LLMs as a judge: a model scores each piece of retrieved information on a relevance scale of one to five. Human input still matters though. Engineers first had humans label documents and measured how often the judge disagreed with them. The initial prompt had an 8% disagreement rate. Refining the prompt — for instance, asking the model to explain its reasoning — lowered that number. Switching the judge model to OpenAI’s o3, a reasoning model, brought disagreements down further.

One lingering blind spot was context. A judge model may not recognize internal acronyms or recent terminology — it can’t answer “What is RAG?” if that wasn’t in its training data. The judge sometimes has to fetch context itself rather than rely solely on pre-computed information. Dropbox calls this “RAG as a judge,” and it cut disagreements even more.

Optimizing Prompts at Scale

To push accuracy further, the team turned to DSPy, a prompt optimizer that tunes prompts against a defined set of evaluations. It’s a natural fit for evaluating judges, which have clear rubrics and unambiguous expected outcomes. Getting to zero disagreement may be impossible — even human evaluators don’t fully agree on relevance — but the gains have been substantial.

One surprising outcome: DSPy enabled a workflow where engineers summarize disagreements as bullet points, then let the optimizer improve those bullets to shrink the disagreement set. That created a positive feedback loop that produced notable results.

DSPy’s utility extends beyond judging. Dash currently runs more than 30 prompts across ingestion, offline evaluations, and the online agentic platform. With 5 to 15 engineers continually tweaking prompts, managing them as text strings in a code repository quickly degrades into a whack-a-mole of regressions. Defining prompts programmatically and letting the optimizer generate them is far more maintainable at scale.

Model switching is another win. Every LLM has idiosyncrasies, and a prompt tuned for one model usually needs rework for another. With DSPy, the workflow is simply: plug in the new model, define the goals, and let the optimizer produce a working prompt. That speed matters for agentic architectures, where a planning LLM coordinates smaller, narrowly focused sub-agents. Each sub-agent benefits from a model and prompt tailored to its specific task, so being able to swap models rapidly is a real advantage.

Practical Takeaways

  • The index is worth it, but not cheap. Building the infrastructure, data pipelines, storage, and retrieval logic is a substantial effort. It pays off only at scale.
  • Cross-app relationships work. Connecting knowledge across apps — say, pulling in an org chart when processing prompts — improves results, but requires modeling those relationships in a general way, since you can’t anticipate every query.
  • Guard your context window. Limit tool usage, design “super tools” that handle broader tasks, explore tool selection strategies, and consider sub-agents with strict tool-call limits.
  • Invest in strong LLM judges. The first prompt is rarely the best one. Grinding down disagreement rates lifts the quality of the entire system.
  • Prompt optimizers are essential at scale. They help anywhere, but become critical once many prompts and many engineers are in play.

The overarching principle is a familiar one in software engineering: make it work, then make it better. For teams just starting out, investing in MCP tools and real-time retrieval is the right first step. As usage grows and patterns emerge, that’s the time to layer in knowledge graphs, judge optimization, and prompt tuning. Much of what Dropbox has built took years and a dedicated team; the path there starts with a working system, not a perfect one.