Context Is the Missing Layer for Warehouse-Scale AI Assistants

At Spotify, the data problem was familiar: engineers and analysts needed answers from the warehouse, but finding the right table or the right expert to ask was slow and unscalable. The company built an AI data assistant to close that gap, but the hard part wasn't the model—it was making answers trustworthy across more than 70,000 datasets and petabytes of data.

Simply dumping schemas into an LLM doesn't scale. A million-token context window can't hold an entire warehouse, and schemas carry little semantics. An INT64 column doesn't reveal that values under 100 are legacy test data, or what "active user" actually means. Given the same tables, a model will confidently pick the wrong one. The solution was a curated context layer, owned by the people who understand the data domains.

Clusters: Datasets, Pairs, and Docs

Spotify's data assistant, built on a ReAct loop, answers natural-language questions by selecting context, generating SQL, running it against the warehouse, and returning results with the query and sources. Since August 2025, over 2,100 Spotifiers have used it across 13,000+ conversations and 60,000+ messages, spanning 177 clusters in advertising, podcasts, music, audiobooks, finances, and more. Over a quarter of users never wrote SQL before. It's available as a Slack bot, an MCP server for IDEs, and a dedicated web UI.

The core unit is the cluster, a named data domain owned by an insights team. Each cluster has three parts:

  • Datasets — warehouse tables with full schema, profiling, column cardinality, common value samples, and partition structure. This helps the model write correct WHERE clauses.
  • Pairs — vetted question-to-SQL examples, the few-shot mechanism. Domain experts write or approve each pair to teach query patterns and semantics.
  • Docs — business context: terminology, gotchas, definitions that vary by team, and notes on which columns to use or avoid.

Data scientists and analytics engineers own curation—they decide cluster boundaries, which tables matter, and which examples are canonical.

Why Curation Can't Be Automated

An obvious shortcut was to mine the warehouse's full query history. Take real queries, ask an LLM to infer the question, and use those as training examples. The team tried this during curation, asking cluster owners to review pairs generated from actual queries by data scientists.

They accepted only 12.5% of the proposals.

The rest were ad-hoc exploration, debugging sessions, one-off answers, or queries that used the wrong table or taught a misleading pattern. Query history is mostly noise, and the noise doesn't label itself. Every example runs through an expert because the model reasons over context, but experts decide what's true about the data.

Monitoring Context Health

Data changes, schemas evolve, and business logic shifts. A cluster's context can be accurate one month and wrong the next—a renamed column silently degrades every pair that references it. Each cluster gets a health score computed from continuous signals: how healthy the underlying datasets are, how many curated pairs survive schema changes, how well context covers real user questions, and how reproducible the generated SQL is. When signals degrade, the health score drops and the dashboard suggests actions. Data experts use those scores to decide where to spend curation time.

Feedback Loop and Scale

Every conversation with the assistant is logged—questions, answers, generated SQL, and user feedback—and shown to cluster owners. That closes the loop: each approved pair and clarified doc improves answers for everyone else. Context needs tending, but that effort yields leverage; experts spend less time on one-off questions and more time shaping the knowledge layer that answers thousands.

The architecture isn't Spotify-specific. Any organization with a solid data catalog and engaged domain experts can adopt the same principle: the people who best understand a domain are the right ones to curate the context the model sees. Spotify is now exploring what happens when relevant knowledge lives outside schemas entirely—in process documentation and organizational definitions.