Scaling the Search Org

Search has been central to the Spotify experience since the company’s launch in 2008. What started as a small, cross-functional effort has grown alongside the platform, which counted 345 million users by December 2020. But as the user base grew, so did the complexity of the engineering organization behind Search. This is the story of how Spotify reorganized its Search teams — and why they chose a platform approach over a purely product-focused one.

By early 2019, several teams were already working across the Search stack: infrastructure, machine learning, and backend systems. The initial instinct was to organize these teams around specific user problems. That approach worked for a while, but it created churn. New problems kept emerging, priorities shifted, and teams were being formed nearly every quarter.

Stack Ownership: A First Attempt

In 2020, Spotify tried a different structure. Each team became responsible for one piece of the Search stack: one for data ingestion, one for personalized result quality, one for Search APIs, one for insights, and one dedicated to the company’s podcast push.

The obvious tension was cross-cutting problems. Solving issues like query intent, retrieval, or ranking often required expertise from multiple parts of the stack. Under the new model, a single problem could require coordination across five different teams. The organization also brought in more headcount — internal efforts grew by more than 100% compared to 2018, and 500% compared to 2016 — but the speed of delivery and experimentation did not scale at the same rate.

Diagnosing the Bottlenecks

To understand whether the problems were structural or just perceived, Spotify’s engineering leadership looked at two data points, drawing on research by Chief Architect Niklas Gustavsson: system centrality and system congestion.

Centrality was measured in two ways: indegree (how many teams depend on a given service) and outdegree (how many services a team depends on). Congestion tracked the number of unique teams contributing to the same codebase over time. Search ranked high on all three dimensions. It was a critical system, but also one with heavy coordination overhead — both for the teams building it and for outside teams trying to use it.

Reframing Search as a Platform

The data pointed to a conclusion: Search was not just a feature area, it was a platform. And platform needs differ from end-user needs. Spotify had already built internal tools and platforms to scale other parts of the business, so the question became whether the same logic applied to Search.

From a user perspective, there were clear signals of market-specific dissatisfaction, particularly in regions outside North America and Western Europe, where Spotify expected most new-user growth. From an internal perspective, teams outside the Search area often had to involve multiple Search specialists just to get simple integrations done. The friction was real on both sides.

A Two-Group Structure

In 2021, Spotify split its Search area into two groups, each with its own mandate and success metrics:

  • One group focuses on the personalized core Search experience, with Spotify end-user satisfaction as the measure of success.
  • The other group targets Spotify developer happiness, encouraging experimentation while maintaining service-level objectives (SLOs).

The two groups don’t share the same metrics, but they do share the same broader mission of unlocking human creativity. The belief is that giving each group autonomy over how they work and what they optimize for — leading with context instead of control — makes the whole organization better prepared for future growth.

Growth, as Spotify’s Chief HR Officer puts it, is a mantra, and change is a constant. The new structure is not presented as the final iteration. The company is still expanding into new markets and languages, investing in podcast topic search, and improving retrieval. The Search org will likely evolve again — but this time with an organizational model designed to adapt.