Finding the Familiar in a Sea of New

Spotify's recommendation engine is famous for surfacing new artists and genres through features like Discover Weekly. But personalization isn't just about novelty. A significant part of the listening experience is replaying the content listeners already know they love. The challenge is balancing a homepage that pushes fresh discoveries while simultaneously giving users quick access to their current favorites.

To tackle this, Spotify launched Shortcuts, a set of six playable "entities"—playlists, albums, or podcasts—displayed at the top of the Home screen. The goal was to get listeners back into recently played and heavily rotated content with a single tap. Damien Tardieu, Engineering Manager at Spotify, outlines the methodology behind building the feature in just six months.

Why Shortcuts Exist

Before writing a line of code, the team needed to validate the premise itself. They analyzed user listening behavior to measure how much time is spent on familiar versus new content. The findings were clear: over a given week, a large portion of a user's total listening time is concentrated on a relatively small set of entities—whether a personal playlist, a podcast feed, or a staple like Rap Caviar. Even though users might discover new content on various surfaces like Search or the Library page, most listening is driven by repetition.

This insight shaped the design concept: a condensed, dedicated space on Home for current favorites will reduce the effort required to access the content a user wants to hear "right now." An A/B test placed "Your Heavy Rotation" at the top of the page, with the standard Recently Played shelf directly beneath it. The positive results confirmed the hypothesis that users would listen from the top of Home even when they previously accessed the same content through other parts of the app.

Starting with Simple Baselines

The team resisted the urge to immediately deploy a complex machine learning model. Instead, they set a baseline using a naive heuristic: simply combine content from the existing Recently Played and Heavy Rotation shelves, pick the top six items, and display them in the new space. This approach is enough to estimate the lower-bound ceiling of accuracy for predicting future plays.

The Case for Heuristics

The rationale for iterating on heuristics first was threefold:

  • Provide an immediate improvement to the existing user experience.
  • Enable rapid iteration cycles compared to the overhead of model training and maintenance.
  • Escalate confidence in the offline evaluation metrics before investing in production-ready ML models.

Heuristics are cheaper to monitor and debug than models. If a complex model barely outperforms a simple rule-based algorithm, the heuristic remains the wiser production choice. The team tested various heuristic variants with different decay functions (weighing recent plays more heavily), and different ranking philosophies, such as play frequency versus play duration. The best-performing heuristic was validated by correlating offline calculations—how much listening history gets represented by the top six entities—with actual listening volumes from online A/B tests of the shelf itself.

Moving to a Neural Network

Once the heuristics were refined and well understood, shifting to a machine learning model became a natural next step. The models used the same general data as the heuristics but with a twist: they were trained on sequences of listening history, predicting what the listened-to items were at specific timestamps.

The engineering team used TensorFlow for model building and training. The evaluation standard was Normalized Discounted Cumulative Gain (NDCG), a standard metric in recommender systems. However, the model had a unique constraint: the output ranking had to consist exclusively of content items that appeared in the input. The algorithm wasn't figuring out what's new for the user to hear; it was ranking what the user already played to present back to them.

After over one thousand experiments and months of research, the final architecture combined two key steps. First, a dense network transforms each play event—including the timestamp, whether the play was intentional or passive, and the entity characteristics—into a relevant embedding. Second, these embeddings for a given entity are aggregated and passed into another dense network that assigns a final score, which powers the ranking of those familiar entities. This selected architecture showed a relative improvement of 26.7% over the original heuristic model during offline testing.

From Prototype to Production

Moving the model to handle hundreds of millions of users added an extra layer of complexity. Significant resources went into designing systems that could monitor data flows, authenticate the serving features, and detect potential degradation in recommendation quality.

To achieve this, the team implemented multiple hourly-batch pipelines that generate statistics on the served features and model outputs. These statistics are stored in BigQuery tables and rendered into monitoring dashboards equipped with alert functions. Should user plays spike or drop unexpectedly, the team now has the observability infrastructure to detect and respond to issues efficiently.

Methodology Over Technology

The main lesson from the six-month sprint is that methodology trumps a shiny new tech stack. The project's success hinged not on the novelty of its TensorFlow models, but on the team's willingness to start simple—examining data from several angles, testing hypothesis via A/B tests, and iterating on raw heuristics before ever touching a neural network.

Treating the heuristic as a baseline for comparison allowed the ML team to grasp what truly moved the needle regarding user needs and expectations. Only after establishing vision and metrics was it easy to choose the proper objective function, training data, and network architecture.

The implementation involved an entire cross-functional team spanning UX, user research, and product marketing. The groundwork of proving value early was just as important as the internal engineering. By communicating the opportunity clearly, Spotify got the flexibility and latitude to bring Shortcuts from a hunch to a spearhead feature on the Home screen in only half a year.