Why Spotify rebuilt its ML platform around Ray

Spotify’s ML platform team has a mandate: make production machine learning reliable, responsible, and accessible to every ML practitioner at the company. Since the platform’s founding in 2018, that has meant building centralized tooling for the majority of internal ML teams. But a closer look at user demographics revealed an imbalance—while most ML engineers rely on the platform’s standardized stack, data scientists and researchers are underrepresented. The root cause is that the platform was designed first and foremost for one specific workflow: TensorFlow-based supervised learning in production.

That single-journey focus is too narrow for a company where ML ideation happens across many disciplines and tools. Spotify’s internal research indicates that the platform needs to broaden its appeal by extending production support beyond TensorFlow, simplifying access to GPUs and distributed compute, speeding up the research-to-prototype loop, and providing a path to production for advanced paradigms like reinforcement learning and graph neural networks.

The lifecycle gap: research versus production

ML work at Spotify flows through a funnel. At the wide end, data and research scientists rapidly test ideas using diverse, heterogeneous methods that resist standardization. As promising ideas prove out, ML engineers take over, and standardization becomes critical for reliability and speed to market. The platform was built to serve the narrow end—ML engineers—because their needs were easier to codify. That left the wide end underserved, making it harder for innovators to experiment and harder still to transition their prototypes into production systems.

In 2022, the platform team refreshed its strategy to close that gap. The goal is to meet practitioners where they are, giving them flexible infrastructure that supports both day-one experimentation and hardened production pipelines with minimal friction.

Ray as the new foundation

After exploring options, Spotify settled on Ray, an open-source framework for scaling AI and Python workloads. Ray’s value proposition is simple: it scales compute-heavy tasks—feature engineering, training, hyperparameter tuning, batch inference—without forcing developers to rewrite their code or abandon their preferred ML libraries. That lowers the barrier between local notebooks and distributed clusters, a critical requirement for Spotify’s diverse user base. Ray is already widely adopted in the industry, with high-profile references like OpenAI using it to train large-scale models.

Building Spotify-Ray: a managed platform

Spotify built a centralized platform called Spotify-Ray so that practitioners don’t have to manage Ray or Kubernetes infrastructure themselves. The platform combines server-side infrastructure, a client-side SDK and CLI, and integrations with the broader Spotify engineering ecosystem. The design goals were accessibility, flexibility, availability, and performance.

Accessibility: gradual learning curve

The platform practices progressive disclosure of complexity. New users get sensible defaults, while power users can dig into deeper abstractions. A single CLI command spins up a Ray cluster with preinstalled ML tooling—including a notebook environment, an in-browser VS Code server, and SSH access. Users can list, describe, scale, customize, and delete clusters through the CLI or equivalently via a Python SDK.

Underneath, Spotify-Ray runs on Google Kubernetes Engine (GKE) with the open-source KubeRay operator. The CLI creates a custom Ray cluster resource; KubeRay translates that into a live cluster. New users start in a shared playground namespace before graduating to team-specific namespaces, with permissions and resource limits configured automatically from a team configuration file.

Flexibility: library and hardware choice

Because Ray is framework-agnostic, Spotify-Ray supports PyTorch, TensorFlow, XGBoost, and other major libraries out of the box. Computing resources are configured through a unified abstraction, but users can customize their requests—for example, picking a specific GPU type and count—without touching Kubernetes YAML.

Availability and performance

By building on managed GKE, Spotify avoids the operational burden of self-managing Kubernetes. Each Ray worker runs on its own GKE node, and each team gets an isolated Kubernetes namespace. To speed up container startup, the platform leverages GKE’s image streaming, slashing GPU image pull times from several minutes to a few seconds.

From prototyping to production

The initial version of Spotify-Ray focuses on the funnel’s mouth: early stage research and experimentation. That’s deliberate—it’s where the platform had the biggest gap. But the team sees Ray as a bridge across the whole lifecycle. With native integration with Spotify’s Flyte orchestration service and high-level APIs planned for data loading, artifact logging, and experiment tracking, Ray could become the backbone of a path to production that serves both the data scientist prototyping next quarter’s idea and the engineer hardening today’s critical workflows.

Graph Learning for Recommendations at Spotify

Spotify’s Tech Research group—historically an underinvested ML Platform end user—recently set out to evaluate graph neural network (GNN) algorithms for content recommendations. Unlike typical research prototypes that run on ad hoc tooling before a production rewrite, this project needed to be production-ready from the start to quickly assess GNNs against real business use cases. The researchers required infrastructure that was both flexible and easy to productionize, which led them to build the graph learning implementation on Spotify-Ray.

After promising offline results on internal datasets, the team launched an A/B test to measure how GNN-based algorithms affected the home page’s “Shows you might like” recommendations. Running these tests posed a challenge because GNN workflows diverge materially from standard ML pipelines. Spotify-Ray was adopted as the infrastructural foundation, and the team implemented a set of components to train and deploy GNN models at scale:

  • Data creation (graph construction): Real-world graph construction is iterative, requiring tools that transform large volumes of data using simple Python functions. Ray Datasets were used to build the graph from the data warehouse, leveraging flexible APIs for common transformations like distributed mapping.
  • Feature preprocessing: The constructed graph consisted of nodes and edges with associated features. The team used Ray AIR’s default preprocessors and extended their base API for feature transformations—standardization, categorical transforms, bucketing, and more.
  • Graph learning: Graph data and preprocessed features were fed into a graph learning algorithm implemented in PyG. Ray trainers extend readily to frameworks like PyG, enabling seamless distributed training.
  • Inference at scale and evaluation: Custom predictors for batch inference and evaluators were built on top of Ray Datasets.

With these building blocks, the Tech Research team assembled an end-to-end pipeline for show recommendations using GNN-based models and launched an A/B test in under three months—a timeline that was extremely difficult to hit with prior supported ML infrastructure. The test yielded significant metric improvements and a better user experience on the home page’s “Shows you might like.”

What’s Next for Ray at Spotify

Demand for PyTorch among Spotify’s ML practitioners has grown considerably, especially for emerging NLP and GNN use cases. The plan is to use Ray to support and scale PyTorch to meet that demand, helping users across different roles stay productive.

Introducing a new framework carries a risk of fragmentation. However, with stronger foundational building blocks in place, Spotify aims to build a more flexible, representative, and responsible ML platform experience that unlocks broader ML innovation.