Building ML Home: Product Lessons from Spotify’s Internal ML Portal
Platform teams face a notoriously difficult challenge: building tools for machine learning practitioners, a group known for bespoke needs and a strong do-it-yourself ethos. Spotify’s internal Machine Learning Platform team took on this challenge with ML Home, the user interface for its ML Platform. The goal was to create a central hub for ML workflows, project metadata, and collaboration across the company’s growing applied ML ecosystem.
Spotify’s ML Platform was built on a “walking skeleton” approach, starting with an end-to-end workflow and then deepening functionality once value was proven. The early components covered the supervised ML lifecycle, including workflow standardization via Spotify Kubeflow (based on TFX), feature engineering with Jukebox (TensorFlow Transform), model serving through Salem (TensorFlow Serving), and audio processing with Klio (Apache Beam and Dataflow).
As more teams onboarded, the platform team identified two critical gaps: a centralized metadata layer for defining platform entities and relationships, and a metadata presentation layer where users could store, track, and manage workflow metadata. The latter became the focus of ML Home.
Balancing Vision with Iterative Strategy
Initial user research revealed that teams were relying on spreadsheets to track ML metadata, along with hyper-specific feature requests and broader unmet needs such as project discovery and team collaboration. In response, the team designed an MVP using a T-shaped approach: horizontal features addressing common needs like collaboration, paired with a vertical solution for a specific platform gap — better offline evaluation tooling for ML engineers.
Feedback on the MVP split along predictable lines. Individual contributors wanted concrete, immediate problems solved, while leadership asked about long-term user scale and impact. The team learned that conflating product vision with product strategy was a pitfall. Scoping down to only concrete needs would have produced a narrow point solution, while over-indexing on broad ambitions would have yielded a nice-to-have without must-have utility.
By keeping the product vision broad and future-looking (“one-stop shop”), the team retained runway for bigger solutions. At the same time, an iterative and concrete product strategy — starting with offline evaluation — ensured that daily workflows improved incrementally and the product became entrenched over time.
The Real Value of an MVP
Adoption after the MVP launch hit a ceiling quickly. Aside from early co-designers, few users switched from existing tooling. Users understood the value proposition but did not see enough depth to make the change. The team’s expectations about what the MVP would deliver did not match the depth of user needs.
The lesson: the most important outcome of an MVP is not user volume but validation. The MVP succeeded because it de-risked the strategy, gathered detailed workflow feedback from engaged ML teams, and laid the technical foundations. Daily active user counts mattered less than having enough users who saw value and continued to participate in feedback sessions, steering the product toward a more valuable state.
Differentiation Through Ecosystem, Not Features
Post-MVP, the team faced a feature-parity problem. To make ML Home compelling, they initially hunted for a single “silver bullet” feature that would tip adoption. After probing and stack-ranking workflows, the conclusion was that ML Home’s differentiators were not individual features but the integration with the rest of the ML Platform.
Enriched with training, evaluation, and system metadata from platform components, ML Home delivered cumulative value greater than any one function could. Annotation capabilities such as tagging and notes allowed teams to customize the interface to mirror their own workflows. When paired with faster interactions and information-rich model comparisons, these capabilities tipped the balance.
Work in this phase centered on consolidating entities and concepts across the platform via the metadata service. By launch of the second version, more ML teams were actively using ML Home in daily practice.
Scaling Toward the One-Stop Shop
With clearer product-market fit, the team expanded the scope beyond the single workflow the MVP served. The insight that deep integration beat stand-alone features guided the roadmap. Today, ML Home surfaces artifacts and workflow metadata for all models passing through platform components, supporting offline experiment tracking, evaluation visualization, deployed model monitoring, feature exploration, and production readiness certification.
The product now hosts 220+ ML projects at Spotify. Daily active users grew 200% during the past year of scaling, and ML Home is embedded in the workflows of many of the company’s most important ML teams.
Three Takeaways
Reflecting on the development process, three lessons stand out:
- Balance vision with strategy. An inspiring, future-looking vision and a concrete, iterative product strategy serve different purposes. Conflating them early in development limits both.
- MVPs validate more than they deliver. Treat an MVP primarily as a tool to test and de-risk direction, not as a finished product expected to drive adoption.
- Know the true differentiators. The winning advantage may not be a standout feature. It can come from ecosystem integration and opportunities that make switching worthwhile.
Still in Progress
ML Home is far from finished. The ML community has proposed new capabilities, including production readiness certification, built directly into the interface. Work is also exploring explainability for model interpretability and observability for model health tracking, in addition to inspirations from hack week projects. Growth continues as Spotify’s teams find inventive ways to evolve the platform.
One Interface for the Whole ML Lifecycle
Spotify’s ML practitioners used to juggle a tangle of disconnected tools. Experiment tracking lived in one place, model registration in another, and feature documentation somewhere else entirely. ML Home consolidates those workflows into a single entry point, giving engineers one searchable surface for everything from feature discovery to model deployment.
The platform was built on a simple observation: most ML work is not writing models. It is finding the right dataset, checking whether a feature already exists, reviewing a previous experiment, or debugging a failed training run. Each of those tasks previously required context-switching across several internal systems, and the cognitive cost added up quickly.
The Shape of the Product
ML Home is organized around the objects ML practitioners actually interact with daily: projects, features, models, and experiments. Rather than forcing users to navigate by team structure or repository layout, the home page surfaces recent activity and frequently accessed items. Search is the primary navigation mechanism, and it spans all object types at once.
Each object type has a dedicated detail view. Feature pages, for instance, display ownership, data type, lineage, and production status in one place. Model pages link to their training experiments and deployment history. This cross-linking matters because the same logical entity often shows up under different names in different systems—a feature might be called user_listening_minutes in one pipeline and active_minutes in another.
Building for Trust, Not Just Convenience
The team quickly learned that a portal is only useful if people trust its contents. Early feedback showed that engineers would search for a feature, find a promising match, but then go verify the result in the original source system before using it. That double-checking defeated the purpose of consolidation.
Spotify addressed this by making provenance explicit and unavoidable. Every object in ML Home shows its source system, last updated timestamp, and owning team. When a model card says a model is in production, clicking through reveals the exact deployment record. The interface does not try to replace source systems—it treats them as the system of record and positions itself as an aggregator with transparent links back to origin.
The goal was not to create a new home for ML data, but a reliable front door to the homes that already exist.
Search as a Discovery Tool
One of the more surprising outcomes was how search changed discovery behavior. Previously, engineers tended to reuse the features and models they already knew about, simply because finding alternatives was too costly. With unified search, users began stumbling onto relevant assets they did not know existed.
This had a measurable effect on duplication. Teams that searched before starting new work were more likely to adopt an existing feature than build a parallel one. The search index includes not just names and descriptions but also metadata like data types, owner teams, and associated projects, which surfaces relevant results even when the searcher does not know the exact terminology.
What the Build Process Taught Them
Several lessons from building ML Home apply well beyond Spotify’s specific stack:
- Start from workflows, not object models. The first prototypes followed the team’s internal data model, and users found them confusing. The design that worked traced actual daily workflows: search first, filter, then drill into details.
- An aggregated view is only as good as its freshness. The team invested heavily in synchronization pipelines before building UI features, because stale data is worse than no data—it actively undermines trust.
- Adoption requires solving a universal pain point. Discovery and search were struggles shared across all ML roles, while other potential features appealed only to narrow segments. The team prioritized the universal needs first.
- Ownership must be visible by default. Displaying owning teams on every card and detail page reduced the friction of asking questions and encouraged direct collaboration between teams.
A Platform Product, Not a Dashboard
ML Home was never conceived as a read-only visualization layer. The team deliberately made it a workspace where actions—registering a model, updating documentation, initiating a deployment—can happen without leaving the interface. That choice shifted the product from a reference tool to a daily driver, and it changed how the team measured success: not by page views, but by whether users could complete an entire ML task from start to finish within the platform.
Because it is an internal platform, the product team could iterate quickly and incorporate feedback loops that would be impractical for external software. Weekly user interviews and telemetry on search behavior fed directly into the roadmap. The result is a tool that evolved its structure based on observed usage patterns rather than upfront assumptions about what ML engineers want.
