Why integration needed its own playbook
The Foundation Model for personalization centralizes member preference learning: one model trained on large-scale user interaction histories and content data distributes its learnings to the specialized models behind the Netflix homepage, replacing the cost of maintaining and innovating on each individually. While prior posts covered development of the model and its use of hierarchical multi-task learning for session intent, integration into production systems is less well documented — there is extensive literature on training and inference of large-scale transformer models, but little practical guidance on wiring them into existing applications.
Netflix experimented with three integration approaches — embeddings, subgraph and fine-tuning — driven by differing application constraints: latency requirements, tech stack limitations and appetite for using the full power of the model. All three are now in production.
Approach 1: Embeddings
The embedding approach derives user and item representations from the model: the hidden state of the last user event becomes the profile embedding, and weights from the item tower become item embeddings.
Refreshes run daily. The model is pre-trained from scratch every month, then fine-tuned every day on the latest data; daily fine-tuning also expands the entity id space to cover newly launching titles. Once the daily fine-tuned model is ready, batch inference refreshes profile and item embeddings for publication to the Embedding Store.
Embedding stabilization is essential here. Retraining from scratch produces entirely different embedding spaces between runs because of random initialization, and embeddings drift during daily fine-tuning even though that job warm-starts from the previous day's model. Mapping each day's embeddings into a consistent space lets downstream models consume pre-computed embeddings as features.
The Embedding Store itself is a feature store specializing in embeddings, built by the platform team. It handles versioning and timestamping and exposes interfaces for offline and online access.
This is typically the first and simplest way for an application to use the Foundation Model. Embeddings act as features for other models or as a candidate generation mechanism, retrieving appealing titles for a user or supporting title-to-title recommendations. Existing pipelines absorb embeddings with relative ease, and the expected impact on training and inference costs is smaller than with the other approaches. The drawback is staleness: the gap between embedding computation and downstream inference hurts freshness and limits gains in use cases that need real-time adaptability.
The team's biggest learning was that embeddings offer low cost and high leverage, but only with a resilient generation framework and embedding store behind them. That infrastructure proved valuable enough to justify building a near-realtime embedding generation framework, which can update embeddings from user actions within a session and make both the embeddings and downstream models more adaptive. It cannot serve very large models, but it points toward addressing staleness and recommendation adaptiveness.
Approach 2: The model as a subgraph
Here the Foundation Model decoder stack becomes a subgraph of the application model's graph. The subgraph processes raw user interaction sequences and emits representations that feed the rest of the downstream graph.
Applications can fine-tune the subgraph as part of their own training, potentially outperforming static embeddings. No time gap separates Foundation Model inference from application inference, so the most current learnings always apply, and applications can tap specific Foundation Model layers that the Embedding Store does not expose.
The costs are real. Application models must generate every feature the subgraph needs, adding time, compute and complexity to feature generation jobs, and merging the subgraph enlarges the application model and its inference time. The Foundation Model team mitigates the first problem with reusable code and jobs for more compute-efficient feature generation; for inference, Netflix splits the subgraph so it runs only once per profile per request and is shared across all items in that request.
Subgraph integration suits high-impact use cases where metric gains justify the cost and complexity. It exposes more of the model's power, but complicates training code and forces a trade-off among metric improvement, compute and time.
Approach 3: Fine-tuning and direct use
The third pattern resembles fine-tuning an LLM on domain-specific data. The Foundation Model is trained with a next-token prediction objective over tokens that represent different user interactions, and those interactions carry different weight on different surfaces — recent interactions on trending titles matter more for the 'Trending now' row than older ones. Application teams can fine-tune on product-specific data and let the resulting model power the product directly, choosing full parameter fine-tuning or freezing some layers, and optionally adding an output head with a different objective. A fine-tuning framework supports building custom versions of the model.
A fine-tuned model is optimized for its application's data and objectives, and it doubles as a de facto baseline for new work: teams can skip months of feature engineering and new model stacks by starting from a fine-tuned Foundation Model. The trade-off is organizational — more models and pipelines to maintain — plus the need to carefully optimize latency and Service Level Agreements for each application.
Ongoing work
Work on all three approaches continues. The Machine Learning Platform team is building near-realtime embedding inference to address embedding staleness, while a smaller distilled Foundation Model is in development to cut the inference latency of the subgraph approach. The fine-tuning framework keeps lowering the effort of producing custom models, and the APIs used across the approaches are being refined and standardized for application teams.
