Inside the intelligence layer behind AI Discover

AI Discover is a new tab in the TV app, offered to a small group of members, that replaces silent ranking with steerable discovery: a member picks a theme such as "Quick Watches," receives a row of matching titles with a short note on why they fit, and gets narrower themes to try next.

Delivering this with generative AI on TV required a new intelligence layer — the agent harness behind AI Discover. For each request, it decides how to combine LLMs with the existing recommendation systems, then plans, executes, serves and evaluates the result. Below are the challenges the team designed for and the decisions that followed.

Why a dedicated harness

Every Netflix page blends recommendations with rules: titles a member may enjoy, filtered by eligibility, maturity and safety rules. An LLM-based experience needs the same governance plus code that decides when an LLM is called and how its output is handled. That code is the agent harness.

Handing an entire request to an LLM would leave governed results to probabilistic output — unacceptable when every turn ends on a living-room screen, and when rare failures occur many times a day at scale. Seven constraints shaped the design:

  1. Fitting LLMs into an architecture not built for them. Traditional recommenders rank on past behavior; LLMs can additionally interpret a request and plan a response.
  2. Grounding recommendations. Every pick must be a real, available title from a changing catalog, chosen via retrieval and rankers working with the LLM.
  3. Handling slow, unpredictable LLM calls. Latency tails can spike and output may not match the ask, so LLMs are used selectively and treated as untrusted input.
  4. Surviving model and prompt churn. The product cannot depend on any single model, tool or prompt.
  5. Building for TV. Typing is hard, hardware varies and apps update slowly, so changes must ship without an app release; a typical TV request returns in moments while multi-step generation takes several seconds.
  6. Holding the safety bar at scale. Members must always get a page, even when an LLM fails, with choices and content standards enforced on every request.
  7. Learning from stated intent. Member-told preferences must shape what they see next, alongside viewing behavior.

Controller, executor, and validated plans

The TV app sends each theme selection to the Page Server, which builds pages for all Netflix apps and passes the request asynchronously to the intelligence layer. That layer has two parts.

The controller decides what to run. It gathers request context, including real-time signals such as time of day, then settles on a plan — either established or newly drafted by its planner — validates it, and pulls only the member details that plan requires from Memory Hub, the per-member memory store. These may include taste summaries from MemberLLM, the in-house language model for personalized recommendations, or member feedback.

A plan is a graph of approved steps, each invoking a capability: a named, versioned unit of work in a family such as retrieval, ranking, generation or packaging, declaring its inputs, outputs, visible context and failure behavior. Because of those declarations the graph is verifiable before execution — every step approved, every input produced, no loops, exactly one page at the end — and the controller checks the result before serving.

The executor runs the plan. Typed outputs pass between steps through declared contracts; independent steps run in parallel, each with its own deadline and fallback. LLM calls are optimized for a single task each — writing row titles, suggesting next themes — and act only inside the validated plan. Both components write to the request records used for offline evaluation.

Seven design decisions

An LLM-native product with composable building blocks

Rather than inserting LLMs as rankers under existing latency budgets, the experience was built around steering: LLMs generate themes, explain rows, summarize taste and help choose titles fit for the member and the moment. Theme selection happens by remote, so no typing is needed, and themes and explanations get native TV UI elements. Behind the screen, LLMs draft plans and asynchronous serving gives multi-step orchestration its time.

Grounding comes from wrapping proven systems — search, large-scale ranking, catalog metadata access. Each capability needs a clear purpose and semantic interface. Granularity is a tradeoff: too fine and the planner has more ways to err; too coarse and it composes less. Plans built only from these blocks stay grounded even when drafted at runtime.

Plan first, then execute

Instead of a reason-call-read loop, every request is planned before anything runs, so step count and path are known up front. Plans come from established, tested set or from an LLM drafted on the fly; a router the team controls chooses, and both face identical checks. If the planner is slow or its plan fails validation, the system falls back to a tested plan. The planner can also trade cost, latency and quality per task, from a lean single-call plan to a richer multi-step one.

Model and capability agnosticism

Each task is assigned to the model or capability version that fits it, currently a per-task setting that can be overridden live; only some steps in a request use an LLM. All LLMs sit behind one common interface, and candidates are scored on identical tasks by automated checks and LLM judges before any swap. Behavioral logic lives in server-side prompts and configuration, so models, tools and capabilities change without new code or an app release — upgrades become plan or setting changes rather than migrations.

Asynchronous page generation

Holding a request open for seconds across services ties up threads, so generation requests are asynchronous. The remote press returns an acknowledgement immediately, the TV shows a thinking state explaining what is being prepared, and the executor works in the background; when the result is ready the server notifies the TV, which fetches it. Slow generation no longer degrades the serving path, and every press gets immediate on-screen feedback.

Safety enforced in code

Rules that must hold regardless of LLM output — content and maturity standards, per-step access, member exclusions — live in code, not prompts. Generated text is cleaned before display; the planner sees a stripped-down request and only the member details the validated plan needs.

Inputs such as the profile's maturity rating are referenced through placeholders rather than literal values; after validation, the controller fills them from the request, matched by declared type — a protection against prompt injection as well. Each request leaves a structured record of the plan, each step and outcome, and the rules applied, so a miss can be traced to plan, data, rule or model. The same records feed offline evaluation and replay.

Memory that spans visits

A taste summary — favorite genres, tone, recurring themes — is generated ahead of time by MemberLLM from viewing history, keeping that LLM work off the request path; steps that write themes and explanations use it as context. Members can also tune recommendations and state what they want directly, with choices applying immediately and carrying across visits. All of it lives in Memory Hub alongside taste summaries, with new memory kinds addable as needed and applied to later requests where relevant, together with real-time context.

Scope at launch

Interaction is limited to selecting themes and tuning options such as "don't recommend"; there is no free-text input or open-ended chat, which keeps the LLM's job narrow by design. Profile settings and safety and privacy policies apply as everywhere else, and safety evaluations of theme labels, row titles and explanations were run before launch.

The layer's governed execution is what makes an experimental, AI-shaped experience shippable at consumer scale — not the choice of any single model.

Open questions

The intelligence layer is built so that more than AI Discover can run on it. Two lines of inquiry guide the next phase: how much members want to shape their own recommendations — steer versus lean back, which themes help, and how to blend stated preferences with existing personalization — and what tradeoffs the technology brings, from production cost and latency as traffic ramps, to how long members will wait for generated results, to attribution when errors originate in the model, the harness or existing systems. Multi-turn sessions, the authors note, demand different evaluation and observability than single requests.

The longer-term aim is a system that improves itself from the record each request leaves behind. Today past requests are replayed against new prompts, models or rules and judged by automated checks, LLM judges and human evaluations; eventually the system should propose improvements, test them safely and retain only what proves better, within privacy and data-use policies. More work may migrate into LLMs over time, and the layer lets the team move that line deliberately.