Part 3: A Survey of Analytics Engineering Work at Netflix

Designing Dashboards That Work

Design is often mistaken for visual polish, but its real purpose is function: making it easy for users to reach their goals. Good design doesn't waste time or mental energy. A practical shortcut is to rely on established patterns—users already understand that blue, underlined text is a link, for instance—so knowing which patterns exist and what they imply helps you choose the right one for the job.

The first step is understanding your user. Talk to them early and often, throughout the product lifecycle. Ask why, then ask why again, to separate symptoms from problems from solutions. Prioritize ruthlessly: less is more. Distill your offering to what is differentiated and delivers the most value.

Think about where users fall on key axes—do they need to explore or just monitor? Is their task frequent or occasional? Don't try to serve multiple positions on these axes within a single view. If different user groups need different things, create separate views or separate dashboards.

Structure around mental models. A user's mental model should dictate your information architecture. Consider these questions:

• Do different user groups need to accomplish different things? Split them into different apps or views.
• What belongs on one page? All the information a single user type needs to complete a single “job.” Different jobs get different pages.
• What belongs in one section? Information needed to answer a single question.
• Does the dashboard feel difficult? You likely have too much information. When in doubt, keep it simple—hide complexity under an “Advanced” section if necessary.

For page layout, lead with the most-used information above the fold. Choose infinite scrolling or pagination based on user expectations. Use clear labels and signposts so users always know where they are. Group related items with cards or borders, and use nesting to create well-understood “scopes of control”: users expect a controller to affect elements below it (horizontal layout) or to its right (vertical layout).

Interactive chart design. Charts present unique challenges. Keep these tips in mind:

• Titles: Reflect active filters in the title or subtitle so the chart is scannable and screenshot-friendly.
• Tooltips: Put core details on the page; use tooltips for deeper context. Annotate multiple points when you have only a few lines.
• Annotations: Explain shifts in values directly on the chart so all users have that context.
• Color: Use a limited, consistent palette. Too many colors or inconsistent meaning dilute the signal.
• Onboarding: Separate first-time onboarding from routine usage flows.

These are guidelines, not rigid rules. Adapt them with good judgment to fit your product. The goal is always the same: users must be able to leverage the data insights to perform their work. Good design is the means to that end.

Building an Analytics API in Production: Lessons Learned

At Netflix Studio, data augments the expertise of creative professionals. One use case is production budgeting: determining what a show or movie should cost to produce. Comparing new budgets against similar past projects was highly manual, so the team built a tool that automatically selects comparable productions and flags anomalies for Finance review.

Real-time delivery, integrated into existing tools, was essential. The team developed a GraphQL endpoint using Metaflow and embedded it into the budgeting product. The MVP is live, but the road to launch was more complex than expected. For an analytics engineer used to quick solutions, deploying a production-grade API required much more effort.

Here are the key takeaways from that experience.

Is real-time actually required? Assess whether real-time results are truly necessary before committing. Batch processing can deliver comparable impact with far less complexity, and it is easier to develop and maintain. For proof-of-concept work, the upfront investment in real-time infrastructure is often not worth it. Scrappy solutions are frequently the best choice in analytics.

Explore all available solutions. Netflix had multiple established methods for building APIs, but none fit this use case perfectly. Metaflow already supported REST APIs, but that did not align with the engineering team's preferred workflow. Large response sizes made REST integration unreliable, requiring extra filter parameters. The target product used GraphQL, so deviating from that pattern was not ideal. Federation—a GraphQL feature—was also advantageous for overlaying results throughout the product. Since no existing solution at Netflix deployed Python endpoints with GraphQL, the team worked with the Metaflow team to build this capability, letting them keep developing in Metaflow while engineers stayed on their paved path.

Align on performance expectations early. Managing API latency was a major challenge, and much of it was preventable by aligning on expectations from the outset. The team's assumptions about acceptable response times differed significantly from what users and engineering partners actually needed. The full budget analysis took about 7 seconds on average. Users accepted that wait when modifying a budget, but not every time they opened a budget. Caching, implemented with Metaflow, reduced response times to roughly 1 second for cached results, with a nightly batch job pre-populating the cache. Even so, the team had to stay mindful of GraphQL's 30-second limit, which made continuous monitoring of response-time impact essential.

Real-time analysis demands rigorous testing. Load testing with Locust measured endpoint response times under reasonable and elevated loads. FullStory, already used in the product, helped estimate expected calls per minute. Unit tests and full integration tests were implemented to ensure the analysis returned correct results—especially important when end users would otherwise be the first to encounter errors or incorrect information.

Align workflows to collaborate effectively. This project was the first time the DSE team integrated an API directly into an engineering partner's product, and both sides had gaps in understanding each other's workflows. Deployment paths exposed this clearly: engineering partners followed a strict path, while the DSE side was more flexible, testing on Metaflow feature branches before pushing to production. That flexibility caused issues, like inadvertently deploying production changes before the product updates were ready and struggling to manage a test endpoint. The team deferred to engineering partners to define the deployment path, working with the Metaflow team and data engineers to implement it. Work planning was another mismatch—engineering worked in sprints, while DSE planned quarterly—and reconciling those cycles remains an ongoing challenge.

The partnership between teams has required significant investment but is expected to yield substantial benefits on future projects.

Beyond the Sessions

The discussions at the Summit were framed by a keynote from Benn Stancil, founder of Mode Analytics, who walked through the history of the modern data stack and led a conversation about the future of analytics.

Analytics Engineering is a key part of Netflix's data culture, with a large group of practitioners both applying and advancing analytical capabilities across the company. The annual Analytics Summit offers visibility into work across business verticals, celebrates collective impact, and highlights what is next for analytics practice at Netflix.