The problem: scattered metric definitions

Metrics underpin data-driven decision-making, but in many organizations they end up fragmented, inconsistently defined, and hard to discover or trust. At Netflix the issue surfaced first in the experimentation platform, where metrics are the basis for interpreting test results and the original tooling made both definitions and values difficult to understand and introspect.

Scattered definitions, redundant datasets, and duplicated pipelines are a natural side effect of a fast-moving company with a large analytical workforce. A well-governed, centralized semantic layer addresses this, but solving it across departments with differing rules and regulations is hard.

For the Experimentation Platform team the legacy system demanded extensive business knowledge, familiarity with experimentation conventions, and complex engineering practices. Metric authoring could take weeks — prohibitive for the data scientists and analytics engineers trying to onboard new metrics.

Why DataJunction

Netflix evaluated several candidates: an internal framework called Mando, Airbnb's open-source Minerva semantic layer, and DataJunction (DJ), also open source. Each had strengths, but DJ stood out for a solid SQL parsing and SQL generation engine. It was also, at proof-of-concept time, the only open-source option that stored all its definitions in a connected graph, similar to how relational databases store metadata on views and queries. That combination — easy SQL wrangling with rich metadata — is what moved it into production use.

DJ traces back to an internal Facebook project from around 2015, built by the data science and infrastructure (DSI) team largely to solve metrics portability. Deltoid, then Facebook's main experimentation platform, forced users to rebuild and repopulate their metrics on each platform version; abstracting metric definitions separated the metrics from the stats computation platform and allowed seamless migrations. That work was renamed MDF (Metrics and Dimensions Factory) and became a general semantic layer used across departments. In 2021 Beto Dealmeida started the public DJ GitHub project with basic functionality, later expanded with Netflix engineers. Three Netflix engineers plus external contributors, most notably Nick Ouellet, built out the remaining platform features.

Building under constraints

Resource scarcity shaped DJ at Netflix. Rather than trade away product quality, the team limited scope. Lacking official support and adequate resources, they developed DJ as a side project for more than two years, hoping demonstrated value would eventually draw management attention.

API first, interfaces later

A metrics platform does not need a flashy front end to deliver value. The semantic layer is the core: it holds a consistent set of dimensions and metrics and serves them through a rich API. In the Netflix implementation, DJ services sit in the middle — interacting with underlying data on one side and supplying the access layer with SQL plus metadata, or with data directly, depending on the client request.

The full surface of a semantic layer resembles an octopus, reaching into many functional areas:

  • semantic metadata authoring, management, and monitoring
  • metric and dimension query / definition generation
  • metric and dimension data generation (requires a query engine)
  • data visualization and tracking
  • performance optimization (caching or materialization)
  • integration with other tools
  • all of the above exposed as UI, API, client libraries, MCP servers, and more

Best-in-class coverage of every layer was impossible for a small unofficial team, so they concentrated on the piece that matters: a solid semantic layer engine and API that parses SQL as input to the graph and generates SQL as the serving layer. Everything else can be layered on later without rebuilding the core.

Cubes as the consumption protocol

Multi-dimensional analytical cubes have been a fundamental concept for decades across data processing platforms. Definitions and features vary by tool, but all abstract a subset of semantic objects. DJ adopted cubes as first-class features for two reasons: user familiarity and their fit as a serving interface, efficiently delivering both semantic definitions of metrics and dimensions and fast data access.

Basic cubes are easy to create — they aggregate arbitrary collections of shared metrics and dimensions. Accessing, managing, and visualizing them is straightforward for any DJ client. Custom integrations became trivial to build, including one for Netflix's internal fast Druid analysis tool, Data Explorer, and cubes can be materialized directly in Netflix's Druid clusters.

One of the most powerful integrations links any DJ-originating metric definition back into the DJ ecosystem, producing a feedback loop: when a dashboard displays DJ-defined metrics with links to their definitions, troubleshooting and introspection become simple, and modifying an existing metric for ad-hoc analysis is a trivial step. This works with in-house tools and with open-source products such as Superset.

Metrics in experimentation

Metrics are the backbone of experimentation: they tell us whether a change is meaningful, safe, and worth scaling. Without trustworthy metrics and dimensions, experiments become inconclusive or misleading. Netflix's original metrics layer, integrated into the experimentation platform, was overly complicated — new metrics required tribal knowledge of experimentation data artifacts and familiarity with complex engineering workflows, and onboarding a single metric could take weeks.

DataJunction provides a central hub for defining metrics and dimensions that can be shared and explored through the semantic graph, plus flexible interfaces (UI, API, and various clients) for authoring and managing metrics and other semantic models.

Next steps

With the experimentation domain well supported, the next goal is unifying general analytics at Netflix so users can find and use metric definitions across verticals defined in DJ, and easily access, modify, and experiment with new metrics across the data stack.

That requires strong integrations between DJ and other tools — work already underway, and by the authors' account one of the most enjoyable parts of the project, since most of the capability exists and the task is enabling existing systems to exchange information in a structured, reliable way. With a system like DataJunction, Netflix's internal LLMs can deliver accountable information about business metrics and explain how they are defined and used.