Reliability as a Dedicated Discipline

At Netflix, keeping customers streaming depends on more than just well-built services — it depends on a system that keeps working under real-world conditions. Many teams contribute to that reliability, each owning the uptime of their own services. But there is one team whose sole mandate is the reliability of the Netflix service as a whole: the Critical Operations and Reliability Engineering (CORE) team.

CORE is composed of Site Reliability Engineers, Applied Resilience Engineers, and Performance Engineers. Unlike most SRE groups, CORE does not own or operate customer-facing services, nor does it routinely ship production code or embed with service teams. Instead, the team's focus is squarely on keeping the service up. That translates into three main activities: identifying systemic risks, managing the full lifecycle of incidents, and providing reliability consulting across the organization.

Stepping In When Things Go Wrong

Netflix follows a service ownership model, meaning the teams that build a service also operate it. This works well most of the time — service owners usually detect and fix issues before customers are affected. When an issue does escalate into a customer-impacting incident, CORE takes over.

The team configures and maintains alerts that monitor high-level business KPIs, such as stream starts per second. When one of these alerts fires, the CORE on-call engineer determines the scope of impact, identifies the services involved, and brings in the appropriate service owners to start mitigation. From that point, the on-call engineer typically takes on the role of Incident Manager.

Incident management at Netflix does not follow standard models like ITIL. Instead, the Incident Manager performs or delegates several key functions:

  • Coordination — pulling in relevant service owners to focus on investigation and mitigation
  • Decision Making — making calls that facilitate mitigation, such as whether to evacuate a region
  • Scribe — documenting incident details, including involved teams, mitigation steps, and impact graphs
  • Technical Sleuthing — helping service owners understand which systems are contributing to the incident
  • Liaison — communicating incident status across internal and external business functions

Learning After the Impact

Once customer impact is mitigated, CORE coordinates the post-incident analysis. The most common approach is "memorialization": a written account of what happened, what mitigations were applied, and what follow-up work was discussed. For incidents that are particularly unique, interesting, or impactful, CORE may host an Incident Review or conduct a deeper, long-form investigation.

Much of this analysis, especially for impactful incidents, is done with an Applied Resilience Engineer. All post-incident work examines the sociotechnical aspects of an incident — the interplay between people, processes, and technology. This lens tends to surface practical learnings that CORE actively socializes beyond the directly involved teams to share insights across the company.

Beyond On-Call

When CORE engineers are not responding to incidents or analyzing them, their work varies widely based on individual skills and interests. In broad strokes, that includes:

  • Preserving operational visibility by fixing and improving dashboards, alerts, and automation
  • Providing reliability consulting on architectural decisions, observability, application performance, and on-call health training
  • Partnering with teams to identify and fix systematic risks revealed by incidents
  • Building and maintaining internal tooling that supports incident response
  • Continuously learning the changes in a complex, fast-moving system
  • Maintaining relationships with other teams

Netflix has found that this centralized model of reliability work suits its needs well. Because CORE is not tied to any single service, it has the bandwidth to proactively explore business-critical risks and respond to them effectively. Its broad view of the system makes it easier to spot systemic issues as they develop, share learnings efficiently across engineering, and consult with teams on short notice. By keeping reliability as its singular focus, CORE is able to reveal sociotechnical risks, coordinate effective responses, and help ensure Netflix continues to deliver.