From Heroics to Headroom: Slack’s Reliability Metric
Early-stage startups often rely on a “hero engineer” to keep services running. This individual—or a small, dedicated operations team—can react to incidents and fix systemic issues manually. While this approach works at a small scale, it does not hold up as a product and organization expand. Complexity grows, knowledge becomes tribal, and burnout becomes a real risk as a few people shoulder the entire burden of system health.
Slack’s reliability practitioners have long used incident management processes and service ownership to counter this trend. However, these efforts require a common yardstick to measure reliability and drive prioritization. That yardstick is the Service Delivery Index (SDI), a metric that aligns teams, leadership, and customer expectations.
Defining the Service Delivery Index
The Service Delivery Index – Reliability (SDI-R) is a composite measurement of how well Slack delivers on the jobs users need to accomplish—such as sending a message, loading a channel, or using a huddle—combined with uptime as reported on the public Slack System Status site. It also incorporates successful API calls and content delivery measured at the network edge.
The math is expressed in two steps:
API Availability
availability api = successful requests / total requests
Overall Availability
availability overall = uptime status site * availability api
Uptime and availability are distinct concepts here. Uptime is derived from monitoring critical user workflows: if availability of any key interaction drops below a set threshold, the minutes spent below that level count as downtime. Since infinitesimal availability changes (~0.0001) can materially affect the customer experience, Slack converts availability into a “9s” format, where 99% represents two 9s, 99.9% represents three 9s, and so forth.
Daily and hourly aggregates are tracked over time so teams can spot trends, identify regressions, and confirm improvements. Company-wide goals are expressed in terms of how many days per quarter the service meets those availability targets.
Ownership for responding to and triaging regressions that threaten those targets rests with the Reliability Engineering team, but they are not alone in this effort:
- Engineering Leadership: Determines prioritization and removes blockers for systemic and tactical solutions.
- Service Owners: Debug, understand, and mitigate root causes of regressions while improving the services they own.
- Reliability Engineering: Assists service owners, develops tooling, and identifies threats to continued availability.
All parties blend SDI-R regressions with incident and customer impact data to decide which issues deserve attention and drive them to closure. Slack has found that treating SDI-R like a “canary in the coal mine”—rather than waiting for a formal incident—enables proactive fixes. Issues caught early are easier to debug, offer more time for a well-scoped solution, and are frequently resolved before users notice any impact at all.
From a Concept to a Program
The SDI originated from an idea by Slack’s Chief Architect, Keith Adams, who set out to quantify quality along four axes: Security, Performance, Quality, and Reliability.
- Security: Speed of addressing vulnerabilities, tracked via ticket close rate.
- Performance: Timeliness of responses to customers, tracked via API latency or client performance.
- Quality: Speed of addressing open software defects, tracked via ticket close rate.
- Reliability: Consistency in delivering requests, tracked via error rates.
Each of these areas has grown into its own program with company-wide key metrics. The Reliability program exists because Slack maintains a 99.99% availability SLA in customer agreements. Reaching such a high bar requires, first, visibility into how close the service is to meeting that standard; that visibility comes next.
Once the signal is clear, Slack introduces accountability by publishing the metric to a leadership or company-wide group and embedding a reliability objective into planning cycles. Publishing a key result is not sufficient on its own; the company maps SDI-R regressions to specific services, and those services to owning teams. With that link in place, teams can prioritize fixes or tradeoffs before a regression breaches the SLA.
Scaling Response, Learning, and Prioritization
In practice, SDI-R functions as an error budget. It clarifies when teams should be shipping new features and when they must shift focus to availability. Having a shared view of user impact helps balance investment priorities across the company.
Because Slack is committed to Service Ownership, it has invested in tooling and processes that scale both understanding and resolution of issues that drag down SDI-R. Two tools stand out. The first, the Webapp Ownership Tool, automates the setup of alerts, service level objectives (SLOs), and dashboards for Slack API endpoints using a consistent set of metrics and infrastructure. In many cases, service owners resolve an alert before it evolves into a registered SDI-R regression. The second, Omni, is Slack’s Service Catalog—a system of record for ownership and escalation. Omni displays SDI-R data alongside owned APIs and infrastructure, making it possible to escalate issues in dependencies and automatically route regressions to the correct team.
These tools handle acute issues, but organizational alignment requires more permanent structures. The Engineering Monday Meeting serves as a recurring forum where leadership reviews ongoing customer issues and SDI-R regressions to decide whether to re-prioritize investments. Group- and team-level SDI-R aggregates also flow into these sessions, letting teams track their own performance over time. Often, these reports surface chronic issues that degrade the customer experience slowly, escaping detection by traditional incident and alerting processes.
Evolution and Adaptation
SDI-R has gone through several refinements since its inception. First, not every API call weighs equally. Regressions affecting only a few users can be substantial to those users while registering as tiny blips in the overall metric. In response, Slack now segments SDI-R data for its largest organizations and weights APIs by importance to more accurately reflect customer impact. Large customers often encounter issues first because they exercise the products and infrastructure most heavily, so a segmented view allows the team to fix problems proactively, before they affect the broader population.
Second, because SDI-R reporting is inherently somewhat delayed, there was originally a lag between an issue occurring and its appearance in the metric. This disconnect has become less significant as service-specific alerting has matured; by the time an issue affects SDI-R, it is usually already caught by an alert. Slack now places more value on keeping availability headroom by fixing issues proactively before targets are at risk, which in turn reduces operational load and keeps debugging skills sharp.
Third, the approaches themselves must evolve. New products, changing infrastructure, and shifting customer needs require Slack to keep one eye on the metrics to make sure they are still measuring what matters. Slack now applies SDI-R to new products and infrastructure, including the GovSlack environment. The company stresses that no business is static, and that reliability improves by learning from failures and iterating on the measurement over time.
For a growing company, balancing availability with feature velocity is hard. Slack’s SDI-R program points a way forward: a clear, data-backed scoreboard that ladders to teams and actions, backed by forums to reallocate effort as soon as the numbers demand it.
