Inside the path a request takes to reach Slack

Slack's edge network is built on globally distributed edge regions, or Points of Presence (PoPs), placed in AWS datacenters closer to users. These PoPs reduce latency and connect client traffic back to Slack's primary region in us-east-1, where its core services and storage live. That main region is structured for availability zone (AZ) resilience, so failures in a single datacenter can be absorbed without broad impact.

Incoming traffic splits into two categories: websocket and non-websocket. The edge regions run a set of services, collectively called Edge Services, that handle both. Before any of that happens, though, the client has to find its way to the edge.

DNS: finding the nearest PoP

When a user launches the Slack client, it issues background DNS queries for domains such as slack.com, wss-primary.slack.com, and slack-edge.com. If the answers aren't cached locally, the queries flow through a recursive resolver and land at the authoritative name server.

DNS resolution steps before packets arrive Slack infrastructure. 1) User DNS request is routed to the DNS Recursive Resolver. 2) DNS Recursive Resolver queries Root Servers, then Top Level Domain (TLD) servers, and finally Authoritative name server, which is Route53/NS1 based on traffic type. 3) Once the DNS request is resolved, traffic arrives at Slack infrastructure fronted by NLBs.

Slack uses Amazon Route53 (R53) as the authoritative name server for most domains. The websocket records, however, are subdelegated to NS1, which Slack chose for its configurable Filter Chain and load-shedding capabilities. Both DNS providers return IPs for the region closest to the user, using the edns-client-subnet extension (ECS) defined in RFC 7871. ECS lets recursive resolvers forward the client's originating network information to the authoritative server, enabling geo-aware responses.

The returned list contains public-facing Network Load Balancers (NLBs) fronting either the websocket or non-websocket stacks. These NLBs operate as layer-4 pass-through devices; they forward packets to the appropriate layer-7 load balancing tier at the edge without inspecting application data. The client picks an IP from the list, opens an HTTPS connection, and upgrades it to a websocket — and then it's ready to send messages.

When packets travel from a regional PoP toward the main region, they stay encrypted thanks to Nebula and route over the AWS backbone. That keeps traffic off the public internet and reduces exposure to external fiber cuts or peering issues.

Websocket traffic and failover paths

Websocket connections carry Slack's real-time messaging. They're ingested by envoy-wss, a set of Envoy Proxy servers dedicated to websocket traffic, and are reachable via wss-primary.slack.com and wss-backup.slack.com. NS1 manages these DNS records.

1) Websocket traffic after resolved by NS1 is sent to NLBs then envoy-wss service in Edge PoP. 2) Depending on the request type, traffic is either forwarded to the Gatewayserver in the edge PoP or envoy-wss in the us-east-1 region. 3) us-east-1 envoy-wss requests are sent to Applink and then to WebApp.

For a wss-primary.slack.com query, NS1 applies a Filter Chain that routes traffic to NLBs in the closest region that is up and has capacity. The filter logic depends on the client subnet information provided by ECS.

Filters in the Filter chain are in the following order: 1) Up 2) Shed Load 3) Geotarget Country 4) Select First N (N=1)

Once traffic reaches the NLB in a region, it lands on envoy-wss, where TLS termination happens. Based on the request parameters, the connection is routed to one of two internal services:

  • Gatewayserver — maintains websocket connections for Slack's own clients.
  • Applink — lets customer-built apps connect over websocket to receive their event subscriptions.

Primary-to-backup websocket failover

If wss-primary.slack.com becomes unreachable, clients fall back to wss-backup.slack.com with retries, backoff, and jitter designed to make the transition safe. The difference between those two domains is entirely in the NS1 Filter Chain: where the primary record points to the single closest healthy region, the backup record selects the three closest regions instead.

A title "Total Sockets" at the top of the graph. Legend includes "Primary websockets" (green) and "Backup websockets" (yellow). The x-axis includes dates which are for 7-days and the y-axis indicates the number of sockets. Two lines plot: green to represent primary websocket and yellow for backup websocket. The backup websocket count is very small as compared to primary websocket. Filters in the Filter chain are in the following order: 1) Up 2) Shed Load 3) Geotarget Country 4) Select First N (N=3) 5) Shuffle 6) Select First N (N=1)

That spread has a useful side effect. If one of those regions fails, traffic is distributed across the remaining two plus the failed one, so a third of new backup connections might still hit the broken region. Those clients retry again quickly — the DNS TTL is short — but the initial redistribution avoids a thundering-herd problem where everyone rushes to the same alternate PoP.

Degraded mode when websockets fail entirely

Websocket failure is planned for harder scenarios too. If a client can't establish a websocket to either primary or backup, it enters Degraded Mode. In that state, a desktop or supported browser client can still talk to the Slack API, just not over a persistent websocket.

With degraded mode, Slack can still function and not be completely unusable just like the house.

Degraded Mode appears as a gray banner in the client. Users can still do most basic tasks, like sending messages, browsing channels, and clicking "Load more messages," which fetches new content through regular API calls. The banner disappears as soon as a websocket is re-established. Slack credits this fallback for helping keep clients usable during regional incidents and for supporting high customer SLAs over the years.

Non-websocket edge API traffic

Everything that isn't a websocket — Slack API calls, slackbot, webhooks, third-party apps — goes through the edge API tier, envoy-edge. The traffic arrives via various domains, each aimed at a specific purpose:

  • app.slack.com — API traffic bound for Webapp
  • edgeapi.slack.com — user and channel information
  • files.slack.com — file uploads and downloads
  • slack-imgs.com — image unfurling
1) Edge API requests resolved by Route53 are sent to “NLBs” then “Envoy-edge” service in Edge PoP. 2) Depending on the request type, traffic is either forwarded to the "Supra/Miata", "Imgproxy", or "Gatewayserver" service in the edge PoP, or "Envoy-www" service in the us-east-1 region. 3) us-east-1 “Envoy-www” service forwards requests to WebApp.

Route53 or NS1 resolves those domains and points the request to the closest NLB. From there, envoy-edge routes to internal services:

  • Flannel — an edge cache used by first-party clients for critical user and channel data.
  • Imgproxy — proxies image content.
  • Supra — handles file downloads.
  • Miata — handles file uploads.
  • Envoy-www — an internal load-balancing tier for API traffic going to Webapp.

Regional failover for edge API traffic

When a service or an entire PoP degrades, an internal tool called Edge Pop Drain detects the problem and automatically removes the affected service(s) — or the region — from DNS. New traffic then routes to the next closest available PoP.

1) wss-primary.slack.com user requests in eu-west-1 is routed by NS1 to eu-west-1 NLBs and then to “Envoy-wss” in the same region. 2) Envoy-wss has a fire icon indicating service down. 3) In us-east-1, there is “Edge-pop-drain” service which receives a trigger from the “Automation” that "Envoy-wss" is not healthy in eu-west-1". Edge-pop-drain then points to NS1 with a text "Toggles eu-west-1 envoy-wss NS1 record". 4) Users in eu-west-1 have dotted lines to eu-west-2 NLBs indicating "NS1 resolves wss-primary.slack.com to the next closest edge PoPs".

The tool restores normal routing once the region is deemed healthy. It also exists as a command-line utility so operators can manually shift traffic from one region to another. Slack credits this tool with shortening regional and service failover times during incidents.

Static content and the CDN

Slack uses AWS CloudFront as its primary CDN, which operates over 400 PoPs across 90 cities in 47 countries. Static assets needed to boot the Slack client are served on the a.slack-edge.com domain, with a backup on b.slack-edge.com. The client has built-in failover logic: it tries the backup when the primary CDN doesn't respond, moving back and forth with backoff and jitter. Beyond app boot assets, CloudFront also serves images and files from edge locations so those resources stay close to the end user.