Using Server Sent Events to Simplify Real-time Streaming at Scale - Shopify

Making Real-Time Streaming Practical with SSE

Real-time updates have become table stakes for modern web applications, but the infrastructure choices behind them remain surprisingly old-school. While WebSockets and gRPC streams dominate conversations around bidirectional communication, Server-Sent Events (SSE) offer a simpler path for the vast majority of use cases where data flows in one direction: from server to client. At Shopify, engineering teams have adopted SSE as a primary mechanism for pushing updates at scale, and the results highlight where this technology shines—and where it demands careful design.

The Default Is Not Always the Best

Many teams reflexively reach for WebSockets when they need real-time behavior. The protocol supports full-duplex communication, making it ideal for chat, collaborative editing, and gaming. However, a significant portion of "real-time" features—order status updates, inventory changes, notification feeds—are strictly server-to-client pushes. For these, WebSockets add complexity without proportional benefit.

SSE operates over standard HTTP, which brings several practical advantages. It works seamlessly with existing load balancers, proxies, and authentication middleware. It automatically reconnects with built-in event IDs and retry logic, eliminating the need to hand-code heartbeat and resumption routines. The protocol also supports named events and fields, allowing the server to send structured updates without inventing a custom message format.

A Practical Implementation Pattern

Shopify's approach to SSE centers on a simple but powerful abstraction: a single long-lived HTTP response that streams events as they occur. The pattern begins with a client subscribing to a URL like /events. The server holds the connection open, sending data in the text/event-stream format. Each event is a block of lines starting with data:, optionally preceded by event: and id: fields, terminated by a blank line.

To avoid the common pitfall of unbounded buffer growth, the implementation writes events promptly and flushes periodically. Proper use of gzip compression becomes critical here—without it, response sizes balloon and latency spikes appear. These details matter at scale, as any single stream can serve thousands of concurrent connections.

A notable design choice in Shopify's setup is the use of message brokers as intermediaries. Rather than having the application server generate every event directly, producers write to a broker, and SSE relays consume those messages and forward them over open connections. This decoupling allows horizontal scaling of both producers and consumers independently.

Scaling Connections with a Relay Layer

The crux of production-grade SSE is connection management. Holding thousands of idle HTTP connections on a single machine eventually exhausts file descriptors and memory. Shopify's architecture addresses this by separating the relay (the component maintaining network connections) from the business logic that produces values.

Relays subscribe to a topic on the broker, receiving batched updates. They then fan out these messages to all connected clients, applying per-connection filters based on subscription metadata. This design keeps relays stateless in terms of business data, letting them scale horizontally behind the load balancer. A client that loses its connection simply points to a different relay on reconnection.

Message serialization receives equal scrutiny. Rather than sending JSON back and forth at high frequency, Shopify encodes events in a compact format on the wire. This reduces both parsing overhead and network payload, which directly impacts the number of messages a relay can route per second. For example, an update payload that might textually be 200 bytes can often drop below 50 bytes after removing redundant field names and whitespace.

Handling Backpressure and Resumes

One of SSE's hidden strengths is its lifecycle management. The Last-Event-ID header lets a client inform the server of the last event it processed, enabling precise resume after a disconnect. Shopify takes advantage of this by persisting a small window of events in a short-term store. When a relay detects an out-of-sync client (one whose Last-Event-ID is older than the newest persisted entry), it closes that connection immediately.

The client then reconnects to a different relay, which can fire a replay request to a separate broker. Using the last event ID as a logical offset, the system retrieves all missing messages from a durable log and forwards them. The broker in this scenario acts as a distributed redo log, not a message queue—it does not delete entries after delivery; it retains them for the replay horizon. This contrasts with typical queue semantics and avoids the "at-most-once" guarantees that complicate state recovery.

Flow control matters equally on the inbound side. If a producer generates more events in a short window than a relay can flush to slow clients, the system uses batching—coalescing consecutive events targeting the same connection into a single flush. On the client library side, Shopify has moved from django-sse to a custom Go relay, given Python's limitations in managing thousands of concurrent socket writes without excessive memory overhead.

When SSE Is Not the Answer

SSE is not a cure-all. The protocol's spec does not support multiplexing—a single connection can carry only one stream of events. For applications that need distinct logical channels, a client must open multiple connections, increasing overhead. Similarly, loading an initial snapshot via a single SSE connection is impractical when the snapshot is large relative to typical event sizes. Shopify recommends pairing SSE with an explicit "sync" call: fetch the full state using a standard request-response, then open the SSE stream for subsequent deltas.

Authentication also needs an explicit design. Since SSE is plain HTTP, credentials must appear in the URL, header, or cookie. Cookie-based auth works but requires cross-CORS care if the event endpoint lives on a different origin. In a microservices context, the relay may need to validate tokens independently of the producer's session, which Swift's URLSession delegation often obscures (a gotcha encountered when integrating on Apple platforms). The recommended interception point is a networking layer that rewrites headers prior to connection establishment.

Pro tip: use an absolute URL for the SSE endpoint to revert to the global session whenever cookies are involved, preventing leaked authorization across contexts.