Pushy to the Limit: Evolving Netflix’s WebSocket proxy for the future

From best-effort delivery to critical infrastructure

Pushy started as Netflix's WebSocket server for pushing data to devices without requiring constant client polling. It originally supported voice control scenarios — for example, receiving a command from Alexa and delivering the "play this title" message to a Fire TV — and RENO, the Rapid Event Notification System, which replaced frequent UI polling for row updates with server-pushed messages. That initial design served a candidate pool of roughly 30 million devices across Fire TVs, PS4s, Samsung TVs, and LG TVs.

Today Pushy connects to nearly a billion candidate devices, including mobile apps and the website experience. Even older devices that lack TLS/HTTPS support are covered through an encryption/decryption layer on the client and server, keeping messages confidential over insecure transports.

Growth-driven redesign

Over the last five years, Pushy has grown from tens of millions to hundreds of millions of concurrent connections, and it regularly handles 300,000 messages per second. That growth forced a revisit of older design decisions — not just to keep up, but to build a stable foundation for new features that will depend on Pushy. The service is currently delivering messages with a 99.999% reliability rate.

The message processor rewrite

The previous message processor was a Mantis stream-processing job that pulled messages from the queue. It was efficient, but scaling and deploying it required manual work — it had a fixed job size and no built-in path for canary rollouts or automated red/black deploys. As message volumes grew and the team made more changes to the processor, those manual steps became a burden.

The team rewrote the message processor as a standalone Spring Boot service using Netflix's paved-path components, which brought automatic horizontal scaling, canary configuration, and better observability. The rewrite was a judgment call, not the default option, but the component was small enough with clear goals and success criteria to make the risk manageable. Since the rewrite landed in mid-2023, the message processor has run hands-off with zero manual touch.

Moving the Push Registry off Dynomite

Pushy's Push Registry has long used Dynomite — Netflix's open-source wrapper around Redis — for storing device connection metadata. Dynomite delivered the low latency and easy record expiry that Pushy's workload demands, but it needed manual scaling as the system grew. That ceaseless intervention prompted a move to KeyValue, the Cloud Data Engineering team's "HashMap as a service" offering. KeyValue abstracts over the storage engine, letting the team pick whatever best meets their SLOs — in this case, low latency for fast message delivery. After migration and some tuning for Pushy's access patterns, the registry has scaled itself on autopilot.

Rethinking node capacity

Netflix's edge services like apiproxy are CPU-bound and scale on CPU usage. Pushy is different: connections sit mostly idle, waiting for an occasional message, so CPU stays low. Pushy autoscales on connection count with exponential scaling that adds more instances as the average connection count moves further past the target. A reconnect protocol, where devices reconnect every 30 minutes or so, provides a steady stream of connections to balance across instances.

The old policy kicked in at an average of 60,000 connections per instance, which for hundreds of millions of devices meant thousands of Pushy instances and pressure on NLB connection limits. The team's solution was to raise connection capacity per node through a careful evaluation of instance types and performance tuning — considering CPU, memory, cost, and the risk of a thundering herd when a node goes down and all its devices try to reconnect at once. They settled on an average of 200,000 connections per node, with room to scale up to 400,000 if needed.

Reliability hardening

Recent feature development surfaced connectivity edge cases between Pushy and devices. Silent failures — where a connection breaks without Pushy knowing, and Pushy tries to send on it — were a recurring problem, especially with older devices. Client teams improved their connection handling and reconnect flows. Pushy added heartbeat monitoring, idle connection cleanup, and better connection tracking to reduce the number of stale connections kept around.

From best-effort to direct delivery

Pushy's original role was straightforward: backend services dropped messages onto an asynchronous queue, and Pushy picked them up and delivered them to connected devices. That indirect path still exists, but the team has built out a direct push alternative that skips the queue entirely. When a service calls the Push library directly, it looks up the Pushy instance connected to the target device in the Push Registry, sends the message straight to that instance, and receives a status code back indicating success or failure.

This synchronous feedback loop is a significant upgrade over best-effort delivery. Services can now retry immediately if a targeted device has gone offline. The direct path was added years ago by Susheel, Pushy's original author, but only recently has adoption taken off. Today, direct messages make up the majority of Pushy's traffic — over a recent 24-hour period, direct push averaged roughly 160,000 messages per second versus about 50,000 per second on the indirect path.

Devices talking to devices

With bidirectional WebSocket connections already in place, the team extended Pushy's concept of a sender. Instead of only server-to-device delivery, Pushy now supports device to device messaging. This capability powered early phone-to-TV communication for games like Triviaverse, and it's the messaging foundation for Companion Mode, where TVs and phones exchange messages back and forth.

Device to device messaging requires broader awareness than a single connection. The system needs to know which devices are connected for a given account. Pushy emits device connection events over a Kafka topic, and a separate service consumes those events to maintain a per-account device list. Devices can subscribe to these events, so they learn when another device on the same account comes online and becomes available for pairing or messaging.

Once a device has that list, the flow works like this:

1. The source device sends a device to device message over its WebSocket connection to Pushy.
2. Pushy looks up the target device's metadata in the Push Registry.
3. Pushy forwards the message to the second Pushy instance that the target device is connected to.
4. That Pushy delivers the message to the target device.
5. The original Pushy receives a status code and passes it back to the source device.

The team defined a basic JSON-based message protocol for this, deliberately keeping it generic. Device teams build their own application-specific protocols on top of it, with Pushy acting purely as a transport layer. That abstraction has paid off operationally: the core functionality was built in October 2022 and has required only minor tweaks since, even as client teams built features on top of it without needing infrastructure involvement. Pushy currently sees an average of 1,000 device to device messages per second.

Under the hood

Incoming WebSocket messages are handled by PushClientProtocolHandler, which extends Netty's ChannelInboundHandlerAdapter and sits in the Netty pipeline for each client connection. Its channelRead method parses incoming messages; if a message is a device to device message, it's passed along with the ChannelHandlerContext and PushUserAuth connection identity data to the DeviceToDeviceManager.

The DeviceToDeviceManager validates the message, performs bookkeeping, and kicks off an asynchronous call that checks whether the device is an authorized target, looks up the target's Pushy instance in a local cache (falling back to a data store call on a miss), and forwards the message. The async design avoids blocking the event loop. The manager also handles observability, tracking metrics for cache hits, data store calls, delivery rates, and latency percentiles.

Security was a primary design consideration. All connections run over HTTPS, and device to device messages are restricted to authenticated WebSocket connections. The team added rate limiting and authorization checks to prevent any device from sending arbitrary data to another device it shouldn't be able to reach.

Latency optimizations

Interactive features within the Netflix app make latency a critical concern. Pushy now caches information that changes infrequently, such as a device's allowed list of targets and the Pushy instance a target device is connected to. Some lookups are still required for the initial messages to establish routing, but subsequent messages avoid KeyValue lookups entirely. With caching in place, median latency from incoming message to response is under one millisecond, with the 99th percentile below 4ms. Even during brief periods of elevated KeyValue read latencies, device to device latency showed little increase while other parts of Pushy, like client registration, were affected.

The approach that keeps it moving

Beyond technical design, the team credits iterative development and a focus on solving the hardest problems first. Projects frequently begin as quick prototypes to prove out a feature, with an eye toward generalization from the start. The early Triviaverse work solved hard cross-device problems that later became the generic device to device solution.

Pushy development also depends heavily on cross-team collaboration, often involving at least half a dozen teams per project. Building working prototypes and sharing them with partner teams has proven more effective than describing capabilities — seeing a TV respond to a phone controller's first click seeds ideas that abstract discussions cannot.

What's next

Pushy continues to evolve. The team has new features in the pipeline, including WebSocket message proxying and tracing, a global broadcast mechanism, and subscription functionality to support Games and Live. With the foundation now stable and the connection infrastructure proven, Pushy is positioned to support the next generation of Netflix interactive and companion experiences.