Why graceful restarts matter at scale
Every piece of software eventually needs to be replaced by a newer version. The question is how that replacement happens without disrupting the people relying on it. For a proxy handling live HTTP traffic, an abrupt stop can mean broken page loads, dropped video calls, and frustrated users. The stakes are higher still at Cloudflare, where services must stay available around the clock.
One common strategy for zero-downtime upgrades is to start the new version, gradually shift traffic to it, and only terminate the old version once it has nothing left to do. For connection-oriented services, the safest moment to restart is when there are no active clients to interrupt — but that moment rarely exists in practice. Long-running connections for backups, real-time updates, and remote shells are always present somewhere. Even if there were a quiet window, a critical security fix cannot wait for it.
Sockets outlive the process
The Oxy proxy is written in Rust, so Cloudflare's Go-based restart library, tableflip, was not directly reusable. That library works by starting a new process that inherits listening sockets from the old one, then signals readiness once it is accepting connections. The Rust rewrite, named shellflip, handles the process spawning and signaling portion, but socket inheritance is solved differently — through systemd.
Many Cloudflare services configure their sockets through systemd socket units. This decouples socket lifetime from application lifetime. When systemd manages the socket, the listening endpoint remains open even if the Oxy process is stopped or restarted. Incoming connections queue at the socket level and are served as soon as the new process starts. If Oxy managed its own sockets, a restart would close them and clients would receive "connection refused" errors.
Waiting for tasks to finish
Graceful shutdown requires the old process to know when all in-flight work is complete. Go provides WaitGroup for this purpose, but the tokio async runtime used by Oxy has no direct equivalent. The implementation leverages tokio's multi-producer, single-consumer (MPSC) channels instead.
Each task holds a clone of the producer side of the channel. When a task completes, its producer instance is dropped — an automatic consequence of Rust's RAII rules. The consumer side receives a notification when all producer handles are closed, which signals that every task has finished. This avoids explicit bookkeeping and makes it harder to accidentally leave a task running.
Restarting with feedback
Traditional reload mechanisms use Unix signals, often SIGHUP, to tell a process to re-read its configuration. This approach has a significant flaw: sending the signal does not guarantee the reload succeeded. If the configuration file is malformed or references a missing resource, the program simply keeps its old state, and the only way to detect the problem is to dig through log output. For automated configuration management, this can produce a false sense of success.
Oxy replaces signals with a coordination socket. A separate restarter process validates the configuration, connects to the Unix socket, and sends a "restart requested" message. The running proxy instance then spawns a new process, which inherits a pipe for reporting its own startup result. The old instance waits for that report, sends a "restart response" back to the restarter, and the restarter surfaces the outcome through its exit code. This gives automated systems a reliable signal of whether the restart actually applied the new configuration.
Using a Unix socket for this coordination preserves the security properties of signals. Writing to the socket requires the appropriate file permissions, just as sending a signal requires the process to belong to the user or to root. This prevents other users on the same machine from triggering restarts.
Remaining limitations
Even with these mechanisms in place, the old process cannot run indefinitely after a restart. A hard upper bound on its lifetime is necessary to avoid accumulating zombie processes that consume memory. When that limit is reached, remaining connections are forcibly terminated.
Configuration changes are also constrained by the fork-and-inherit model. Resource limits may be adjustable at runtime, but adding new sockets still requires a full service restart. For UDP-based protocols such as HTTP/3, the situation is more complex: there is no single listener socket to inherit. Incoming packets are balanced across all unconnected UDP sockets for a given address, making it difficult for the old process to drain existing sessions without receiving traffic destined for the new one.
Carrying application state between processes could mitigate some of these issues, but it remains a hard problem — even languages with built-in hot code upgrade support still run old tasks under old code versions. Crashes from segfaults or the OOM killer are also a possibility, rare though they may be in Rust. The shellflip source is available in its GitHub repository, and it represents a first step toward more reliable proxy upgrades, with further improvements planned.
