Meet Bandaid, the Dropbox service proxy

Why Dropbox built Bandaid

Bandaid started life as a reverse proxy to compensate for inefficiencies in Dropbox's server-side services, then grew into a service proxy that accelerated adoption of the company's Service Oriented Architecture. A reverse proxy forwards requests from multiple clients to backend servers, commonly providing load balancing for web applications. Other typical uses include web acceleration, SSL termination, and security features.

Companies running private clouds with significant traffic volumes often build their own reverse proxies rather than using off-the-shelf solutions. The reasons include better integration with internal infrastructure, reuse of well-known internal libraries, reduced dependency on external projects, and the ability to address specific company use-cases.

Bandaid supports a rich set of load balancing methods (round-robin, least N random choices, absolute least connection, pinning peer), SSL termination, HTTP/2 for downstream and upstream connections, metro rerouting, buffering of requests and responses, logical isolation of endpoints, dynamic reconfiguration without restart, service discovery, per-route stats, gRPC proxying, HTTP/gRPC health checking, and weighted traffic management for canary testing.

Internal design

Bandaid is written in Go, which allowed tight integration with services and a shortened development cycle. Its primary components are the Request Handler, which sends requests to a Queue; queues that pop requests to Workers (goroutines); and Workers that process requests and send them one-by-one to sets of hosts called Upstreams.

Queueing

Request queueing inside the proxy enables better management of overloaded backends. Bandaid always processes requests in Last In, First Out (LIFO) order. When the system is not overloaded, the queue is empty or nearly so, making the pop order irrelevant. Under overload, LIFO processing reduces overhead by handling the newest requests first—older requests are likely to time out soon, so spending CPU cycles on them is wasteful.

Bandaid queueing can optionally drop requests once the queue reaches a configurable maximum capacity. This is not recommended, however, since it's hard to tell whether a full queue results from overload or bursty traffic. Two additional options control how many requests from the same queue can be processed concurrently and how quickly requests leave the queue.

Because Bandaid always accepts TCP connections and pushes read requests into its own user-space queues, the kernel TCP accept queue is always empty. This design choice matters: clients may close a connection unexpectedly while the backend is still processing data, wasting resources. Keeping the kernel accept queue empty lets Bandaid track connection timing in its own queues, detect and propagate connection closure sooner, and free backend resources. It simply fails requests that sit in the queue beyond a configurable timeout, and instead forwards newer requests that have a lower probability of being closed by the client.

Managing connections in the user-space queue gives more control over both the queue and requests. Consider the problem scenario:

1. A client connects to the server. If the kernel accept queue isn't full, the three-way handshake succeeds.
2. The client sends requests, unaware of the server's state—it will send even if the server cannot accept or process them.
3. With a slow server, the client waits until its timeout, then closes the connection with FIN-ACK flags because no response arrived.
4. The server application eventually gets the delayed connection via accept(). It doesn't detect the client already closed the connection (a normal FIN, not an RST), and continues to process.
5. The server reads the request, processes it, tries to respond—and gets an RST after the first write() call because the connection is already closed. The next write raises SIGPIPE or returns a broken pipe error.

Aggressive client-side retries worsen the situation: the overloaded server must process extra requests, and closed connections fill the kernel accept queue. Possible solutions include exponential backoff between retries, decreasing the kernel accept queue size, or managing connection timeouts in the user-space queue. Controlling retry timing on the client side is often impractical (for example, with third-party applications accessing servers through an API).

Bandaid implements the third approach: it keeps the kernel queue empty with a user-space LIFO queue, and closes connections that wait longer than a specified period without processing them. New requests proceed normally.

Request handling

Bandaid supports multiple queues, so the Request Handler needs a way to decide where to push each request. Currently it distinguishes requests only by URL and hostname, matching them against a configurable list of patterns assigned to each queue.

Worker pool

Bandaid uses a fixed-size pool of workers to process requests, rather than running an unbounded number of goroutines. This makes upstream concurrency precisely controllable. The worker count is configurable but depends on the number of healthy upstream hosts; setting it much higher than that number causes oversubscription. Each worker loop pops a request from the queue and invokes the current request processor. The count must be tuned so there are enough workers to fully utilize upstream capacity—but not so many that overloaded services receive excess requests, which would increase failure rates or latency. Bandaid drops these extra requests to keep upstream load at an appropriate level.

Upstreams

Each upstream consists of queues that receive incoming requests, a single dequeuer that acts as a multiplexer, and a request processing work pool. Multiple upstreams can exist, and the dequeuer enables several use cases.

Key use cases

Weighted traffic management

This feature enables canary deployments: Bandaid can route a configurable percentage of traffic to a deployment running a particular software version (for example, 10% to a new deployment). Multiple queues may belong to a single upstream, and each queue can have its own properties—weight, queue size, rate limit, priority level, and concurrent connection count. Two additional interfaces built on top of queues enable weighted traffic management and prioritization.

The enqueuer interface decides where to push requests based on queue weights; higher-weighted queues are more likely to take new requests. This supports traffic shifting—for example, 90% to production and 10% to canary. Bandaid supports hot config-reloading, so a new configuration applies without restart, letting operators see results within seconds of pushing changes.

The dequeuer determines pop order based on queue priority: higher-priority queues are drained before lower-priority ones. Under overload, low-priority requests are therefore more likely to be slowed down. When multiple queues share the same priority, the dequeuer semi-randomly shuffles them and pops in the shuffled order to ensure fairness.

Strict priority-based dequeuing can cause starvation—lower-priority requests may never be served if higher-priority queues are always full. To avoid this, Bandaid provides an option controlling how fairly queues are popped, ranging from treating all queues as equal priority to always favoring high-priority requests.

Logical isolation within an upstream

Some backends serve both critical and non-critical routes from the same host. Degraded performance on non-critical routes can then affect critical routes, since each host has finite capacity. Serving these route types from separate hosts is one solution; isolating at the proxy level is another that reduces operational overhead and hardware instances. Bandaid lets you configure rate limiting, the number of concurrent connections, and queue priority to control how critical and non-critical routes behave.

The diagram shows two queues, but there is no such limit—an upstream may handle requests from any number of queues.

HTTP/gRPC reverse proxy

This is the classic load-balancing use case, with the load balancing methods described above.

Limiting concurrent connections

Backend servers may cap the number of supported concurrent connections. Bandaid can accept a large number of incoming connections and control how many (typically far fewer) it forwards to each backend process. It can also be configured to respond with a specific status code when the limit is reached.

The diagram shows how Bandaid reduces outgoing TCP connections by multiplexing. Each host runs multiple client instances, each establishing its own TCP connection to Bandaid. Bandaid then reuses inactive connections (via keep-alive and HTTP/2) to minimize concurrent connections with the backend service.

HTTP protocol transition

Some services still run HTTP 1.0. Bandaid can translate between the newest and oldest HTTP protocol versions in either direction.

How Bandaid Distributes Traffic

Bandaid's current version offers several load-balancing strategies, each suited to particular backend conditions. No single method works best universally, so the proxy lets operators choose based on their service's behavior.

Round Robin

The simplest option, round robin sends an equal number of connections to every host in the serving set. It ignores host performance, connection speed, or current load. When backends have uneven processing times, this can lead to a pile-up: slow hosts keep receiving new connections even while still busy with old ones.

The severity of the problem depends on how many slow hosts exist and how much slower they are. The probability that a worker gets stuck on slow backends is given by V = KR/(1 + K*(R-1)), where:

• C – number of backends in a bad state
• Lc – average latency across those C machines (time from accepting a connection to finishing processing)
• P – number of healthy backends
• T – total machines, T = C + P
• Lp – average latency across the P healthy machines
• K – ratio of bad hosts to total, K = C/T
• R – latency ratio, R = Lc/Lp

The curves in the figure plot V for varying proportions of slow backends at different R values. The blue line shows R=2 (bad hosts twice as slow), red R=5, and green R=50. The green line rises sharply: when slow hosts are 50 times slower than healthy ones, they can absorb roughly 70% of capacity even if they make up only 5% of the upstream.

Least Connections of N Random Choices

This method suits backends with uneven processing performance. Instead of treating every host equally, it sends fewer requests to slower machines. The algorithm picks N random hosts (here N=2) from the serving set and selects whichever has the fewest active connections.

One weakness: if a host is failing requests at a high rate, it may still be chosen when it happens to have the fewest connections, since the algorithm has no view of health or resource utilization. This risk is higher with smaller serving sets.

A related variant, Absolute Least Connections, locks the current connection state while Bandaid scans for the host with the fewest connections, preventing race conditions where counts change mid-search. Bandaid also randomizes the starting position of the scan, so hosts that share the same lowest connection count receive a more even distribution of new connections rather than always favoring the first one in the list.

Pinning Peer

Here each Bandaid worker (a goroutine) is permanently bound to one upstream host. A worker fetches the next queued request only after the host finishes the previous one. Because the worker is throttled by the backend's own speed, slower hosts naturally receive less work without any coordination logic.

Synthetic Test Results

Before putting a strategy into production, it helps to validate expected behavior in a controlled environment. Bandaid's load-balancing methods were tested under the following conditions:

1. 100,000 requests waiting in the queue
2. 100 workers in the workpool
3. 10 backend hosts in one upstream

Each backend was assigned a distinct processing latency, as shown below.

The resulting request distribution across backends is illustrated in the next chart. Round robin spreads requests almost uniformly, with no correlation to processing time. The other three methods — absolute least connections, least connections of N random choices, and pinning peer — all direct more traffic to faster backends. That better balancing lowers total processing time and raises the overall request rate across the upstream.

Retry Exclusion

When Bandaid retries a failed request, it should avoid resending to backends that were already tried. Otherwise, retries can keep hitting the same unhealthy hosts.

The chart above plots the failed-request ratio for each load-balancing method, both with and without retry exclusion. The test simulated 20% failing hosts (immediate error responses) and allowed up to four retry attempts. Enabling exclusion reduced errors for every method except pinning peer, which is expected: since a worker is bound to a single host, all its retries go to that same backend regardless of exclusion settings.