From US-Only to Global Edge: The Dropbox Network Buildout
When Dropbox moved the bulk of its storage and serving onto custom-built infrastructure with Magic Pocket in 2016, the work was only half done. With roughly 75 percent of users located outside the United States, the company needed a network capable of delivering the same performance in Singapore as in San Francisco. That meant expanding from a US-centric footprint to a presence spanning 14 cities across seven countries and three continents, adding hundreds of gigabits of transit capacity, and building a custom edge-proxy architecture to handle user connections closer to home.
The Starting Point: 2014
In 2014, Dropbox’s entire network lived in the US: two data center regions (one per coast) and five points of presence (PoPs) in high-traffic cities. Every global user was served from the US, and the network leaned heavily on transit providers because of the limited peering footprint—roughly 100 peered networks. Traffic was unevenly balanced across PoPs, capacity sat in the hundreds of gigabits, and shifting traffic flows between locations was difficult. The model worked for a US user base but left international users at the mercy of often higher-latency paths.
Routing Architecture: 2014
Dropbox’s routing plane in 2014 was built on BGP at the edge, connecting to transit and fabric peers, with OSPF handling NLRI resolution inside the network. Extended BGP communities tagged prefixes, and standard BGP path attributes—such as AS-path, local preference, and MED—governed egress path selection when multiple routes existed.
2015: Foundation Work
The first major change came in early 2015 with a routing architecture overhaul. OSPF was replaced by IS-IS, BGP communities were revised, and MPLS-TE was introduced to balance traffic dynamically across the backbone, avoiding over- and under-utilized links. By mid-year, attention shifted to user experience: how to cut round-trip time and get traffic off the US-only paths.
Growth outside the US drove the next decisions. Dropbox targeted Europe, choosing three PoP locations that connected it to major peering exchanges. By the end of 2015, new PoPs were live in Palo Alto, Dallas, and New York, alongside hundreds of gigabits of new transit capacity and a meaningfully larger peering footprint. Transit still carried most traffic, but the geographically expanded peering edge made broader traffic engineering possible—and laid groundwork for the edge proxy design to come.
2016: Expansion and Optimization
Entering 2016, Dropbox pursued three priorities:
- Deploying new PoPs in Europe and Asia to get closer to users and improve sync performance
- Building custom edge proxy, routing, and IP transit architectures
- Growing peering relationships to expand the network’s direct-connection footprint
Traffic analysis pointed to London, Frankfurt, and Amsterdam as the European locations offering the greatest access to eyeball networks. Those PoPs were deployed in 2016, connected in a ring topology via the backbone, and backhauled to the US through New York and Ashburn. Asia followed in Q3, with PoPs in Tokyo, Singapore, and Hong Kong, selected to serve both local users and regional networks across the Asia-Pacific area.
Edge Proxy Stack
The new PoPs needed a way to handle user connections locally. The edge proxy stack—composed of IPVS and NGINX machines—handles SSL termination and maintains connectivity to backend servers across the Dropbox network, removing the need to send every TLS handshake to US data centers.
Routing Redesign
With proxies in place, routing had to change. The original policy of advertising all public prefixes from every PoP made sense with centralized front-ends, but it created asymmetric and sub-optimal routing once edge capacity grew. The redesign introduced the concept of a "metro," grouping regions into individual failure domains with these principles:
- Users near a metro are routed to the closest PoP, cutting latency
- Metros can have multiple PoPs for redundancy
- Each metro is its own failure domain—traffic can be withdrawn by pulling metro-specific prefixes
- Prefixes stay contained within their region/metro and only advertise outward from that location
- Intra- vs. inter-routing decisions are clearer by design
BGP communities were extended to support the metro model. Prefixes carry an Origin tag, internal and external routes get assigned a Route-Type, and each route has a Route-Scope—metro, region, continent, or global. A set of Actions applies internal router semantics, while a Tag carries additional community information for special handling.
Transit Architecture
Before mid-2016, transit capacity was uneven: each metro had different port sizes, making it difficult to drain a metro when needed. Dropbox standardized transit capacity across all PoPs to guarantee sufficient headroom when moving traffic during outages or disaster recovery testing. Ingress traffic arriving from transit providers was also imbalanced, so the company collaborated with tier-1 providers to balance incoming traffic across its ASN. Edge routing policies now prefer the shortest AS-path for egress; when multiple tier-1 paths tie, MED breaks the deadlock.
Peering Growth
Until Q1 2016, a relatively small peering footprint meant most traffic egressed via transit. Dropbox identified top ASNs behind each metro’s transit providers, analyzed netflow data, and initiated peering discussions with the highest-value eyeball networks. By the end of 2016, closed to 30 percent of traffic had moved to peering.
Results and Current State
The shift shows up in measurable improvements. For European users, median download speeds rose roughly 40 percent after edge proxy deployment, while median upload speeds increased approximately 90 percent. In Japan, median download speeds doubled; uploads tripled. The following charts show TCP/SSL connect times dropping across Europe and Asia-Pacific after edge stacks went live in every PoP, along with a customer latency trace reflecting the same trend.
By the end of 2016, Dropbox had added six PoPs across Europe and Asia, bringing its total to 14 and pushing edge capacity into the terabit range. Transit, fabric, and private peering grew by hundreds of gigabits, and the network picked up more than 200 new ASNs through peering. The majority of Dropbox’s internet traffic—including the final graph and performance data above—now moves between the user’s best/closest PoP and the network directly over peering connections, which is a substantial shift from the transit-reliant, US-only architecture of two years prior. However, those gains are regional: performance improvements were tested and validated in European and Asian markets, while other regions saw less immediate benefit. Update (November 14, 2017): Miami, Sydney, Paris, Milan, and Madrid have since been added to the Dropbox Edge Network.
Scaling the Edge: How Dropbox’s Network Evolved
Dropbox has long operated a globally distributed network that handles everything from user uploads and syncs to real-time collaboration traffic. Over time, the demands on that edge network have shifted dramatically. This piece looks at the engineering decisions the company has made to keep the network reliable, performant, and future-proof.
Initial Architecture: Simple Centralization
In its early days, Dropbox relied on a small set of colocation facilities with high-bandwidth connections to major cloud providers. This kept the architecture simple. When the team did deploy in regions closer to users, it was largely to optimize uploads from specific networks. The expectation was that most traffic was asymmetric: the vast majority of data flow was downloads, which could be served well from a handful of locations with strong peering.
The rise of file collaboration changed that assumption. The product’s growing desktop and mobile sync features created a more balanced flow of data. Downloads used to dwarf uploads by a significant margin; now the upload path must handle large amounts of data just as efficiently. As the platform expanded, Dropbox hit a series of practical constraints: colocation required manual hardware stacking, purchasing capacity for future peaks was expensive, and expanding globally was a slow process. A more dynamic model was needed.
Building a Purpose-Built Network
Today, Dropbox’s network is layered across a set of distinct components, each solving a specific part of the traffic equation.
- Last mile access: The physical connection between users and the network, handled by the local Internet service providers and mobile carriers.
- Points of presence (PoPs): Edge locations that terminate user TLS connections, host the proxy stack, and connect to the rest of the network. All routing, protocol termination, and traffic management logic happens here.
- Backbone: A dedicated, CLOS-based fabric connecting PoPs with data centers. It manages the aggregated traffic between edge nodes and storage, carrying data either over leased dark fiber or through transit providers.
This functional separation allows Dropbox’s infrastructure teams to treat each component with specialized tooling. Most recently, the company has focused on the detailed design of network links and the bandwidth-delay product, the physics that determines how large the data window must be to keep a high-speed link saturated.
Tuning Throughput at the Link Level
Commodity network interfaces have consistently pushed throughput upward: from 10G, to 25G, and now 100G per link. But a faster link is not just a bigger pipe—it interacts with protocol behavior in surprising ways. An underutilized high-latency path, for example, stops behaving like a link and starts behaving like a long thin pipe: the send window must be measured in the context of the high round-trip time. To achieve full throughput on a 100G link, the network must either maintain deep buffering to absorb the large amount of data in flight, or implement flow control mechanisms that compensate for the lack of buffering at the switch level.
The current generation of ASICs has essentially eliminated traditional dynamic buffering. In their place, Dropbox relies on global congestion control systems that manage flow rates across the network as a single coordinated unit, rather than per-switch buffers. This shift changes the tuning problem completely; it is no longer enough to configure per-switch drop parameters. The entire system has to be designed as one integrated pipeline.
Simulating Protocol Behavior for Real-World Gains
To address that, the network infrastructure group (the net-edge-oriented team) has put significant effort into using simulation tools to model lossy, long-distance links. One approach they pursued was the pantheon and similar harnesses—components that allow engineers to test new congestion control algorithms before ever shipping them.
Simulation gave the teams a crucial tool to validate a layer’s behavior before production. By modeling the target network paths in software they could determine whether a change to the Linux TCP stack or the proxy configuration would have a measurable effect on throughput, or whether the bottleneck existed elsewhere, such as in the receiving application’s buffer. This helps avoid chasing performance dead ends, and instead isolates whether a given problem is a tuning problem or a design flaw that warrants deeper architectural changes.
Meeting New Expectations at the Application Layer
Beyond the transport path, Dropbox’s proxy stack has had to keep up with new behaviors in the application syntax. Even minor enhancements to the protocol supported by proxies—such as extensions for long-lived streaming—required redirecting how the proxy layer interprets connection lifetimes and application-agnostic features. For less standard proxy states, engineering teams remove generic fallbacks in favor of explicit per-message routing that maps directly to the backend implementations.
Dropbox is also increasingly involving QUIC-based transports at the edge to reduce connection setup latency over high-packet-loss networks, shifting more traffic onto UDP instead of the ever-crowded TCP streams. They underline that work by tracking the path through which a user connects (last mile → PoP → backbone) and test extensions that allow proxies to enqueue or prioritize segments of transfers depending on the identified flow type.
Future Directions
The architecture canvas is far from static. The next milestones for the network involve looking deeper into the dynamics of the bandwidth-delay product, proving that a longer thin-pipe path—like a transcontinental link with high RTT—can behave as well as a short and fat one. To do that, Dropbox plans to lean more on closed-loop control solutions: collections of network signals that interact with each other to create network-wide data plane decisions.
This works with a strategy of embedding a dedicated probe element into the data path, providing the network layer with richer telemetry that normal packet processors cannot deliver alone. The goal is to shift from the classic open-loop approach—configure a static window and let TCP react—to a closed-loop model that reacts to live, global network states.
Hiring for the Next Phase
All of these efforts require specialized skill sets. Dropbox says it is hiring experienced network engineers to scale the edge network past its current multi-terabit traffic level and 100G uplinks. Roles are also open for engineers to work on the CLOS-based fabrics of the backbone, plus a hybrid security/networking position for a Network Security Engineer in San Francisco. For those focused on higher-layer systems, the traffic team is hiring software engineers and SREs to work on TCP/IP packet processors and load balancers, HTTP/2 proxies, and the internal gRPC-based service mesh. Engineering roles are open across San Francisco, New York, Seattle, Tel Aviv, and other global offices.
