SMR at Petabyte Scale
Dropbox’s Magic Pocket storage platform has relied on custom infrastructure to balance efficiency, performance, and cost since its original deployment. The latest iteration of that platform moves to Host Managed SMR (Shingled Magnetic Recording) drives, which Dropbox is now deploying in volume across hundreds of petabytes of new capacity. The company is the first major adopter of the technology in production, and the deployment depends on significant software rework to overcome SMR’s inherent write constraints without compromising the latency requirements of live user traffic.
Why SMR
Conventional PMR (Perpendicular Magnetic Recording) drives support random writes across the full disk surface. SMR increases bit density by overlapping tracks, which forces writes to be sequential: writing to one track can erase the adjacent track. A small conventional area at the outside diameter of the drive buffers random writes, and an SSD cache can supplement this area in the host.
The density advantage is the primary attraction. SMR currently provides the highest per-drive capacity available, ahead of PMR, with a lower cost per gigabyte and a corresponding reduction in total cost of ownership. Dropbox reports that the new chassis store 2.29 times more blocks than the previous architecture, with each machine now holding roughly 1.4PB of raw storage.
Choosing Host Managed Drives
Three SMR drive types exist: Drive/Device Managed, Host Aware, and Host Managed. Dropbox evaluated the latter two and selected Host Managed drives.
Device Managed drives present the same interface as conventional drives. The drive buffers non-sequential writes to its small conventional area and performs its own internal rewrite operations, reading from sequential zones and writing merged data back. Host Aware drives expose zone control to the host, allowing it to open and close zones and monitor write pointers, but leave rewriting behavior on the drive. Host Managed drives delegate all zone management to the host: no internal copying, no caching, and no conventional-area fallback. The host must explicitly open, fill, and close sequential zones.
Host Managed fit Dropbox’s existing storage design. Magic Pocket stores immutable blocks of up to 4MB within 1GB extents, written in append-only fashion. That workload is naturally sequential, and the prior software stack was already built to direct data layout at the block level.
OSD Software Architecture
Magic Pocket runs one Object Storage Device (OSD) daemon per disk per machine, and only that daemon accesses the disk. OSD treats the disk as a raw block device without a filesystem, managing layout and I/O scheduling directly in software. This lets the daemon optimize head movement and prioritize live user traffic over background operations. Disk I/O relies on Libzbc as the foundation for SMR communication.
The software rewrite separates data by write pattern: metadata, which is updated frequently, stays in the conventional area where random writes are supported, while the immutable block data lives on sequential zones. Two properties of the workload made this clean. The logical space divides evenly across sequential zones at 256MB each, four of which make precisely one 1GB extent. And metadata comprises only about 0.03% of the total data, fitting comfortably in the conventional area.
Large writes are far more efficient on SMR drives, averaging 4-5MB. The initial approach of flushing many small writes from the SSD to the SMR disks proved inefficient, so the design now batches writes to larger sizes and reduces flush frequency.
Caching and Write Staging
Removing the RAID controller eliminated its write cache, so Dropbox added an SSD per host to compensate. In earlier storage generations, data from the network was written directly to drives. That is impractical at SMR density because large sequential writes take time and would stall the network. The SSD accepts incoming data asynchronously and lazily flushes it to the SMR disks in the background.
The staging area also solves alignment. All SMR writes must be sequential and aligned to the 4k boundary, and live incoming data does not arrive in neat 4k chunks. The background flush process aligns and batches writes to disk.
The polling priority separates two classes of work. Live traffic, consisting of new incoming blocks and writes supporting user activity, is latency-sensitive and gets top priority. Background repairs require substantial read/write work but are not time critical and proceed more slowly.
Memory serves as an additional staging area for background operations, which matters because SSD write endurance is limited. The host memory was raised to 96GB per machine to hold an in-memory index of blocks and their disk offsets — a necessary increase given that each chassis stores 2.29 times more blocks than the previous generation. The original flush-before-batch strategy was replaced with a model that buffers more in memory.
Hardware Changes
The hardware redesign touched nearly every component of the storage server, driven by the goal of fitting roughly 100 LFF (Large Form Factor) disks in a single 4U chassis at a 42-inch depth limit and standard 19” rack width. This keeps the fleet within standard datacenter rack and flooring specifications; each rack holds eight fully configured chassis.
CPU: The chassis moved from 16 to 20 cores (40 threads) to keep total chassis I/O above 40Gbps write and 45Gbps read.
Storage controller: The system now uses a Host Bus Adapter (HBA) rather than a RAID controller. RAID previously mitigated write latency using cache but came with significant overhead: Dropbox created individual RAID 0 devices and managed associated firmware and bugs just to expose a block device. Removing that complexity cut provisioning time from up to two hours to roughly 30 minutes. Dropbox also enabled Direct I/O to reduce CPU usage from double-buffering.
Cache: The HBA has no cache, so the SSD fills that role. One limitation identified in testing is that as density increases, the SATA bus saturates; the SSDs are showing SATA as a visible bottleneck, and future generations are expected to use NVMe for caching.
Network: Prior chassis had around 240TB and networks scaled as density increased. At 1.4PB per host, faster network throughput is required to keep recovery time within SLA after a failure. The SMR chassis design includes a 50Gbps NIC per chassis, connected to a non-blocking Clos fabric with 100Gbps uplinks. This allows rapid data writes when a chassis deploys and quick draining during repairs.
Expander: An expander distributes the HBA connection evenly across all drives, but it was initially incompatible with SMR. Dropbox collaborated with the vendor to co-develop firmware that resolved the incompatibility. Tight physical space in the 100-drive chassis limits how many hardware variations can be tested when bottlenecks surface.
Durability and Availability Targets
Magic Pocket’s quarterly durability standard holds at over 99.9999999999% annual durability and over 99.99% availability. The SMR deployment matches that goal through the combination of the rewrite OSD software, the SSD caching earlier, and the higher network bandwidth for faster recovery.
SMR Code Changes
Dropbox also moved the OSD code from Go to Rust. The change allows direct control of memory allocation and garbage collection, which lets a single daemon handle more disks and larger disks without proportional increases in CPU and memory cost. The full details of the Rust optimization are covered in a QCon presentation on using Rust to optimize Dropbox storage.
Future Directions
The new storage architecture positions Dropbox to absorb subsequent generations of disk technology as they mature. The near-term roadmap concentrates on two areas: packing more capacity into the existing form factor and moving high-volume traffic more cost-effectively. With the number of drives per system already pressing against the physical envelope of the chassis, the engineering work shifts toward mitigating the failure modes that come with large amounts of data concentrated on a single system, while also squeezing more useful compute out of each node.
What This Deployment Unlocks
The SMR rollout is an incremental step in the ongoing evolution of Magic Pocket and the broader Dropbox infrastructure, rather than a one-time project. Introducing a new storage medium at this scale reliably required solving problems on two fronts at once: rethinking the physical layout and mechanical tolerances of the hardware itself, and shipping the substantial software changes needed to manage the new drive behavior. That pairing of hardware and software work depended on close coordination between the respective engineering groups.
The payoff is structural, not just operational. Higher per-drive density directly lowers the cost per stored byte, which feeds back into what Dropbox can offer its users. The infrastructure benefits from both a better cost structure and a smaller physical footprint, and resources freed up by that efficiency can be reinvested into product development.
Project contributors: Chris Dudte, Victor Li, Preslav Le, Jennifer Basalone, Alexander Sosa, Rajat Goel, Ashley Clark, James Turner, Vlad Seliverstov, Sujay Jayakar, Rami Aljamal
