When a monorepo hurts the daily loop

Dropbox’s server monorepo is the central path for nearly every product change. Building AI features often means touching ranking systems, retrieval pipelines, evaluation logic, and UI surfaces, all of which flow through the same cycle of pulling code, building, testing, merging, and shipping. Recently, that loop started slowing down. The repository had reached 87GB, a fresh clone took over an hour, and many CI jobs were routinely paying that cost from scratch. Growth was steady at 20 to 60MB per day with occasional spikes above 150MB, putting the repo on track to hit GitHub’s 100GB hard limit within months.

The growth rate didn’t match what normal development activity at that scale would produce. That discrepancy pointed to a storage problem rather than a data volume problem.

Delta compression that backfired

The usual suspects—large binaries, committed dependencies, generated files—didn’t explain the pattern. Git’s delta compression did. Git stores file versions as deltas between similar files, relying on a heuristic that pairs files based on the last 16 characters of their paths. That works well for most codebases, but not for Dropbox’s i18n files:

i18n/metaserver/[language]/LC_MESSAGES/[filename].po

The language code sits early in the path, so Git often compared files across different languages instead of within the same one. A routine translation update produced a disproportionately large delta against an unrelated file. The content was normal; the interaction between the directory structure and Git’s compression heuristic was not.

Local proof, production constraints

Experiments with Git’s experimental --path-walk flag—which walks the full directory structure to pick delta candidates—confirmed the diagnosis. A local repack dropped the repository from the low-80GB range to the low-20GB range. But GitHub couldn’t support that flag because it conflicted with server-side optimizations like bitmaps and delta islands.

Local repacks alone were never going to be enough. GitHub constructs transfer packs dynamically on the server based on what each client is missing, so even a perfectly optimized local mirror gets rebuilt with GitHub’s own configuration. The repack had to run on their infrastructure.

$ git clone --mirror [email protected]:dropbox-internal/server.git server_mirror
performance: 2795.152366000 s

$ du -sh server_mirror
84G     server_mirror

$ git repack -adf --depth=250 --window=250
performance: 31205.079533000 s (~9h)

$ du -sh server_mirror
20G     server_mirror

Working with GitHub Support, Dropbox tested a more aggressive repack using tuned window and depth parameters—settings that control how thoroughly Git searches for similar objects and how many delta layers it allows. On a mirrored clone, the repack took about nine hours and shrank the repository from 84GB to 20GB. Because this approach aligned with GitHub’s server-side optimizations, it could run safely in production.

Repack as a production change

Repacking changes how billions of objects are physically organized on disk. It doesn’t touch code content, but it alters the structure underlying every clone, fetch, and push. Dropbox treated it like any other infrastructure change.

First, GitHub performed the repack on a test mirror. The mirror dropped from 78GB to 18GB. There was minor movement at the tail of fetch latency, but no stability issues, and that tradeoff was acceptable for a fourfold size reduction.

The production rollout then proceeded gradually over a week, updating one replica per day starting with read-write replicas and leaving buffer time at the end for a rollback if needed. The results held: the repository went from 87GB to 20GB. Clone times fell from over an hour to under 15 minutes in many cases. New engineers stopped starting onboarding with a long wait. CI pipelines began faster and ran more reliably, and internal synchronization services became less prone to timeouts.

Throughout the rollout, fetch duration, push success rates, and API latency all stayed within expected ranges.

Project data size dropped significantly and has remained stable since.

Lessons for large-scale repo management

This project surfaced three broader takeaways about maintaining infrastructure at this scale.

Growth is not just about commit volume. The rapid expansion had nothing to do with the content of commits—no large files, no accidental dependencies. It was structural. The i18n path layout encouraged Git to compute deltas across languages, so routine translation updates created oversized pack files. Tools embed assumptions, and when usage patterns diverge from those assumptions, performance degrades quietly. Understanding Git’s 16-character path heuristic was what made the correct diagnosis possible.

Some fixes require platform collaboration. Dropbox identified the root cause and validated a fix locally, but because GitHub controls packing and transfer, a local repack wasn’t sufficient. The solution had to align with GitHub’s supported server-side configuration. Bringing clear data to the provider, testing collaboratively, and working within their parameters were essential. When a system depends on a managed platform, some problems live at the boundary between your code and theirs.

Repo health is production infrastructure.

A repository repack changes the physical structure underlying every interaction, even though the code itself doesn’t change. Dropbox approached it like any major change: test on a mirror, measure real-world impact, roll out gradually, maintain a rollback path. As part of this effort, they also built a recurring stats job that tracks key health indicators for the monorepo into an internal dashboard: overall size, growth rate, fresh clone time, and how storage is distributed across the codebase. That visibility means acceleration or creeping clone times will be caught early, rather than when engineers start feeling the pain. Monitoring growth trends and investigating anomalies early is part of running a healthy engineering organization.

What the compression work changed

Shrinking the monorepo from 87GB to 20GB produced immediate, measurable gains across the development workflow. New engineers now get a working clone in minutes rather than waiting through a long initial checkout. CI pipelines spin up faster and are more reliable. Teams shipping AI features, which depend on many small iterative changes across services, feel the improvement in every cycle.

The investigation also prompted process changes aimed at preventing regression. The i18n workflow was updated to align better with how Git’s packing algorithm groups files, which lowers the risk of future pathological delta pairing. On top of that, the team now tracks repository growth trends more closely and has a concrete sense of baseline health.

A repeatable playbook

The project produced a template for tackling similar issues elsewhere. When growth accelerates unexpectedly, the team knows how to inspect the compression layer, validate fixes without risking regressions, and coordinate across platform boundaries when needed. Monorepos will keep growing as products evolve, but growth does not have to translate into developer friction. With the right tooling and monitoring discipline, repository size can remain a non-issue for the engineers who depend on it.

Acknowledgments: Samm Desmond, Genghis Chau