GitHub’s own engineers now develop GitHub.com inside Codespaces
GitHub.com turned 14 this year, and its core repository, github/github, has accumulated more than a million commits. For most of that history, the developers committing those changes built and tested on macOS. That model has now been retired: the majority of GitHub.com development happens inside Codespaces, running on Linux hosts. The migration was not just a change of tooling—it forced GitHub to re-examine assumptions baked into the codebase over more than a decade.
Why local development broke down
GitHub had invested heavily in making local setup painless. Its scripts-to-rule-them-all convention let a new engineer clone github/github, run setup and bootstrap scripts, and have a working instance in about half a day. When something failed, an internal Slack channel, #friction, was staffed by engineers who could debug nearly any system configuration.
Even with that support, local environments were fragile. An innocuous change could break an environment and cost hours to recover. The breakage was common enough that the bootstrap script gained a --nuke-from-orbit option, which deleted as much as it responsibly could to restore a known good state. This is a familiar story across the industry: bespoke local environments are brittle and increasingly out of step with an instant-on, access-from-anywhere world.
Collaborating across branches and projects was painful. A branch that introduced new dependencies, shipped schema changes, or branched from a different SHA could trigger a 45-minute bootstrap. GitHub deploys hundreds of changes per day, so this friction was constant. In building Codespaces, GitHub also spoke with other engineering organizations that had built similar platforms; at significant scale, eliminating this productivity loss becomes a clear opportunity.
Treating development environments as infrastructure
In production infrastructure, servers are commodities. No server is unique or indispensable; any piece can be replaced without fanfare. Local development environments are the opposite: each is unique and requires constant maintenance. The next git pull or bootstrap can degrade the environment and force an expensive context shift from building software to recovering it.
Development environments are also deeply personal—engineers tweak and tune their workbenches for productivity and self-expression. With Codespaces, GitHub saw a way to treat environments as churnable commodities while preserving the ability to curate the workbench. Visual Studio Code extensions, settings sync, and dotfiles repos bring the environment to the compute. A broken workbench becomes a minor inconvenience: provision a new codespace at a known good state and get back to work.
Making the migration work
The first stages of the transition were challenging. github/github is nearly 13 GB on disk; cloning alone took 20 minutes. With dependency setup, bootstrapping a codespace took upwards of 45 minutes. And once the repository was mounted, the application would not run. Fourteen years of macOS-centric assumptions had to be undone.
Contributors from across the company worked at the source level to decouple GitHub development from macOS. Eventually, GitHub.com codespaces could be provisioned on Linux hosts, connected from Visual Studio Code, and used to ship work. The remaining problem was speed.
From 45 minutes to 5 minutes
The goal was task-based development: roughly a one-to-one mapping between branches and codespaces. Forty-five minutes was far too slow. The first optimization changed how Codespaces cloned the repository. Instead of a full clone at provision time, Codespaces performed a shallow clone and unshallowed history in the background after creation. That reduced clone time from 20 minutes to 90 seconds.
The next opportunity was caching the network of software and services supporting GitHub.com, including Gemfile-based dependencies, C and Go services, and a custom Ruby build. A GitHub Action now runs nightly to clone the repository, bootstrap dependencies, and build and push a Docker image. That image serves as the base for the devcontainer—config-as-code for Codespaces environments—so a new codespace starts at 95%+ bootstrapped.
These changes, plus service-level optimizations, cut creation time from 45 minutes to five minutes. Still, research suggests people fall out of flow after roughly ten seconds of waiting. More work was needed.
From 5 minutes to 10 seconds
The shallow clone approach speeded up launch but deferred the cost of a full clone, and unshallowing after create generated load with distracting side effects. Any large, complex project faces a similar class of problems where cloning and bootstrapping compete for resources.
The answer was prebuilds: pools of codespaces, fully cloned and bootstrapped, waiting for a developer to connect. GitHub.com codespaces are now created and ready in 10 seconds. The engineering investment in prebuilds has paid for itself many times over, GitHub says. New hires go from zero to a functioning environment in less time than it takes to install Slack, and engineers can spin off codespaces for parallel workstreams with no overhead. When an environment falls apart, a new one can be created quickly.
New leverage and new capabilities
Codespaces also provided a wedge for optimizations that would not make sense in local environments. With prebuilds, GitHub now primes the language server cache and gem documentation, runs pending database migrations, and enables both GitHub.com and GitHub Enterprise development modes—tasks that previously meant another loop through bootstrap and setup.
Machine specs can be upgraded for every engineer with a single configuration change. The migration started with 8-core, 16 GB RAM VMs; GitHub.com runs a network of services that will consume every resource available. GitHub now provisions 32-core, 64 GB RAM VMs for its Codespaces users, upgrading every engineer’s machine with a one-line change.
Codespaces has also begun replacing the internal “review lab” platform used for previewing changes with internal collaborators. Previously, an engineer had to commit and deploy to a review lab instance, often requiring peer review, before sharing work. With Codespaces, a colleague gets a preview URL directly, with no commit, push, or deploy.
Supporting shell-based workflows
Visual Studio Code is the primary interface for Codespaces, but not every GitHub engineer uses it. Vim and Emacs users needed a path forward. The prebuilt image was updated to initialize sshd with GitHub public keys, open port 22, and forward the port out of the codespace. Shell-based engineers can now connect and run Vim, Emacs, or even ed.
Reception and status
Change is hard, especially when it involves development environments. GitHub engineers have become Codespaces superfans, and Codespaces is now the default development environment for GitHub.com. The #friction Slack channel for debugging local environments is slated to be archived.
| I used codespaces yesterday while my dev environment was a little broken and I finished the entire features on codespaces before my dev env was done building lol ~@lindseyb My friends, I’m here to tell you I was a Codespaces skeptic before this started and now I am not. This is the way. ~@iolsen I really was more productive with respect to the Rails part of my work this week than I think I ever have been before. Everything was just so fast and reliable. ~@jclem Whomever has worked on getting codespaces up and running, you enabled me to have an awesome first week! ~@bestra I do solemnly swear that never again will my CPU have to compile ruby from source. ~@latentflip |
More services and engineers are onboarding throughout GitHub daily. Across the stories emerging from the migration, one theme resonates: a better tool, more productivity, and no going back.
