Automated Deployment at GitHub Scale

GitHub’s Deploy Team is a small group—six engineers, a manager, and a product manager—responsible for keeping hundreds of applications shipping to github.com around the clock. Senior Software Engineer Nina Kaufman describes the team’s mission as ensuring that engineers across the company can deploy “with a high velocity, safely and securely.” That translates to roughly 120–150 production deploys per week for github.com alone, with over 400 pull requests shipped in a typical week.

From Chat Command to Production

The deployment pipeline is built on GitHub’s own platform and data centers, driven largely through ChatOps. A series of Hubot commands gives engineers a simple interface to roll out changes. When someone wants to ship a change, they run .deploy with a link to their pull request. The system parses the link and uses the GitHub API to check CI status, authorization, and authentication before anything moves forward.

Once those checks pass, the change progresses through a series of deployment stages. The engineer can merge the pull request and move on; the system handles the rest, determining which servers are involved and what needs to run. The person who initiated the deploy doesn't need visibility into those details.

Canary Rollouts and Monitoring

Before a change reaches full production, it goes through a canary deployment. A small subset of production hosts receives the new code first, so if something breaks, only a tiny percentage of servers are affected. During this phase, engineers can watch dashboards to see how the change is performing—whether it’s increasing error rates or driving engagement. Because the rollout happens incrementally, the impact of a change is visible and manageable before it reaches all users.

Measuring Success Beyond Velocity

The Deploy Team tracks its performance through a set of service level objectives (SLOs) covering metrics like deployment time and local development setup time. They also run internal developer satisfaction surveys and interviews to gather feedback on how deployments are perceived and where the process can improve.

Rollback frequency is another key metric. The team found its rollback rate was fairly low, indicating that engineers were shipping changes that were well-tested and performed as expected. That data helped build a “culture of trust,” where developers are trusted to know what they’re shipping, and changes can progress to full rollout with minimal intervention.

That trust has real throughput benefits. Although the number of deployments per week hasn't changed much compared to several years ago, the team now batches far more changes into each deploy. Engineers spend less time waiting in line to ship, and there's greater confidence that a change will play nicely with the many others it's shipping alongside.

Treating Infrastructure as a Product

Kaufman's primary advice for teams looking to improve their deployment workflows is to treat internal infrastructure as a product, with the same rigor applied to external offerings. That means implementing developer surveys, satisfaction scores, and interviews to understand what engineers experience day to day. "Having the empathy to understand the problems of other engineers within the organization can definitely improve the product," she says, noting that feedback can sometimes be surprising or hard to hear, but it leads to a better outcome.

GitHub uses an NSAT score to measure internal satisfaction. Between May and September 2020, the company made a deliberate push to raise that score. Feature teams and infrastructure teams were brought together to discuss interface improvements and workflow changes. The collaboration produced UI/UX changes that improved shipping velocity, cut support hours spent debugging deploy issues, and raised developer satisfaction.

Consolidating tools and processes is another priority. GitHub dogfoods its own platform for authentication, access control, project management, and CI/CD with GitHub Actions. The team maintains discoverable documentation for internal APIs and runs dedicated support channels with the same first-class responsiveness you'd expect for external customers.

Distributed Team, Shared Momentum

With team members spread across Berlin, Vancouver, and the US, maintaining connection requires deliberate effort. The team schedules bi-weekly coffee chats, book clubs, and art chats to keep camaraderie strong. Kaufman describes a typical async workflow where an issue she documents overnight gets picked up and fixed by a colleague in Berlin by the time she wakes up. "It doesn't feel like I'm isolated because there's always an ongoing conversation," she says. "We're always deploying GitHub, always improving it."