Containers as a service at GitHub

GitHub’s developer experience engineering group maintains an internal “paved path” for deploying, scaling, and running containerized services. Built on top of Kubernetes, this ecosystem bundles conventions, tooling, and automation into a single workflow for engineers shipping everything from small internal utilities to APIs supporting github.com workloads.

Kubernetes provides the foundation across a multi-cluster, multi-region topology. Above that layer sits a mix of Hubot-driven ChatOps commands, GitHub Apps, CI pipelines, and custom runtime applications that give developers a consistent deployment experience without requiring them to manage clusters directly.

Why centralize the runtime layer

Running hundreds of services on dedicated VMs would create obvious inefficiencies: fragmented capacity planning, duplicated infrastructure expertise, limited application visibility, and difficulty standardizing security controls. Centralizing on Kubernetes nodes lets GitHub handle capacity planning once for the node pool rather than per service, so small and large workloads share machines effectively.

This approach also enables:

  • Rapid scaling, driven by centrally managed capacity.
  • Unified configuration and deployment management via one control plane.
  • Consistent observability and performance insight per service.

Scaffolding and provisioning

Onboarding a new service starts with a ChatOps invocation in its repository:

hubot gh-platform app scaffold monalisa-app

A custom GitHub App then opens a pull request that adds the deployment scaffolding, including:

  • A deployment.yaml defining the service’s environments.
  • Kubernetes manifests specifying Deployment and Service objects.
  • A Debian-based Dockerfile running a trivial web server as a starting point.
  • CI configuration that builds images on each push and stores them into an internal container registry.

Every service receives its own Kubernetes namespace, named <app-name>-<environment>. This isolates both separate services and separate environments of the same application, with most services running staging and production variants.

Deployment mechanics

Branches are promoted to environments via Hubot ChatOps. Deploying the branch bug-fixes from monalisa-app to staging looks like:

hubot deploy monalisa-app/bug-fixes to staging

The command fetches the Docker image tagged at the branch’s latest commit, updates the Kubernetes manifests accordingly, and applies them to the clusters backing that environment—typically spanning multiple geographic sites within the runtime platform’s coordinated regions.

Merge queue and deployment pipelines orchestrate automated rollouts for active branches, with full visibility into progress during deployment.

Security architecture

Platform security combines automation by Security and Platform teams with standard engineering policies like two-person PR reviews and branch protection on production repositories. Additional measures include:

  • Pre-built base Docker images vetted to contain minimal, auditable dependencies with current security patches.
  • Continuous vulnerability and dependency scanning of packages and running images via Dependabot and GitHub’s software supply chain tooling.
  • Secret and code scanning across repositories using GitHub’s advanced security features.
  • Restricted direct access to underlying Kubernetes infrastructure via layered authentication.
  • Full telemetry for threat detection, with services confined to GitHub’s internal network by default.

Secrets for each service and environment are stored in a centralized secret store. Kubernetes injects the appropriate credentials into pods for each container at deployment time.

The complete flow, merge to live

The end-to-end process is diagrammed below.

  1. A developer merges a PR into a target branch—for example, bug-fixes in the monalisa-app repository, which also holds the Kubernetes manifest templates.
  2. The merge triggers CI. One workflow builds the container image from the Dockerfile and pushes it to the internal artifact registry.
  3. After CI succeeds, the engineer runs hubot deploy monalisa-app/bug-fixes to staging.
  4. Build systems pull the manifest files from the branch, inject the latest image from the registry, add secrets from the secret store, and output a ready-to-apply manifest.
  5. Deployment tooling applies the manifest to the appropriate clusters and monitors the rollout.

The end goal is straightforward: let GitHub developers focus on the code that differentiates their service while the paved path handles the underlying runtime, capacity, security, and rollout concerns.