December Availability Overview
GitHub recorded no incidents with service downtime in December. The engineering team continued work on a resiliency issue surfaced in November, and this report details the root cause and the remediation steps taken.
Technical Follow-Up: November 27 Incident
Deep-dive analysis of the November 27 outage (16:04 UTC, duration 1h01m) revealed an edge case that caused an unusually high volume of GitHub App token requests. The request surge drove abnormal replication lag in a MySQL cluster supporting GitHub Actions, which in turn amplified query load and degraded the database nodes responsible for handling App token requests.
GitHub Actions passes an App token to each invoked Action for authenticated tasks. When the replicas lagged, some token lookups failed because the replicas lacked current data. To prevent this failure mode, the team is modifying the relevant queries so that token request bursts can no longer overwhelm the database tier.
Reliability Investments
Work on engineering systems continued through December, with ongoing initiatives focused on flaky test management and CI workflow performance. Further details on current engineering efforts are available on the GitHub Engineering blog.
