July Service Disruption Traced to Partial Data Center Power Loss
GitHub recorded one incident in July that caused degraded performance across its services. The disruption lasted 59 minutes and stemmed from a partial power outage at one of the company's redundant data centers.
Timeline of the Outage
The incident began on July 21 at 13:07 UTC when the power loss resulted in reduced compute capacity. GitHub changed the status of six services to yellow at 13:12 UTC. Most customer-facing impact occurred within the first 10 minutes, up to 13:17 UTC, as requests were rerouted internally to other nodes within the data center. The status remained at yellow until full capacity was restored, a decision GitHub says was made out of caution.
Post-Incident Review
Following the event, GitHub is conducting reviews of all power feeds with each of its data center partners. The company has also identified improvements aimed at reducing recovery time once power is restored and is evaluating strategies to shorten the time required to fail over all traffic.
