Planning for the Unknown: How Shopify Prepares for BFCM
For Shopify, the Black Friday/Cyber Monday (BFCM) weekend is the moment of truth for its infrastructure. Merchants depend on the platform to absorb a massive spike in buyer traffic, and engineering teams must ship a year’s worth of changes knowing that a single bottleneck could put storefronts at risk. The challenge is twofold: understanding the magnitude of the traffic ahead, and securing the right amount of compute and storage to handle it—plus enough headroom for regional failover and unexpected events.
Since 2017, Shopify has run its platform on Google Cloud Platform (GCP). The capacity planning process starts with forecasts from the data science team, which model expected traffic based on historical BFCM numbers and a safety margin. From there, engineers build a master resourcing plan, projecting how CPUs, storage, and other resources will scale.
Owners of the top ten resource areas are tasked with producing detailed estimates that break down machine types, geographic locations, and quantities. Components are partitioned into separate GCP projects, which makes quota management across each project much simpler. Buffers are baked into the overall numbers to leave room for flexibility—whether that means moving machines between projects or failing traffic over to another region.
Planning for a Year Unlike Any Other
2020 presented a unique forecasting problem. Typical BFCM planning leans on multi-year trends, but the COVID-19 pandemic had rapidly shifted commerce online in the spring, making historical data less reliable. Shopify’s planners had to weigh scenarios ranging from a massive surge in online traffic to a global economic downturn.
The investment risk was significant: scaling for the most aggressive scenario means spending heavily on infrastructure that might sit idle, while under-provisioning risks outages during peak shopping hours. Shopify chose to plan for its more aggressive growth scenarios to ensure platform stability, keeping finance, vendor, and internal teams informed of the reasoning behind that decision and its operational implications.
The Load Testing Calendar
A capacity plan on paper only goes so far. Scaling to projected numbers surfaces hard limits across a complex stack: the maximum number of VMs in a network, the packet throughput of a busy Memcached server, the megabytes per second a logging pipeline can absorb. Historically, these surprises surfaced only after full scale-out, leaving little time for mitigation.
That changed in 2018, with the decision to run a "faux" BFCM mid-year. The goal was to find scalability unknowns outside the November window, building organizational resilience through regular, large-scale exercises with internal codenames: Mayday, Spooky scale-up, and Oktoberfest. Running these tests on GKE means compute can be provisioned only for the exercise window, and Shopify only pays for those hours.
The load testing itself is powered by internal tooling built on the go-lua open source project. Embedded Lua scripts let engineers model realistic user journeys—browsing storefronts, adding items to cart, checking out, and processing through a mock payment gateway.
The Oktoberfest scale-up, run in early October, was designed to validate the 2020 traffic forecasts. It surfaced a set of bottlenecks that were fixed on the spot, and a subsequent test confirmed the mitigations, giving the team concrete confidence ahead of the real event.
For Shopify, BFCM is more than an operational milestone. As the platform grows, the traffic peaks seen during the holiday season become ordinary load levels within a year. The lessons drawn from these exercises—making systems more automated, dynamic, and resilient—are aimed at a infrastructure that can scale with that trajectory. From forecasting and GCP quota management to load testing and failover rehearsals, the goal is a seamless shopping weekend for merchants and a chance to architect for tomorrow’s baseline traffic.
