How Shopify Dynamically Routes Storefront Traffic - Shopify

Routing Storefront Requests With Runtime Rules

When Shopify set out to rewrite its storefront implementation in 2019, the Storefront Renderer team needed a way to shift traffic between the legacy system and the new one without disrupting shoppers. The load balancer layer, running nginx with OpenResty, became the control point. Lua modules on the load balancer could inspect each request, decide which rendering backend should handle it, and optionally verify the result against the other backend.

The team had already built a verification system in Lua to check that the new storefront matched the legacy implementation’s output. To manage the gradual migration, they needed a flexible routing mechanism that could be updated frequently as new endpoints were migrated.

From Deployed Rules to Dynamic Control

Initially, routing rules lived in a Lua file in the nginx repository. A control plane held the enabled/disabled state for each rule, which meant a problematic rule could be switched off immediately without waiting for a deploy. That worked well for toggling rules, but changing the rules themselves still required the full CI and deployment cycle.

The team moved the entire rule definition into the control plane as JSON payloads. Their chatbot, spy, became the interface for creating, updating, and deleting rules — not just enabling and disabling them. Removing the CI and deployment step for rule changes accelerated the iteration loop significantly. Rules affecting a large share of production traffic still required approval from another team member before being enabled.

Anatomy of a Routing Rule

Each rule is defined by several fields:

• rule_name: A descriptive identifier for the rule.
• shop_ids: Targets either all shops or a specific shop for testing. Test shops allow changes to be verified against live rendering without affecting real production data.
• filters: A comma-separated list of Lua function names applied to the request in a functional style. If all filters return true, the rule matches. Filters live in a Lua file, so changing them still requires a full deployment.
• render_rate: The percentage of matching traffic rendered by the new storefront.
• verify_rate: The percentage of matching traffic sent to the verifier for comparison.
• reverse_verify_rate: The rate used when a request is already being rendered by the new storefront. In this reverse-verification flow, the request is first rendered by the new implementation, then sent asynchronously to the legacy backend for comparison. The opposite flow — rendering by legacy, verifying against the new storefront — is called forward-verification.
• self_verify_rate: Samples requests bound for the active region and verifies them against the storefront deployment in the local region.

An example filter, is_product_list_path, targets HTTP GET requests to the storefront products JSON API implemented in Lua.

The self_verify_rate option came from a specific architectural constraint. With the legacy storefront, only one region had access to the MySQL writer at any time, so all requests had to be routed to the active region of a pod. The new storefront decoupled rendering from the database writer, allowing requests to be served from any region with a MySQL replica. As the team began testing region-independent rendering, they wanted to compare the new storefront’s output between the active and passive regions, which led to self-verification.

Beyond Migration: Load Testing and Resilience

The flexible routing system proved useful for more than the migration itself. Before BFCM 2020, the team wanted to understand how the new storefront would behave if dependencies like Redis went down. Load generation tests were unsuitable because the load shedder would drop synthetic requests under heavy load, and testing with real traffic risked affecting customers.

Instead, the team stood up a separate storefront deployment that received no direct traffic. They used the verifier mechanism to send duplicate requests to it, disconnected it from Redis, and ran load generation at full capacity. This produced data on application behavior during partial outages and let the team improve resiliency before the peak shopping season.

How the Lua Module Routes and Verifies

The routing logic is spread across several nginx request phases. During the rewrite phase, before the request is proxied upstream, the routing rules are checked to determine which storefront implementation should handle it. During the header filter phase, the system checks whether the request should be forward- or reverse-verified. In the log phase, if verification is needed, a copy of the original request is queued to be sent to the opposite upstream after the request cycle completes.

Both the renderer and verifier modules call the same find_matching_rule function from a shared storefront rules module to look up the matching rule from the control plane. A routing_method parameter distinguishes whether the lookup is for rendering or verifying the current request.

Verification requests are sent out of band using nginx timers, so the client doesn’t wait for both upstreams. The send_verification_request_in_background function queues the request, preserving the original request arguments and the response state from whichever storefront rendered it first. This information is passed as arguments to the timer, since it isn’t accessible from the timer’s context.

What Comes Next

With the new storefront now serving nearly all storefront traffic, the team is beginning to simplify this system. Once the migration is fully complete, there will be no need to verify against or render with the legacy implementation, and the dynamic routing will revert to a simpler hardcoded approach. The routing rules won’t disappear entirely, though. The flexibility they provided was essential for building the verification system and for standing up isolated deployments for load testing — neither of which was possible with the legacy storefront. The rule system will evolve to support new features as the platform continues to develop.