RankFlow and TurboDSL: Closing the gap between ML iteration and C++ performance

Shopify's search infrastructure serves billions of queries during peak events like Black Friday Cyber Monday, spanning real-time indexing, retrieval, ranking, and experimentation across millions of merchant storefronts and the Shop app. The ranking pipeline blends classical information retrieval with modern ML: typo correction, synonyms, faceting, and semantic search, all wrapped in privacy controls, multi-language support, and merchant-facing merchandising APIs.

The core engineering tension: data scientists need to iterate on transformers, neural rankers, and gradient boosting models daily, while the serving layer must sustain millisecond latency at global scale. A pure C++ system delivers speed but stalls iteration; Python or Java accelerate experimentation but introduce latency and memory overhead that don't hold up at this query volume. Hybrid approaches—Python calling into C++—bring deployment complexity, version skew, and operational burden.

Shopify's answer is a pair of systems: RankFlow, a domain-specific language, and TurboDSL, a compiled execution engine. Together they let data scientists write Python-like ranking logic and deploy it in minutes, while the runtime executes at native C++ speed.

What commerce search optimizes for

Commerce search differs from general web search in what it ranks for. The system weighs multiple signals beyond keyword matching:

  • Relevance: Results must match shopper intent directly, not merely contain query terms.
  • Purchase popularity: Products with actual orders outrank those with high view counts but few buyers.
  • Brand trust: Established brands with significant GMV contribution receive higher weight for ambiguous queries like "running shoes."
  • Navigational intent: When a shopper searches a specific brand or product, the engine recognizes that and routes them straight to the relevant store or product page.

Each result is a potential conversion, which means relevance and latency decisions directly affect merchant revenue.

Why build a custom engine

Shopify acquired Vantage Discovery in March 2025 to bring consumer discovery expertise into its commerce search. Off-the-shelf engines like Elasticsearch and Solr are solid foundations, but meeting commerce scale required re-architecture along several axes: real-time inventory updates across millions of merchants, complex pricing and product variants (size, color, location-aware availability), merchant-specific ranking in a massive multi-tenant environment, and worldwide sub-second latency.

C++ was the natural base layer. It delivers the response times needed for millions of concurrent shoppers, memory efficiency for hundreds of millions of items, and fine-grained control over cache behavior and hardware utilization. Every performance gain also reduces infrastructure cost. The missing piece was a way to let ML teams move without waiting on C++ development cycles.

RankFlow: Python-like syntax, C++ execution

RankFlow sits at the intersection of the two demands. Data scientists write ranking modules in a Python-like DSL with no C++ expertise required. The language supports the full range of ML models and features used in production ranking, plus type safety and compile-time validation to catch errors before they reach the serving path.

The DSL abstracts away the performance-sensitive details. Vectorized operations and cache-optimized execution are handled by the runtime, giving predictable latency and stable memory usage while keeping the iteration loop in a declarative layer that's quick to edit, review, and deploy.

Text matching at scale

A notable capability comes from TextStreams: a single line of DSL extracts over 80 text similarity features, applying different matching functions across various document fields (TfIdf for Title, QueryInText for Description, ExactMatch for Brand, and so on).

The ML workflow itself follows a standard pipeline: models trained on historical query data with Catboost, LightGBM, or neural networks; offline validation via precision/recall; online A/B testing; then production deployment with RankFlow handling inference at C++ speed.

TurboDSL: A two-phase build

The execution engine behind RankFlow was built pragmatically. The team's philosophy: ship fast, optimize later, maintain compatibility always.

Phase 1 (weeks 1-4): SimScorerDSL, a C++ engine that compiled DSL directly into optimized C++ code. It delivered essential functionality quickly, letting data scientists begin experiments and proving the DSL was viable for production ranking.

Phase 2 (months 2-4): TurboDSL, a high-performance engine developed in parallel with the ML team's ongoing experiments. It preserved the exact same DSL syntax—no disruption for data scientists—and achieved a 48% speedup in ranking feature computation.

Performance gates on every change

Maintaining that performance requires visibility. Every pull request to the search stack gets automated performance analysis with component-level timing breakdowns: DSL execution, model inference, embeddings, and text matching. Statistical significance testing with a ±2% tolerance distinguishes meaningful changes from noise, classifying each PR as an improvement, regression, or neutral. Side-by-side result comparisons catch accuracy regressions before they hit production.

This automated gate ensures every engineer understands the performance impact of their changes before merging. No guessing, no surprises at BFCM scale.