Product vocabulary: embedding, quantization, and collision handling
The catalog contains several billion products. Raw product IDs are arbitrary database identifiers with no semantic content, and giving each ID its own vocabulary entry would mean billions of representations to learn — most of them from very sparse training data.
Instead, product content is encoded into an embedding, and a tokenizer converts that embedding into a semantic ID — a short sequence of discrete codes the model can learn and generate. This builds on the semantic-ID direction outlined in Shopify's earlier generative recommendations post.
Between the two common tokenizer methods, RQ-VAE and RQ-KMeans, the team selected RQ-KMeans, which produced better downstream performance at the same codebook shape. It clusters product embeddings, subtracts the selected cluster centers, and repeats on the residuals, so each level captures what the previous one left behind. The final configuration uses three codebooks of 8,192 entries each, giving every product a three-token semantic ID.
Because related products can share codes, their training examples update shared representations — letting low-interaction products benefit from training on their neighbors. The catalog makes this concrete: products sharing the prefix [6001] include vests and gilets across styles and fabrics. The branch [6001, 5561] groups padded gilets in various colors, [6001, 5561, 0] narrows to gray padded designs, and a second branch [6001, 4551] covers workwear-style gilets, with [6001, 4551, 879] containing indigo denim vests.
Complete code sequences can collide across multiple products. The team explored several serving-time resolutions, including a lightweight ranking step that uses popularity when mapping semantic IDs back to products; the technical report details these approaches.
Training recipe: four lessons
The recipe turns a pretrained language model into a commerce model in stages: linking semantic IDs to product meaning, learning from buyer journeys, refining predictions per task, and merging specialized experts back into one shared model. Developments along the way produced four key lessons.
1. Search and recommendation reinforce each other
Both tasks are trained jointly over the common product vocabulary. Continued pretraining examples pair a context with a completion the model must predict:
To measure cross-task contribution, the team removed either search or recommendation data from the training mix:
| Training mix | Search relevance OneDCG@10 | Search purchases Recall@10 | Recommendation Recall@50 |
|---|---|---|---|
| Baseline | 28.8 | 33.4 | 21.0 |
| No search data | — | — | 20.9 |
| No recommendation data | 28.5 | 32.6 | — |
When a stream is removed, its share of training tokens is redistributed proportionally across the remaining streams. Dashes indicate metrics not reported for the excluded task.
The results confirm that shared training improves both tasks.
2. Discovery requires explicit training signals
Rule-based candidate generators already handle retargeting products from the buyer's history. The harder problem is surfacing items the buyer has never interacted with, and a model trained on histories full of repeat visits tends to overweight familiar products.
To teach discovery directly, the team upsamples first-purchase streams: a buyer's first purchase of a product, from a merchant, or in a category. Histories are also compressed — old clicks and stale interactions are dropped while purchases are kept — freeing context room for recent behavior.
Discovery recall, which counts purchases of previously unseen products, tracks this goal more directly than overall recall. In an offline experiment, the combined recipe lifted discovery recall by roughly 10–15% relative at cutoffs of 10 and 50, with overall recall up about 2% relative.
3. RL sharpens retrieval quality
Supervised fine-tuning trains the model to reproduce historical examples, but next-token accuracy does not fully reflect retrieval quality — many products can satisfy one intent even when the training example records a single purchase or click. For search, an LLM relevance judge supplies the reward: the model generates candidates, the judge scores their relevance, and reinforcement learning updates the model's choices. This added a +1.5 lift in OneDCG@10.
4. Specialists can be consolidated without regression
Task-specific fine-tuning and post-training turn the shared foundation into dedicated search and recommendation experts. The target is one shared model that keeps the per-task gains, which is achieved through multi-teacher distillation: each expert specializes on its own task, and a shared student inherits the capabilities of both, picking up each expert's improvements as they are made.
Serving the generator within the request budget
Because a live search or recommendation request must be answered under strict time constraints, the serving path is tuned for quality and latency at once — first in model choice, then in how much work goes into producing valid product codes.
Model selection
The model acts purely as a candidate generator and shares the live request's latency budget with the ranking stages that follow. Shopify selected Liquid AI's LFM2-2.6B, which gave the best trade-off between quality and latency and still left room for downstream work. In production the model is served on Liquid's RIPL inference stack.
| Model |
Search relevance OneDCG@10 |
Search purchases Recall@10 |
Recommendation Recall@50 |
p95 latency |
|---|---|---|---|---|
| LFM2-2.6B — selected | 24.9 | 32.3 | 25.3 | 25.5 ms |
| Qwen3-1.7B | 24.9 | 32.5 | 26.0 | 46.1 ms |
| Qwen3-14B | 25.3 | 35.0 | 26.7 | 108.4 ms |
The comparison holds the data mix fixed for all models and evaluates them on the same search and recommendation benchmarks. Server-side p95 latency is measured with TensorRT-LLM in a closed-loop benchmark where each client waits for its request to finish before issuing the next one. Latency here is the wall-clock time of the server generator call alone, with a beam width of 100 to produce 100 candidates; LFM2-2.6B runs a customized TensorRT-LLM engine. Requests are sampled from real production traffic, with prompt lengths of 22 tokens at p50 and 620 tokens at p95.
Cheap constrained decoding
Generated codes must map to eligible products, and only certain token combinations correspond to catalog entries. Generation is therefore constrained by an index of valid semantic-ID paths. The cost of checking those paths is the obstacle, and Shopify builds on STATIC, which encodes the catalog's prefix tree as a sparse GPU index so continuation validity is checked with vectorized operations. A CUDA/C++ plugin inside TurboMind applies the constraints at decode time, bringing constraint overhead from over 50 ms down to about 1 ms per request.
Per-request eligibility filters, such as the buyer's market, add a second problem: they determine which semantic-ID paths may be generated at all. Shopify says it will cover this metadata routing for constrained generative retrieval in a follow-up post.
All three codewords in one pass
Each candidate product is described by three codewords, and autoregressive (AR) decoding emits them sequentially. Multi-token prediction (MTP) instead produces distributions for all three positions in a single model pass and combines them into valid semantic IDs, cutting the sequential work per candidate. The design of MTP, its serving optimizations and its quality/latency/throughput tradeoffs are reserved for a later post.
Figure 6 benchmarks AR against MTP at the current production configuration; non-personalized search prompts on the query alone, while personalized search also supplies buyer-profile context.
The time MTP frees up is spent on a larger candidate set, which a lightweight ranker then trims to the best options. Combined, MTP and the ranker deliver a significant quality gain over AR at lower p95 latency.
Next: one model for the discovery stack
Discovery today is a multi-stage funnel — retrieval pulls candidates from the large catalog, and successive ranking stages narrow and order them. The shared generative model currently works as a candidate generator inside that funnel, and since each stage bounds what the next can see, a relevant product missed at retrieval cannot be recovered by a later ranker.
The longer-term direction is to fold more of the stack into one discovery model that reasons about the buyer's situation, infers intent, generates candidates via semantic IDs and ranks them — a shared understanding serving search, recommendation and ads while honoring the buyer's privacy choices. The fully shared path below is that longer-term target.
That requires a richer, ordered view of the shopping journey: queries, product views, purchases and store visits held in one ordered sequence, with order and timing preserved, so search and recommendation share a view of how interests and intent evolve. Three bets build on this shared context.
- Intent reasoning. For a buyer furnishing a small apartment, a table purchase changes what is useful next — chairs that fit both the table and the space. The aim is to connect such clues across the journey and carry them through generation and ranking.
- Richer multimodal representations, centered on visual information. Images carry shape, texture, pattern and style that descriptions may omit; tying them to text and to the products a buyer has explored should sharpen preference understanding and matching.
- Online continuous training. Fresh clicks, purchases and other outcomes would feed ongoing model updates as preferences and the catalog shift. A shorter path from interaction to update would let what is learned in search improve recommendation, and vice versa.



