Product vocabulary: embedding, quantization, and collision handling

The catalog contains several billion products. Raw product IDs are arbitrary database identifiers with no semantic content, and giving each ID its own vocabulary entry would mean billions of representations to learn — most of them from very sparse training data.

Instead, product content is encoded into an embedding, and a tokenizer converts that embedding into a semantic ID — a short sequence of discrete codes the model can learn and generate. This builds on the semantic-ID direction outlined in Shopify's earlier generative recommendations post.

Product content passes through an encoder to create an embedding, which a tokenizer converts into a three-token semantic ID.
Figure 2. Product content is encoded into an embedding, then converted into a short sequence of semantic-ID tokens.

Between the two common tokenizer methods, RQ-VAE and RQ-KMeans, the team selected RQ-KMeans, which produced better downstream performance at the same codebook shape. It clusters product embeddings, subtracts the selected cluster centers, and repeats on the residuals, so each level captures what the previous one left behind. The final configuration uses three codebooks of 8,192 entries each, giving every product a three-token semantic ID.

Because related products can share codes, their training examples update shared representations — letting low-interaction products benefit from training on their neighbors. The catalog makes this concrete: products sharing the prefix [6001] include vests and gilets across styles and fabrics. The branch [6001, 5561] groups padded gilets in various colors, [6001, 5561, 0] narrows to gray padded designs, and a second branch [6001, 4551] covers workwear-style gilets, with [6001, 4551, 879] containing indigo denim vests.

Two semantic-ID branches share the prefix [6001] for vests and gilets, then distinguish smaller groups as additional tokens are added.
Figure 3. Two code paths from a real catalog sample. Green highlights the token added at each level. The labels describe the products observed at each prefix; the code levels learn from embeddings rather than having fixed category, brand, or color meanings.

Complete code sequences can collide across multiple products. The team explored several serving-time resolutions, including a lightweight ranking step that uses popularity when mapping semantic IDs back to products; the technical report details these approaches.

Training recipe: four lessons

The recipe turns a pretrained language model into a commerce model in stages: linking semantic IDs to product meaning, learning from buyer journeys, refining predictions per task, and merging specialized experts back into one shared model. Developments along the way produced four key lessons.

Shared commerce training develops into specialized search and recommendation experts, whose capabilities are distilled back into one shared discovery model.
Figure 4. Shared commerce learning develops into search and recommendation experts. Multi-teacher distillation brings their task-specific learning back into one shared discovery model.

1. Search and recommendation reinforce each other

Both tasks are trained jointly over the common product vocabulary. Continued pretraining examples pair a context with a completion the model must predict:

Continued-pretraining examples pair input context with target completions. The model learns from the completion tokens rather than the prompt tokens.
Figure 5. Examples used in continued pretraining. Green shading marks the completion; prompt tokens are excluded from the training loss. Each product example uses one attribute. The recommendation target contains the buyer’s last 30 days of interactions. Profiles and semantic-ID code values are abbreviated.

To measure cross-task contribution, the team removed either search or recommendation data from the training mix:

Training mix Search relevance OneDCG@10 Search purchases Recall@10 Recommendation Recall@50
Baseline 28.8 33.4 21.0
No search data — — 20.9
No recommendation data 28.5 32.6 —

When a stream is removed, its share of training tokens is redistributed proportionally across the remaining streams. Dashes indicate metrics not reported for the excluded task.

The results confirm that shared training improves both tasks.

2. Discovery requires explicit training signals

Rule-based candidate generators already handle retargeting products from the buyer's history. The harder problem is surfacing items the buyer has never interacted with, and a model trained on histories full of repeat visits tends to overweight familiar products.

To teach discovery directly, the team upsamples first-purchase streams: a buyer's first purchase of a product, from a merchant, or in a category. Histories are also compressed — old clicks and stale interactions are dropped while purchases are kept — freeing context room for recent behavior.

Discovery recall, which counts purchases of previously unseen products, tracks this goal more directly than overall recall. In an offline experiment, the combined recipe lifted discovery recall by roughly 10–15% relative at cutoffs of 10 and 50, with overall recall up about 2% relative.

3. RL sharpens retrieval quality

Supervised fine-tuning trains the model to reproduce historical examples, but next-token accuracy does not fully reflect retrieval quality — many products can satisfy one intent even when the training example records a single purchase or click. For search, an LLM relevance judge supplies the reward: the model generates candidates, the judge scores their relevance, and reinforcement learning updates the model's choices. This added a +1.5 lift in OneDCG@10.

4. Specialists can be consolidated without regression

Task-specific fine-tuning and post-training turn the shared foundation into dedicated search and recommendation experts. The target is one shared model that keeps the per-task gains, which is achieved through multi-teacher distillation: each expert specializes on its own task, and a shared student inherits the capabilities of both, picking up each expert's improvements as they are made.

Serving the generator within the request budget

Because a live search or recommendation request must be answered under strict time constraints, the serving path is tuned for quality and latency at once — first in model choice, then in how much work goes into producing valid product codes.

Model selection

The model acts purely as a candidate generator and shares the live request's latency budget with the ranking stages that follow. Shopify selected Liquid AI's LFM2-2.6B, which gave the best trade-off between quality and latency and still left room for downstream work. In production the model is served on Liquid's RIPL inference stack.

Model Search relevance
OneDCG@10
Search purchases
Recall@10
Recommendation
Recall@50
p95 latency
LFM2-2.6B — selected 24.9 32.3 25.3 25.5 ms
Qwen3-1.7B 24.9 32.5 26.0 46.1 ms
Qwen3-14B 25.3 35.0 26.7 108.4 ms

The comparison holds the data mix fixed for all models and evaluates them on the same search and recommendation benchmarks. Server-side p95 latency is measured with TensorRT-LLM in a closed-loop benchmark where each client waits for its request to finish before issuing the next one. Latency here is the wall-clock time of the server generator call alone, with a beam width of 100 to produce 100 candidates; LFM2-2.6B runs a customized TensorRT-LLM engine. Requests are sampled from real production traffic, with prompt lengths of 22 tokens at p50 and 620 tokens at p95.

Cheap constrained decoding

Generated codes must map to eligible products, and only certain token combinations correspond to catalog entries. Generation is therefore constrained by an index of valid semantic-ID paths. The cost of checking those paths is the obstacle, and Shopify builds on STATIC, which encodes the catalog's prefix tree as a sparse GPU index so continuation validity is checked with vectorized operations. A CUDA/C++ plugin inside TurboMind applies the constraints at decode time, bringing constraint overhead from over 50 ms down to about 1 ms per request.

Per-request eligibility filters, such as the buyer's market, add a second problem: they determine which semantic-ID paths may be generated at all. Shopify says it will cover this metadata routing for constrained generative retrieval in a follow-up post.

All three codewords in one pass

Each candidate product is described by three codewords, and autoregressive (AR) decoding emits them sequentially. Multi-token prediction (MTP) instead produces distributions for all three positions in a single model pass and combines them into valid semantic IDs, cutting the sequential work per candidate. The design of MTP, its serving optimizations and its quality/latency/throughput tradeoffs are reserved for a later post.

Figure 6 benchmarks AR against MTP at the current production configuration; non-personalized search prompts on the query alone, while personalized search also supplies buyer-profile context.

Comparison of autoregressive generation and multi-token prediction for non-personalized and personalized search, showing lower p95 latency with multi-token prediction.
Figure 6. AR versus MTP p95 latency at 50 QPS per instance on B300 MIG 3g.90gb. Both columns use the same scale.

The time MTP frees up is spent on a larger candidate set, which a lightweight ranker then trims to the best options. Combined, MTP and the ranker deliver a significant quality gain over AR at lower p95 latency.

Next: one model for the discovery stack

Discovery today is a multi-stage funnel — retrieval pulls candidates from the large catalog, and successive ranking stages narrow and order them. The shared generative model currently works as a candidate generator inside that funnel, and since each stage bounds what the next can see, a relevant product missed at retrieval cannot be recovered by a later ranker.

The longer-term direction is to fold more of the stack into one discovery model that reasons about the buyer's situation, infers intent, generates candidates via semantic IDs and ranks them — a shared understanding serving search, recommendation and ads while honoring the buyer's privacy choices. The fully shared path below is that longer-term target.

Proposed future architecture: one shared model handles intent understanding, candidate generation, and ranking, with fresh buyer feedback driving continuous training.
Figure 7. The goal is a shared model spanning intent understanding, candidate generation, and ranking, with online continuous training driven by fresh buyer feedback.

That requires a richer, ordered view of the shopping journey: queries, product views, purchases and store visits held in one ordered sequence, with order and timing preserved, so search and recommendation share a view of how interests and intent evolve. Three bets build on this shared context.

  • Intent reasoning. For a buyer furnishing a small apartment, a table purchase changes what is useful next — chairs that fit both the table and the space. The aim is to connect such clues across the journey and carry them through generation and ranking.
  • Richer multimodal representations, centered on visual information. Images carry shape, texture, pattern and style that descriptions may omit; tying them to text and to the products a buyer has explored should sharpen preference understanding and matching.
  • Online continuous training. Fresh clicks, purchases and other outcomes would feed ongoing model updates as preferences and the catalog shift. A shorter path from interaction to update would let what is learned in search improve recommendation, and vice versa.