Generative Recommenders (GenRecs) move ads ranking away from scoring each candidate ad in isolation and toward reasoning jointly about ad content and user preferences using LLM architecture. That shift lets the system apply pre-trained world knowledge to understand what an ad represents and what a person actually wants, predicting relevance even for brand-new products with no engagement history behind them.

GenRecs Tokenization is the data representation layer underneath it. Instead of holding a dedicated ID for every item, the system describes each item with a set of tokens drawn from a stable semantic vocabulary. In combination with Meta's Generative Ads Model (GEM) and Adaptive Ranking Model, GenRecs Tokenization has delivered a +0.35% lift in conversions across surfaces for advertisers.

Why semantic tokens replace per-item IDs

Language models represent concepts by composing a small vocabulary of semantic tokens. GenRecs Tokenization adopts that approach for ads, products, services, pages and pixels: an item is described by what it is, so the model can connect an item with no engagement history to items it already knows well. A newly launched "trail shoe" inherits everything learned about that category instead of starting from an empty representation.

The approach supports GenRecs in both retrieval and ranking across every item type in the system.

How the tokenizer works

Pre-trained multimodal models read the text and images of each ad, product and page and encode it into an embedding carrying world knowledge. The tokenizer merges that content signal with representations learned from engagement, then quantizes the result into a short sequence of discrete tokens. Items that resemble each other share tokens the way related words share roots — two running shoe ads from successive seasons overlap, so a new campaign starts from what the model learned about the previous season's products.

The pipeline has three stages:

  • Encode the item into a semantic embedding space.
  • Decompose the embedding into independent subspaces.
  • Quantize by mapping each subspace to an entry in a codebook.

The right combination of stages depends on entity and feature semantics. For many ad representations, GenRecs Tokenization uses prefix n-grams combining a learned encoder, residual decomposition, learned vector codebooks and token parametrization.

Intrinsic token quality tracks ranking accuracy

Tokenizer quality can be measured without reference to downstream ranking gain, using:

  • Cluster purity: items sharing a code tend to share a category, giving the model a cleaner per-token signal.
  • Distribution balance: items spread evenly across semantic IDs rather than crowding into a few.
  • Codebook utilization: a larger share of codes holds at least one item, so the vocabulary can express more distinctions.

Across the eight tokenizer versions tested, these intrinsic metrics correlated strongly with downstream ranking prediction accuracy: versions that scored better also produced better CTR prediction accuracy for ads models.

Where the gains come from

  • New entities: semantic tokens let models infer relevance and build relationships through shared concepts, which matters for newly created ads, niche products and pages with limited engagement.
  • Cross-campaign transfer: a stable vocabulary carries insight across an advertiser's campaigns.
  • LLM scaling: token-based representations let gains compound with the LLM scaling curve.
  • World knowledge: pre-trained language models get a direct interface for reasoning about ad content and user intent.

Once items are migrated to tokens, models learn from token patterns much as LLMs do. In production, this yields more accurate prediction, a compounding feedback loop (better tail predictions produce better lessons from tail ads, which enrich training data for all ads), and a new scaling axis across sequence length, training data and model architecture.

The tokenization flywheel

For high-volume campaigns, tokenization surfaces deeper semantic patterns across large product catalogs; for tail ads with few impressions, it lets models generalize through shared content structure, recovering cold-start performance and enabling cross-entity transfer. When a sportswear brand launches next season's trail shoe, the new product and its ads build on what the model learned from last season's shoe even before anyone sees the new ad. Better tail predictions also improve the training data, which in turn improves predictions for all ads — a self-reinforcing loop.

Foundations for LLM-native recommendation

Tokenizer improvements are not confined to the model they were tuned for. Optimizing codebook quality through diversity loss, dead code resets and contrastive objectives measurably improves downstream performance when the resulting tokens train generative recommenders. Meta's work on LLM-native retrieval illustrates the reach: a two-tower architecture on a shared LLM encoder reaches state-of-the-art retrieval quality while remaining efficiently servable, matching classic two-tower performance on a fraction of the training data. Work is now extending reasoning-aware ranking on top of the same vocabulary.

Next steps

Development is proceeding along three directions toward the ROI-optimal frontier:

  • Cold-start generalization: scaling tokenization coverage across all items, since tail inventory is where semantic tokens contribute most and advertiser impact is largest.
  • Tokenizer quality as a first-class objective: engagement-aligned training, a unified cross-entity vocabulary and codebook improvements. Because intrinsic measures predict downstream ranking gains, the tokenizer can be iterated without a full model training cycle.
  • Efficient serving of LLM-native architectures: fully verbalized tokenization gives the richest view of an ad but needs context windows that are impractical at ads scale, so compacted verbalized tokens are being combined with semantic IDs to keep reasoning quality while holding latency acceptable.

Further out, Meta is exploring whether one vocabulary could unify representation across surfaces — a shared token space spanning text, image, audio and video that lets models reason about a person's interests and an advertiser's offerings with a single representation rather than separately learned embeddings.

Production lift measured on Meta internal data in February 2026.

Acknowledgements

Weifeng Cui, Ling Leng, Luke Simon, Hadar Ohana, Jimmy Chen, Minghai Chen, Chiyu Zhang, Dinesh Ramasamy, Debajyoti Roy, Shawn Shi, Gajanan Golegaonkar, Xiao Yang, Ashkan Aleali, Anuj Gargava, Xizhou Feng, Liquan Huang, Frank Shyu, Xiaoyu Chen, Kungang Li, Chonglin Sun, Matt Steiner, Sandeep Pandey, Santanu Kolay, Anuj Madan, Tak Yan, and the entire team behind the development and productionization of the first large scale semantic tokenization effort in Meta's ad recommendation system.