One Model to Rule Thousands of Categories

With over 1M business owners on Shopify and billions of products across the platform, product diversity is massive. Two merchants selling nearly identical socks might describe them completely differently: one as a “woolen long sock,” the other as a “blue striped long sock.” Identifying similarity in that sea of text is a serious engineering challenge with equally serious payoffs.

Why Categorization Pays Off

Knowing what merchants sell enables a range of platform features: pre-filling category data for new sales channels like Facebook Marketplace, powering marketing recommendations (e.g., “t-shirts are trending, run an ad for your apparel”), identifying candidates for Shopify POS, and supporting market segmentation. The categorization system currently serves 20+ internal teams.

The Hard Parts: Scale and Structure

The classification problem has two dimensions of difficulty. First, scale: the Google Product Taxonomy (GPT) alone defines over 5,000 hierarchical categories, and the product count is well past a billion. Second, structure: unlike a flat dog-vs-cat problem, GPT is a tree. The taxonomy hierarchy means a product labeled Clothing is also an instance of Apparel & Accessories, and that parent/child relationship must inform predictions.

Featurization: Text Over Images

Featurization starts with choosing what distinguishes products. Options include title, images, description, and tags. Following Occam’s Razor, the project avoided image processing complexities and stuck with text: product title, description, collection, tags, vendor, and merchant-provided product type.

For vectorization, TF-IDF, Word2Vec, and GloVe were all candidates. But given the data volume, sophisticated embeddings requiring in-memory vocabularies wouldn't scale. The choice was a simple term-frequency HashingTF featurizer built on PySpark, capable of fixed-length numeric features regardless of vocabulary size. While semantic representation fidelity might suffer, the ability to train on all available data proved the deciding factor. Standard preprocessing preceded: stop-word removal, stripping HTML and URLs, and tokenization via splitting strings into word arrays.

## A Clever Construction for Multi-Class Learning The key modeling insight came from Kesler’s Construction. This technique turns n one-vs-all classifiers into a single binary classifier by exploding the training data with respect to labels: the target class is prepended to each token in a feature vector, repeated for every class in the output space per sample. The result is a larger binary-classification dataset, but the model itself is a single Logistic Regression. The algorithm:
  1. Start with an empty dataset modified_training_data.
  2. For each feature_vector in the original set:
    1. For each class in the taxonomy, prepend the class to every token in the feature vector to form a modified_feature_vector.
    2. Assign a binary label of 1 if the feature vector's ground truth is the class or a descendant of it; otherwise assign 0.
    3. Append the result to modified_training_data.
  3. Return modified_training_data.

This approach delivers four wins:

  • Scale: one model supports thousands of classes, avoiding thousands of classifiers.
  • Structure: embedding classes into features lets parent-category signal propagate down to children.
  • Efficiency: training a single (albeit larger) dataset demands lower computational resources than training many models.
  • Simplicity: Logistic Regression remains interpretable with minimal hyperparameter tuning.
## Inference Through Greedy Tree Traversal With a trained model, predictions flow across all Shopify products. For a sample item like a pair of socks:

  • Aggregate all text fields (title, description, tags, etc.).
  • Clean the text into something like “Check out these socks.”
  • To score the product, prepend a target class to each token for every category in the taxonomy.
  • Multiply the resulting feature vector against learned coefficients to score each category.
  • Begin at root-level categories, track the highest score, and descend only into the children of that winner, repeating down to a leaf node.

The final prediction is the full root-to-leaf path in the taxonomy. This greedy approach skips scoring most of the tree’s branches, keeping inference computationally tractable as every product in the catalog passes through the model.

Evaluating Hierarchical Predictions

Standard classification metrics treat every mistake equally, but in a hierarchical taxonomy a wrong prediction can be off by one level or by several. Misclassifying a shirt as a dress is different from classifying a phone as apparel — the former is close in the tree, the latter is in a completely different branch, but flat metrics would punish both identically.

Shopify addressed this by applying established work on hierarchical evaluation measures that use taxonomy structure to weight errors. These include hierarchical accuracy, precision, recall, and F1. Each variant's calculation is essentially the same as its flat counterpart, but scoring is regulated by the distance to the nearest common ancestor rather than treating all incorrect predictions as equally wrong. For example, Dresses and Shirts & Tops share a common ancestor at Clothing, just one level up; Phones and Shirts & Tops, by contrast, are several levels away from any shared node.

Using that distance as a proxy for the magnitude of an error lets the team assess model performance in a way that matches how the taxonomy will actually be used. The lesson here is to verify that conventional metrics fit your problem before relying on them.

Handling Misclassifications

A probabilistic model will always make some wrong predictions, and chasing 100% accuracy is neither realistic nor the right target. Since the categorization output feeds downstream features and business decisions, the more important question is what happens after a mistake is discovered.

Shopify built a feedback loop around schematized Kafka events and an in-house annotation platform, enabling a flexible human-in-the-loop workflow. Any downstream consumer of category data can plug in and submit corrections or additions, which then flow back into the model's training data. The same mechanism also supports entirely new streams of data — including category information directly from merchant-facing products — rather than only rectifying existing errors.

Known Limits and Future Directions

The baseline model performs well but has clear areas for improvement, all tied to data and product expansion.

Class Imbalance

Shopify's merchant base skews heavily toward certain product types, which biases training data toward those categories. That could leave merchants in less-common industries with weaker categorization results. Re-balancing techniques like minority class oversampling (for example SMOTE), majority class undersampling, or weighting loss by class size are all potential paths forward.

Non-English Text

The model is trained on English-language product descriptions, but Shopify's international growth depends on supporting other languages. Pre-trained multilingual models, such as Google's Multilingual Sentence Embeddings, offer a low-friction way to extend coverage without building language-specific pipelines from scratch.

Product Images

Images are a language-independent signal that could improve categorization for products from any region or merchant. Training an image model from scratch is resource-intensive, but Shopify has experimented with using pre-trained image embeddings like Inception v3 and attaching a CNN for classification, which reduces the computational overhead enough to be practical at scale.

The simple model architecture was a deliberate trade-off between interpretability and resource usage, and it paid off in the ability to solve this problem across Shopify's merchants. A shared product language opens up downstream capabilities — identifying trending categories, flagging industries prone to fraud, or improving storefront search — that are otherwise difficult to build on top of free-form merchant data.