Why Shopify Rebuilt Its ML Platform
Shopify’s machine learning platform team supports a broad range of use cases, both internal—such as fraud detection and revenue forecasting—and external, like product categorization and recommendation systems for merchants and buyers. Those workloads frequently conflict in their requirements for inputs, data types, dependencies and integrations. Rather than force a single rigid workflow, last year the team decided to redesign the platform from scratch around flexibility.
The result is Merlin, a machine learning platform built on open source tooling. The choice was deliberate: open source end to end lets Shopify both draw from and contribute to active communities while keeping the platform adaptable as user needs evolve. Merlin is designed to let teams train, test, deploy, serve and monitor models quickly, with three priorities:
- Scalability: robust infrastructure that scales machine learning workflows.
- Fast iteration: tooling that narrows the gap between prototyping and production.
- Flexibility: support for whatever libraries or packages a model requires.
The first iteration of Merlin focuses on training and batch inference.
Under the Hood: Workspaces and the API
Most large-scale data modeling and processing at Shopify runs elsewhere in the data platform, using tools like Spark, with results stored in the data lake or Pano, the feature store. Merlin treats those as inputs for its jobs: preprocessing, training and batch inference.
Each Merlin use case runs in a dedicated environment called a Merlin Workspace, defined by its tasks, dependencies and resource requirements. These environments support distributed computing, which is what lets machine learning tasks scale. Behind the scenes, a Merlin Workspace is a Ray cluster deployed on Shopify’s Kubernetes infrastructure, designed to be short-lived for batch processing that runs for a defined period.
The Merlin API is a consolidated service that provisions workspaces on demand. Users interact with their workspace from Jupyter Notebooks for prototyping, or orchestrate it through Airflow or Oozie.
Ray as the Core
Merlin’s architecture—workspaces especially—is enabled by Ray, an open source framework with a universal API for building distributed systems and parallelizing machine learning workflows. Ray includes a broad ecosystem of libraries for distributed scikit-learn, XGBoost, TensorFlow, PyTorch and more.
With Ray, a cluster distributes computation across CPUs and machines. A typical workflow starts with ray.init() to launch a runtime locally or connect to an existing cluster locally or remotely. That means the same code that runs on a laptop can run distributed without changes; remote execution uses the Ray Client API. For example, the Ray–XGBoost integration distributes training by defining the number of Ray actors and resources like CPUs and GPUs per actor.
Shopify chose Ray largely because machine learning development at the company is done in Python. Ray keeps entire workflows in Python, integrates with the libraries Shopify already uses, and adds distribution and scaling with little or no code changes. Each Merlin project includes Ray as a dependency, using it for distributed preprocessing, training and prediction.
Ray also smooths the path from prototype to production. Data scientists can start on local machines or in notebooks and still distribute work on a remote Ray cluster from the outset, allowing them to run at scale early.
Because Ray evolves quickly with short release cycles, Merlin has adopted several of its newer capabilities:
- Ray Train for distributed deep learning with TensorFlow and PyTorch.
- Ray Tune for experiment execution and hyperparameter tuning.
- Ray Kubernetes Operator for managing Ray deployments on Kubernetes and autoscaling clusters.
Inside the Merlin User Journey
Merlin is designed to carry a machine learning use case from initial prototype to scheduled production job. The workflow follows a consistent cycle: create a project, prototype in a workspace, productionize, automate, and iterate when the model needs updating.
- Create a project: Define a Merlin Project containing code plus the system-level packages and Python libraries the work requires.
- Prototype: Spin up a Merlin Workspace, a sandbox backed by a distributed Ray cluster, and experiment using Jupyter notebooks.
- Move to production: Push updated code back to the Merlin Project; CI/CD builds a new Docker image.
- Automate: Schedule workflows through orchestrators such as Airflow, which call the production Merlin API.
- Iterate: Launch another Workspace to try different models, features, or parameters whenever changes are needed.
Merlin Projects and Workspaces
A Merlin Project is built around a Docker container with its own virtual environment (Conda, pyenv, or similar) to isolate code and dependencies. A config.yml file declares machine learning libraries and other dependencies, while source code lives in src. Pushing code to a branch triggers the CI/CD pipeline to build a custom Docker image.
The Merlin API abstracts infrastructure details such as Kubernetes namespace creation, ingress configuration, and service accounts. A Workspace request specifies the resources the job needs—GPUs, CPU count, memory, and machine type—and the API provisions a Ray cluster from the project's prebuilt Docker image. The following example payload requests a Workspace for Shopify's product categorization model:
That configuration creates a cluster of 20 Ray workers, each with 10 CPUs, 30 GB of memory, and one NVIDIA T4 GPU, with the ability to scale to 30 workers. When the job finishes, the Workspace can be shut down manually or automatically, releasing resources back to Kubernetes.
Users write and test code from Shopify's centrally hosted JupyterHub, launching notebooks from the Merlin Project's Docker image. From the notebook, they connect to the Workspace remotely using the Ray Client API, distributing work as Ray Tasks and Ray Actors across the cluster. Working directly against Ray resources from the start keeps prototyping close to production behavior.
Production Orchestration and Observability
Productionizing a prototype means pushing the finished code back to the Merlin Project so CI/CD builds the hardened Docker image. Merlin integrates with Shopify's existing data tooling: orchestration is defined as declarative YAML templates or as a DAG in Airflow. Scheduled jobs call the production Merlin API to create Workspaces, run training or prediction, and tear the Workspace down.
The DAG above shows a typical training flow: create a Workspace, train, then delete the Workspace to release cluster resources. Monitoring is built in: every Merlin Workspace receives a dedicated Datadog dashboard for tracking job load and resource consumption, and all job logs go to Splunk for debugging.
Porting a Real Workload: Product Categorization
The product categorization model was an early Merlin adopter, chosen because it demands large-scale computation and has complex training and batch inference workflows. Both were migrated and rewritten to use Ray.
Training with Ray Train
Distributing the TensorFlow training code required only modest changes thanks to Ray Train. The training step logic remains inside a function; the main function adds Ray configuration specifying worker count, backend, and GPU usage. The rewritten main function is shown below:
Batch Inference with Ray ActorPool
Each inference step was migrated separately. Ray ActorPool, analogous to Python's multiprocessing.Pool, distributes batch prediction across a fixed pool of actors. A Predictor class (implemented as a Ray Actor) loads the model and performs predictions; the main function sizes the pool from ray.available_resources()["CPU"] before sending dataset partitions. The team plans to migrate to Ray Dataset Pipelines for more robust load distribution independent of partition count or size.
Merlin's Roadmap
Shopify intends for Merlin to become the single home for its machine learning workflows. Current milestones include migrating all existing use cases onto the platform with a low-code onboarding path, adding support for real-time model serving, introducing a model registry and experiment tracking, and building monitoring tailored to machine learning. The platform, while young, is already delivering the scalability and iteration speed it was designed for.
