
Unlocking Real-time Predictions with Shopify's Machine Learning Platform - Shopify
Learn how Shopify Data built new online inference capabilities into its Machine Learning Platform to deploy and serve models for real-time prediction at scale.
305 articles from Shopify Engineering.

Learn how Shopify Data built new online inference capabilities into its Machine Learning Platform to deploy and serve models for real-time prediction at scale.

The Ruby LSP is a new language server built at Shopify that makes coding in Ruby even better by providing extra Ruby features for any editor that has a client layer for the LSP. In this article, we’ll cover how we built the Ruby LSP, the features included within it, and how you can install it.

Monkey patching is considered one of the more powerful features of the Ruby programming language. However, by the end of this post I’m hoping to convince you that they should be used sparingly, if at all, because they are brittle, dangerous, and often unnecessary. I’ll also share tips on how to use them as safely as possible in the rare cases where you do need to monkey patch.

Let’s talk about technical debt. Let’s talk about practical usable approaches for actually paying it down on a daily, weekly, monthly, and yearly basis. Let’s talk about what debt needs to be fixed now versus what can wait for better planning.

We recently released ShopifyQL Notebooks, a powerful self-serve tool which enables merchants to explore their business metrics without any SQL expertise. Here’s how we used ShopifyQL and commerce data models to make it possible.

While we’re working on getting our Shopify Functions infrastructure ready for the public beta, we thought we’d use this opportunity to shine some light on how we brought JavaScript to WebAssembly, how we made everything fit within our very tight Shopify Function constraints, and what our plans for the future look like.

The intensive data work behind Shopify's Tax Insights feature required building functionality to handle dynamically changing information

The hardest part of writing tests is setting things up. Test suites have complicated helper tools, stubs, and fixtures. Learn 4 things you can do to get started.

Let’s unpack our approach to BFCM Scale Testing to explore some of what it takes to ensure that our ecommerce platform can handle the busiest weekend of the year.

This story looks at the opportunity Dev Degree gave me, the challenges I overcame, and the weaknesses that turned out to be strengths. If you’re thinking about a career in tech, but don’t think you have the stuff for it, this story is for you.

When working with draggable elements in React Native mobile apps, there are some simple ways to help gestures and animations feel better and more natural.

YJIT, a just-in-time (JIT) implementation on top of CRuby built at Shopify, is now production-ready and delivering major improvements to performance and speed. Maxime shares the updates that have been made in this newest version of YJIT, and future plans for further optimization.

Good documentation has many benefits, but first and foremost it boosts engineers’ productivity. Here are a few tips to encourage a strong writing culture.

In this blog post, Pedro explains why we rewrote Shopify CLI into Node instead of Ruby, the tradeoffs that came with the decision, the principles we embraced in this new iteration, and the challenges and ideas ahead of us to explore.

VCR is a Ruby library that records HTTP interactions and plays them back to your test suite, verifying input and returning predictable output. Here’s how and when Prater (Staff Engineer) recommends you use it.

Check out this guide for using a Monte Carlo simulation to identify the size and confidence percentage of your sampled success metric.

A look at React Native Skia in 2022 and what kept Christian Falch, Colin Gray, and William Candillon busy. Plus, a look at what's ahead for the library in 2023.

In 2020, we announced that React Native is the future of mobile at Shopify. As part of that journey, we’ve been migrating Shopify Mobile (our largest app at 300 screens per platform) from native to React Native. Here’s how it’s going.

Shopify is improving CRuby’s performance in Ruby 3.2 by optimizing the memory layout in the garbage collector through the Variable Width Allocation project.

Senior Staff Engineer and Rails core team member Aaron Patterson recaps his favourite Ruby and Rails updates of 2022 and some predictions (okay, so maybe it’s more of a wish list) for 2023.

GitHub personal access tokens (PATs) are like a key: a very, very large key that opens a very, very wide door. Long-lived tokens that have all the access of a developer’s account won’t just cause a leak—it’ll be a flood. Here’s how we rotate tokens automatically, and how you can do t too.

Shopify is adopting Rust as our systems programming language, and joining the Rust Foundation.

Earlier this year, we shared our tips for optimizing large stateful Apache Flink applications. Below we’ll walk you through 3 more best practices.

It’s been around two and a half years since Shopify became a fully remote-first company. James Stanier shares three tips for succeeding remotely.

What do you do with a finite amount of time to deal with an infinite number of things that can go wrong? This post breaks down a high-level risk mitigation process into four questions that can be applied to nearly any scenario in order to help you make the best use of your time and resources available.

We walk through how we implemented an SSE server that's scalable and load-balanced to simplify and improve a real-time data visualization application.

A step-by-step guide for extracting Datadog metrics in Python for in-depth analysis in Jupyter Notebooks.

After Shopify went all-in on React Native, we had to find a way to confirm our mobile apps are fast. The solution is an open-source @shopify/react-native-performance library, which measures rendering times in React Native apps.

Ashwin explains why and how we implemented server-driven UI in the Shop App’s Store Screen, and his experience working on the project as a Dev Degree intern.

Here's how we launched the React Native List library FlashList as an open-source project, and how you can make your next open-source project a success.

Shopify wanted a cache format that would not blow up when we shipped code changes. Part two of Caching Without Marshal describes the MessagePack format, and how we migrated to MessagePack.

Shopify wanted a cache format that would not blow up when we shipped code changes. Part one of Caching Without Marshal describes Marshal, Ruby’s ultimate sharp knife, able to transform almost any object into a binary blob and back.

Shopify's Raman Lally delves into the Apollo GraphQL client cache and the life cycle of objects that are cached within it.

We discovered a query that nearly cost us $1 million USD a month in BigQuery. Below, we’ll share our tips for lowering costs in BigQuery.

We are very excited to announce that the open-source web framework Remix and its team are joining Shopify.

There are 3 management poles for developer infrastructure teams that are thus tricky to balance: management support, system and domain expertise, and road maps.

Words I've often heard associated with cross-time zone software engineering teams include “challenging”, “tricky”, and “difficulties”. Here are a few tips to make asynchronous communication a positive experience.

With the end of office centricity at Shopify, suddenly the work of securing our devices got a lot more important, and a lot more difficult. So, we built Hubble.

We share the framework that’s worked best for structuring our data teams and that's empowered them to influence business decisions and drive impact.

As packages frequently depend on other packages, could some of these big (more than 180M downloads) packages depend on small (less than 180M downloads) packages?

Unorganized pull requests are the bane of large codebases. Follow Yash's tips to prevent your PRs from getting roasted in the group chat.

Do we need to read code before editing it? In order to safely fix a bug or update a feature, we may need to learn some things about the code. However, we’d prefer to learn only that information.

How we built Flex Comp: Shopify's new approach to compensation that gives employees the ability to choose how they want to allocate their total reward between base salary, Restricted Stock Units (RSUs), and Stock Options today, with new features like Shop Cash in the future.

Recommendations from Shopify's Data Migrations team for adding NOT NULL columns safely to your own database tables, based on a recent investigation into schema change safety using using the popular Large Hadron Migrator (LHM) gem.

A few tips for software engineers who want to deliver effective feedback during pull request reviews.

Pair programming is one of the most important tools we use while mentoring early talent in Shopify's Dev Degree program. It’s an agile software development technique where two people work together, either to share context, solve a problem, or learn from one another.

Shopify share with you the challenges we faced building Android widgets and how we solved them.

Shopify shares with you how we approached building iOS widgets and some of the challenges we faced.

At Shopify, we've embraced full stack data science, so we chatted with our data scientists to share what it means to be a full stack data scientist.

We’ll delve further into Shopify’s open source Quilt repo that contains 72 npm packages, one of which is the react-form library.

How Shopify scaled our Server Pixels service to increase its event processing performance by 170%.

While it has many benefits for customizing business logic, WebAssembly poses new challenges when debugging. In this post, Jeff covers four approaches that can be taken to step debug code that will be running in a WebAssembly environment.

This post is about how we moved from focusing on state (what an object is) towards behavior (what an object does) while building our Purchase Options APIs as a way to create a more maintainable codebase.

Watch Shopify's tech talent tackle topics ranging from virtual machines, performance improvements, and investing in open source teams at RailsConf 2022

To prevent a misunderstanding, it’s important for developers to understand how different cultures approach reviewing codes, and how diverse teams can establish healthier communication practices when giving and receiving feedback.

Shopify’s guide to Opportunity Sizing, a method that data scientists can use to quantify the potential impact of an initiative ahead of making the decision to invest in it.

The story of Oxygen, Hydrogen’s counterpart that makes hosting Hydrogen custom storefronts easy and seamless on Shopify.

A simplified overview of how we built two-day delivery in the Shopify Fulfillment Network (SFN).

Shopify’s React Native Foundations team created FlashList, a fast and performant list component that can be swapped into your project with minimal effort.

I had this conversation over and over before I really understood it. It goes: “X language can be as fast as a compiled language because it has a JIT compiler!” “Wow! It’s as fast as C?” “Well, no, we’re working on it. But in theory, it could be even FASTER than C!” “Really? How would that work?” “A JIT can see what your code does at runtime and optimize for only that. So it can do things C can’t!”