
What is a Full Stack Data Scientist? - Shopify
At Shopify, we've embraced full stack data science, so we chatted with our data scientists to share what it means to be a full stack data scientist.
317 articles from Shopify Engineering.

At Shopify, we've embraced full stack data science, so we chatted with our data scientists to share what it means to be a full stack data scientist.

We’ll delve further into Shopify’s open source Quilt repo that contains 72 npm packages, one of which is the react-form library.

How Shopify scaled our Server Pixels service to increase its event processing performance by 170%.

While it has many benefits for customizing business logic, WebAssembly poses new challenges when debugging. In this post, Jeff covers four approaches that can be taken to step debug code that will be running in a WebAssembly environment.

This post is about how we moved from focusing on state (what an object is) towards behavior (what an object does) while building our Purchase Options APIs as a way to create a more maintainable codebase.

Watch Shopify's tech talent tackle topics ranging from virtual machines, performance improvements, and investing in open source teams at RailsConf 2022

To prevent a misunderstanding, it’s important for developers to understand how different cultures approach reviewing codes, and how diverse teams can establish healthier communication practices when giving and receiving feedback.

Shopify’s guide to Opportunity Sizing, a method that data scientists can use to quantify the potential impact of an initiative ahead of making the decision to invest in it.

The story of Oxygen, Hydrogen’s counterpart that makes hosting Hydrogen custom storefronts easy and seamless on Shopify.

A simplified overview of how we built two-day delivery in the Shopify Fulfillment Network (SFN).

Shopify’s React Native Foundations team created FlashList, a fast and performant list component that can be swapped into your project with minimal effort.

I had this conversation over and over before I really understood it. It goes: “X language can be as fast as a compiled language because it has a JIT compiler!” “Wow! It’s as fast as C?” “Well, no, we’re working on it. But in theory, it could be even FASTER than C!” “Really? How would that work?” “A JIT can see what your code does at runtime and optimize for only that. So it can do things C can’t!”

Shopify CTO Allan Leinwand explains how to stay technical as an engineering leader while balancing the demands of people leadership.

The process of getting SFN working on Spin, Shopify’s cloud-development tool, is a great example of the kind of hidden difficulty in technical work that's more related to human systems.

In this post, Colin Gray, Principal Developer at Shopify, delivers a crash course in how to make sense of stable values in React and when they matter most.

Top ten tips and tricks for building resilient payment systems from a Staff Developer working on Shopify’s payment infrastructure.

Our journey building a message classification model for Shopify Inbox with the data-centric approach, from defining our classification taxonomy to training our annotators on labeling.

Ebun breaks down how she used a not-so-great performance review to set clear development goals for herself and increase alignment with her lead, ultimately leading to a successful next review.

The Spin infrastructure team works hard at improving the stability of the system. In February 2022 we moved to Container Optimized OS (COS), the Google maintained operating system for their Kubernetes Engine SaaS offering. A month later we turned on multi-cluster to allow for increased scalability as more users came on board. Recently, we’ve increased default resources allotted to instances dramat

This guide pulls back the curtain on what to expect during the interview process for technical roles at Shopify and sets you up for success with tips for each stage.

Shopify and Rails have grown up together. Both were in their infancy in 2004, and our CEO (Tobi) was one of the first contributors and a member of Rails Core. Shopify was built on top of Rails, and our engineering culture is rooted in the Rails Doctrine, from developer happiness to the omakase menu, sharp knives, and majestic monoliths. We embody the doctrine pillars. Shopify's success is due, in

You may have read that Shopify has built an in-house cloud development platform named Spin. In that post, we covered the history of the platform and how it powers our everyday work. In this post, we’ll take a deeper dive into one specific aspect of Spin: Isospin, Shopify’s systemd-based tooling that forms the core of how we run applications within Spin. The initial implementation of Spin used the

A 90-day plan to help managers develop a way of working remotely and gain mastery over the tools and skills that build trust in the new environment.

To address the new RBI framework for recurring payments in Shopify’s billing platform, we worked with a local payment provider that could accommodate both card-based payments as well as a local payment method called Unified Payments Interface (UPI).

What’s it really like for new engineering leaders at Shopify? I asked Avanti Ketkar and Maria Robledo, who both joined the team this past year, to spill the tea about their first few months. They share their perspectives on how the onboarding experience helped them build context, relationships, and trust.

At Shopify, we recognize the positive impact data-informed decisions have on the growth of a business. But we also recognize that data exploration is gated to those without a data science or coding background. To make it easier for our merchants to inform their decisions with data, we built an accessible, commerce-focused querying language. We call it ShopifyQL. ShopifyQL enables Shopify Plus merc

We’ve been building Hydrogen for more than a year. Here’s a look behind the scenes at how we did it, what we learned from making big bets, and what it was like building a new framework from the ground up on experimental technology.

Zack Deveau, Senior Application Security Engineer at Shopify, shares the details behind a recent contribution to the Rails library, inspired by a bug bounty report we received. He'll go over the report and its root cause, how we fixed it in our system, and how we took it a step further to make Rails more secure by upda

Shopify Party is a browser-based internal tool that we built to make our virtual hangouts more fun. With Shopify’s move to remote, we wanted to explore how to give people a break from video fatigue and create a new space designed for social interaction. Here's how we built it.

We asked our data scientists and engineers to curate a list of the top upcoming data conferences for 2022. Whether you’re looking for a virtual conference or in-person learning, we’ve got something that works for everyone.

How we reinvented our developer environment by going cloud native to keep up with increasing complexity and exponential growth.

A tutorial to create the basic form design that's meant to introduction to React including additional logic to emulate a fully functional React form.

An in-depth look at threads vs processes in Ruby web applications, and when you should use each.

Ruby is one of the few programming languages that get equality right. I often play around with other languages, but keep coming back to Ruby. This is because Ruby’s implementation of equality is so nice. Let's walk through all forms of equality in Ruby and how to implement them.

The Rails community gathered in Portland for the first in-person RailsConf in two years. Shopify developers recap a few highlights from the three-day event.

Shopify shares some of the lessons we learned and solutions we built in order to run Airflow at scale.

Shopify’s now on a journey of mindfully unraveling these default office behaviors and replacing them with remote-focused ways of working.

Explore how Shopify built a data pipeline using double entry transition tables to answer the question: how many Shopify merchants are using Shopify Balance?

Shopify is investing on Ruby on Rails at scale by funding high-profile academics to focus their work towards Ruby and the needs of the Ruby community.

An overview of Shopify Flow's new orchestration language Maestro, highlighting its design and implementation as well as how neatly it integrates with and addresses the requirements.

In this post, I want to give a nuanced perspective on our experience porting YJIT from C to Rust. I'll talk about the positives, but also discuss the things that we found challenging or suboptimal in our experience.

We asked our tech talent which podcasts they’re tuning into and curated a list of their favorites. We think you’ll find something on this list you’ll enjoy.

React Native Skia, an open source project, provides a set of powerful 2D drawing primitives that are consistent across iOS, Android, and the Web.

Data science is a balancing act between math, science, art, and communication. In this article, learn how to tell an effective data story illustrated with examples from Shopify.

Companies organize and automate their internal processes with a multitude of business systems, Shopify, Salesforce, HubSpot, Airtable, Netsuite, Google Sheets and many more. Since companies function as a whole, these systems need to be able to talk to one another.

How Shopify built the one network edge to front our services and systems.

An informal look at what makes encoding-aware strings in Ruby functional and performant, providing insight into all the wonderful things the Ruby VM does.

In February, we released the second version of Marketplace Kit: a collection of boilerplate app code and documentation allowing third-party developers to integrate with Shopify, build selected commerce features and launch a world-class marketplace in any channel.

Merlin, Shopify’s machine learning platform that can handle different (often conflicting) requirements, inputs, data types, dependencies, and integrations.

Hydrogen is a framework that combines React and Vite for creating custom storefronts on Shopify. It maximizes performance for end-users and provides a best-in-class developer experience for you and your team. Since it focuses on evergreen browsers, Hydrogen can leverage modern capabilities, best practices, and the latest tooling in web development to bring the future of ecommerce closer. Creating

Shopify software engineers, developers and development managers name the books they think everyone in tech and engineering should read.

"I just want you to feel it" - Jay Prince, from the song Feel It I use the word feeling a lot when working on animations and gestures. For example, animations or gestures sometimes feel right or wrong.

If you’re a data scientist on a product team, much of your work involves getting a product ready for release. You may conduct exploratory data analyses to understand your product’s market, or build the data models and pipelines needed to power a new product feature, or design a machine learning model.

In 2022, Shopify's Bug Bounty Program is doubling the maximum reward to $100,000 and moving key services into the highest severity level.

Large applications are often a mix of code your team has written and third-party applications your team needs to manage. These third-party applications could be things like AWS or Docker. In my team’s case, it’s Twilio TaskRouter. The configuration of these services may not change as often as your app code does, but when it does, the process is fraught with the potential for errors. This is becaus

At Shopify, we tell a lot of stories through data visualization. This is the driving force behind business decisions—not only for our merchants, but also for teams within Shopify. With more than 10,000 Shopify employees, though, it is only natural that different teams started using different tools to display data.

Not every feature of open-source template language Liquid has been ported over to Liquid-C, such as the If tag and For Tag. I started to look into how to compile Liquid’s If tag from Liquid-C, a process that would improve parsing time.

We’ve spent a lot of time on performance tuning our Apache Flink application. We’ll walk you through key lessons for optimizing large stateful Apache Flink applications

At Shopify we run more than 170,000 tests in our core monolith. Naturally, we're constantly exploring ways to make this faster, and the Test Infrastructure team analyzed the feasibility of introducing a test budget: a fixed amount of time for tests to run. The goal is to speed up the continuous integration (CI) test running phase by accepting more risk. To achieve that goal we used prioritization

V8 is Google’s open source high-performance JavaScript and WebAssembly engine written in C++. v8go is a library written in Go and C++ allowing users to execute JavaScript from Go using V8 isolates. Using Cgo bindings allows us to run JavaScript in Go at native performance. The v8go library, developed by Roger Chapman, aims to provide an idiomatic way for Go developers to interface with V8. As it t