Home/Shopify Engineering
Source

Shopify Engineering

317 articles from Shopify Engineering.

Career & Teams — How I Define My Boundaries to Prevent Burnout - Shopify

How I Define My Boundaries to Prevent Burnout - Shopify

There’s a way to build a model for your life and the other demands on you. It doesn’t have to be 100 hours a week. It might not even be 40 hours a week. Whatever sustainable model you come up with, prioritize fiercely and communicate expectations accordingly. I’ve worked this way for three decades through different phases of my life. The hours have changed, but the basic model and principles remai

SEShopify Engineering·May 6, 2021Career & Teams
Engineering — A Five-Step Guide for Conducting Exploratory Data Analysis - Shopify

A Five-Step Guide for Conducting Exploratory Data Analysis - Shopify

Have you ever been handed a dataset and then been asked to describe it? When I was first starting out in data science, this question confused me. My first thought was “What do you mean?” followed by “Can you be more specific?” The reality is that exploratory data analysis (EDA) is a critical tool in every data scientist’s kit, and the results are invaluable for answering important business questio

SEShopify Engineering·April 28, 2021Engineering
Databases — Dynamic ProxySQL Query Rules - Shopify

Dynamic ProxySQL Query Rules - Shopify

Resiliency is important at Shopify and although we have many tools to help us during incidents, some problems need to be fixed at the database level. At Shopify, we use ProxySQL to manage connections between applications and our MySQL databases

SEShopify Engineering·April 20, 2021Databases
Performance — How Shopify Dynamically Routes Storefront Traffic - Shopify

How Shopify Dynamically Routes Storefront Traffic - Shopify

In 2019 we set out to rewrite the Shopify storefront implementation. Our goal was to make it faster. We talked about the strategies we used to achieve that goal in a previous post about optimizing server-side rendering and implementing efficient caching. To build on that post, I’ll go into detail on how the Storefront Renderer team tweaked our load balancers to shift traffic between the legacy sto

SEShopify Engineering·April 9, 2021Performance
Security — Updates on Shopify's Bug Bounty Program 2021 - Shopify

Updates on Shopify's Bug Bounty Program 2021 - Shopify

For three years we, Shopify’s Application Security team, have set aside time to reflect on our bug bounty program and share recent insights. This past year has been quite a ride, as our program has been busier than ever! We’re excited to share what we have learned, and share some of the great things we have planned!

SEShopify Engineering·March 11, 2021Security
Databases — Read Consistency with Database Replicas - Shopify

Read Consistency with Database Replicas - Shopify

At Shopify, we’ve long used database replication for redundancy and failure recovery, but only fairly recently started to explore the potential of replicas as an alternative read-only data source for applications.

SEShopify Engineering·February 22, 2021Databases
SRE & Ops — Using Betas to Deploy New Features Safely - Shopify

Using Betas to Deploy New Features Safely - Shopify

For companies like Shopify that practice continuous deployment, our code is changing multiple times every day. We have to de-risk new features to ship safely and confidently without impacting the million+ merchants using our platform. Beta flags are one approach to feature development that gives us a number of notable advantages.

SEShopify Engineering·February 12, 2021SRE & Ops
Engineering — Technical Mentorship Reimagined: Time-bound and No Awkward Asks Necessary - Shopify

Technical Mentorship Reimagined: Time-bound and No Awkward Asks Necessary - Shopify

Authors: Sarah Naqvi and Steve Lounsbury Struggling with a concept and frantically trying to find the answers online? Are you thinking: I should just ping a teammate or Is this something I should already know? That’s fine, I’ll ask anyway. And then you do ask and get an answer and are feeling pretty darn proud of yourself and move forward. But wait, now I have about 25 follow-up questions. Sound f

SEShopify Engineering·February 10, 2021Engineering
Engineering — Using GraphQL for High-Performing Mobile Applications - Shopify

Using GraphQL for High-Performing Mobile Applications - Shopify

GraphQL is the syntax that describes data that a client asks from a server. The client, in this case, is a mobile application. GraphQL is usually compared with REST API, a common syntax that most mobile application developers use. We will share how GraphQL can solve some of the pain points of REST API in mobile application development and discuss tips and best practices that we learned at Shopify

SEShopify Engineering·January 22, 2021Engineering
Career & Teams — Apache Beam for Search: Getting Started by Hacking Time - Shopify

Apache Beam for Search: Getting Started by Hacking Time - Shopify

To create relevant search, processing clickstream data is key: you frequently want to promote search results that are being clicked on and purchased, and demote those things users don’t love. Typically search systems think of processing clickstream data as a batch job run over historical data, perhaps using a system like Spark. But on Shopify’s Discovery team, we ask the question: What if we could

SEShopify Engineering·January 8, 2021Career & Teams
Performance — Simplify, Batch, and Cache: How We Optimized Server-side Storefront Rendering - Shopify

Simplify, Batch, and Cache: How We Optimized Server-side Storefront Rendering - Shopify

On December 16, 2020 we held Shipit! presents: Performance Tips from the Storefront Renderer Team. A video for the event is now available for you to learn more about how the team optimized this Ruby application for the particular use case of serving storefront traffic. Click here to watch the video. By Celso Dantas and Maxime Vaillancourt In the previous post about our new storefront rendering eng

SEShopify Engineering·December 10, 2020Performance
SRE & Ops — Resiliency Planning for High-Traffic Events - Shopify

Resiliency Planning for High-Traffic Events - Shopify

On January 27, 2021 Shipit!, our monthly event series, presented Building a Culture of Resiliency at Shopify. Learn about creating and maintaining resiliency plans for large development teams, testing and tooling, developing incident strategies, and incorporating and improving feedback loops. The video is now available. Each year, Black Friday Cyber Monday weekend represents the peak of activity f

SEShopify Engineering·December 10, 2020SRE & Ops
SRE & Ops — The State of Ruby Static Typing at Shopify - Shopify

The State of Ruby Static Typing at Shopify - Shopify

Shopify changes a lot. We merge around 400 commits to the main branch daily and deploy a new version of our core monolith 40 times a day. The Monolith is also big: 37,000 Ruby files, 622,000 methods, more than 2,000,000 calls. At this scale with a dynamic language, even with the most rigorous review process and over 150,000 automated tests, it’s a challenge to ensure everything works properly. Dev

SEShopify Engineering·December 7, 2020SRE & Ops
Performance — A World Rendered Beautifully: The Making of the BFCM 3D Data Visualization - Shopify

A World Rendered Beautifully: The Making of the BFCM 3D Data Visualization - Shopify

By Mikko Haapoja and Stephan Leroux 2020 Black Friday Cyber Monday (BFCM) is over, and another BFCM Globe has shipped. We’re extremely proud of the globe, it focused on realism, performance, and the impact our merchants have on the world. The Black Friday Cyber Monday Live Map We knew we had a tall task in front of us this year, building something that could represent orders from our one million m

SEShopify Engineering·December 4, 2020Performance
Distributed — Capacity Planning at Scale - Shopify

Capacity Planning at Scale - Shopify

We cover our approaches to capacity planning, and how we rolled it out across the org and to dozens of teams. We’ll also share how we validated our capacity plans with scalability tests to make sure they work.

SEShopify Engineering·December 3, 2020Distributed
SRE & Ops — Vouching for Docker Images - Shopify

Vouching for Docker Images - Shopify

If you were using computers in the ‘90s and the early 2000s, you probably had the experience of installing a piece of software you downloaded from the internet, only to discover that someone put some nasty into it, and now you’re dragging your computer to IT to beg them to save your data. To remedy this, software developers started “signing” their software in a way that proved both who they were a

SEShopify Engineering·December 1, 2020SRE & Ops
Distributed — Adopting Sorbet at Scale - Shopify

Adopting Sorbet at Scale - Shopify

Presenting how Shopify got from no Sorbet in our monolith to almost full coverage in the span of a few months. I’ll explain the challenges we faced, the tools we built to solve them.

SEShopify Engineering·November 19, 2020Distributed
Databases — How to Introduce Composite Primary Keys in Rails - Shopify

How to Introduce Composite Primary Keys in Rails - Shopify

One line of code can 5x your Rails application performance. What if you could make a small change to your database design that would unlock massively more efficient data access? At Shopify, we dusted off some old database principles and did exactly that with the primary Rails application.

SEShopify Engineering·October 29, 2020Databases
Performance — How to Do an In-depth Liquid Render Analysis with Theme Inspector - Shopify

How to Do an In-depth Liquid Render Analysis with Theme Inspector - Shopify

Shopify’s Online Store provides greater flexibility to build themes representing the brand of a merchant’s online store. However, like with any programming language, one often writes code without being aware of all the performance impact it creates. Whether it be performance impact on Shopify’s servers or observable performance impact on the browser, ultimately, it’s the customers that experience

SEShopify Engineering·October 20, 2020Performance
Frontend — Tophatting in React Native - Shopify

Tophatting in React Native - Shopify

To provide a more convenient and painless experience, we built a tool enabling React Native developers to quickly load their peer’s work within seconds.

SEShopify Engineering·September 9, 2020Frontend
Frontend — 5 Ways to Improve Your React Native Styling Workflow - Shopify

5 Ways to Improve Your React Native Styling Workflow - Shopify

In April, we announced Shop, our digital shopping assistant that brings together the best features of Arrive and Shop Pay. The Shop app started from our React Native codebase for our previous package tracking app Arrive, with every screen receiving a complete visual overhaul to fit the new branding. While our product designers worked on introducing a whole new design system that would decide the l

SEShopify Engineering·September 2, 2020Frontend
Engineering — Building Reliable Mobile Applications - Shopify

Building Reliable Mobile Applications - Shopify

Merchants worldwide rely on Shopify's Point Of Sale (POS) app to operate their brick and mortar stores. Unlike many mobile apps, the POS app is mission-critical. Any downtime leads to long lineups, unhappy customers, and lost sales. The POS app must be exceptionally reliable, and any outages resolved quickly. Reliability engineering is a well-solved problem on the server-side. Back-end teams are a

SEShopify Engineering·August 14, 2020Engineering
Networking — Using DNS Traffic Management to Add Resiliency to Shopify’s Services - Shopify

Using DNS Traffic Management to Add Resiliency to Shopify’s Services - Shopify

If you are lacking understanding of what is DNS, traffic management, or why we would even use it, read Part 1: Introduction to DNS traffic management. Distributed systems are only as resilient as we build them to be. Domain Name System (DNS) traffic management (TM) is a well-used approach to do so. In this second part of the two-part series, we’re sharing Shopify’s DNS traffic management journey f

SEShopify Engineering·August 5, 2020Networking
Networking — An Introduction to DNS Traffic Management - Shopify

An Introduction to DNS Traffic Management - Shopify

Distributed systems are only as resilient as we build them to be. Domain Name System (DNS) traffic management is a well-used approach to do so. In this first part of a two-part series, we aim to give a broad overview of DNS and how it’s used for traffic management, as well as the different reasons why we want to use DNS traffic management. If you already have context on what is DNS, what is traffi

SEShopify Engineering·August 5, 2020Networking
Engineering — ShipIt! Presents: AR/VR at Shopify - Shopify

ShipIt! Presents: AR/VR at Shopify - Shopify

On June 20, 2020, ShipIt!, our monthly event series, presented AR/VR at Shopify. Daniel Beauchamp, Head of AR/VR at Shopify talked about how we’re using this increasingly ubiquitous technology at Shopify.

SEShopify Engineering·July 24, 2020Engineering
Engineering — How We’re Solving Data Discovery Challenges at Shopify - Shopify

How We’re Solving Data Discovery Challenges at Shopify - Shopify

Humans generate a lot of data. Every two days we create as much data as we did from the beginning of time until 2003! The International Data Corporation estimates the global datasphere totaled 33 zettabytes (one trillion gigabytes) in 2018. The estimate for 2025 is 175 ZBs, an increase of 430%. This growth is challenging organizations across all industries to rethink their data pipelines. The natu

SEShopify Engineering·July 21, 2020Engineering
Career & Teams — How We Built Size.link - Shopify

How We Built Size.link - Shopify

On July 20, 2020 we held ShipIt! Presents: AR/VR at Shopify. Daniel talked about what we’re doing with AR/VR at Shopify. The video of the event is available. Size.link is a free tool built by Shopify’s Augmented Reality (AR) team that lets anyone view the size of a product in the space around them using their smartphone camera. The tool came out of our April Hack Days focused on helping retailers

SEShopify Engineering·July 13, 2020Career & Teams
Distributed — Media at Scale: Callbacks vs pipelines - Shopify

Media at Scale: Callbacks vs pipelines - Shopify

Shopify recently launched the ability to add videos and 3d models to products as a native feature within the Admin section. Learn how we used pipelines to manage the long-term maintenance and understandability of the code.

SEShopify Engineering·July 9, 2020Distributed
Frontend — A First Look at Reanimated 2 - Shopify

A First Look at Reanimated 2 - Shopify

With Reanimated 2, the team at Software Mansion asked: “How would it look if there were no constraints when writing gestures and animations in React Native?” and started from there.

SEShopify Engineering·July 2, 2020Frontend
Distributed — Spark Joy by Running Fewer Tests - Shopify

Spark Joy by Running Fewer Tests - Shopify

Developers write tests to ensure correctness and allow future changes to be made safely. However, as the number of features grows, so does the number of tests. Tests are a double-edged sword. On one hand, well-written ones catch bugs and maintain a program’s stability, but as the code base grows, a high number of tests impedes scalability because they take a long time to run and increase the likel

SEShopify Engineering·June 11, 2020Distributed
Engineering — ShipIt! Presents: How Shopify Uses Nix - Shopify

ShipIt! Presents: How Shopify Uses Nix - Shopify

On May 25, 2020, ShipIt!, our monthly event series, presented How Shopify Uses Nix. Building upon on my What is Nix post, I show how we rebuilt our developer tooling using Nix, and show off some of the tooling we actually use at Shopify on a day-to-day basis. I wasn't able to answer all the questions during the event, so I've included answers to those ones below. Would runix interop well with lorr

SEShopify Engineering·May 29, 2020Engineering