A Year in Engineering: Ten Posts That Defined 2021
Shopify engineers publish a lot of work: deep technical write-ups, tutorials, and production war stories pulled from projects that span the company. Here are ten posts from 2021 worth revisiting, plus a few older pieces that still get steady traffic.
Building From Scratch and Building Blocks
Maple Ong, Senior Developer, wrote the most-read post of the year with a tutorial that starts from an unusual premise: ignore Rails entirely. The post walks through building a web application using only Ruby's standard libraries. Ong argues the exercise is not just educational but a rite of passage for Ruby and Rails developers.
Shopify Unite generated plenty of conversation around Hydrogen, the React-based framework for building custom storefronts. Ilya Grigorik, Principal Engineer, took readers behind the scenes to show how Hydrogen is built and optimized for personalized, contextual commerce. Hydrogen remains in developer preview, available for experimentation.
Compiler Work That Reached Upstream
One of the year's most impactful projects was YJIT, a new just-in-time compiler built inside CRuby. Led by Staff Engineer Maxime Chevalier-Boisvert, the small team's effort was merged upstream and shipped as part of the Ruby 3.1.0 release. The post covers the project's significance for both Shopify and the wider Ruby community. For further reading, Noah Gibbs published two additional YJIT posts covering hands-on usage and a benchmarking harness story.
Data Science and Infrastructure at Scale
Cody Mazza-Anthony, Data Scientist, outlined a five-step framework for exploratory data analysis (EDA) designed to answer business questions. Key takeaways include watching for missing values, describing and categorizing features, visualizing distributions, noting relationships between features, and identifying outliers early.
Yi Qing Sim, Senior Production Engineer, detailed how the Database Platform team executed the most recent MySQL upgrade at Shopify. Her write-up covers roadblocks encountered during rollback testing, internal tooling built to aid fleet upgrades and scaling, and guidelines for approaching future upgrades.
Doug Turnbull, Senior Staff Engineer, authored a post on using Apache Beam for search at Shopify. Turnbull is known for the book Relevant Search, contributions to AI-Powered Search, and relevance tooling like Splainer, Quepid, and the Elasticsearch Learning to Rank plugin.
APIs, App Development, and Developer Experience
A three-part GraphQL series by Raymond Chung, Technical Educator, became one of the year's top reads. The hands-on tutorial covers GraphQL fundamentals and contrasts the approach with REST.
Christian Bruckmayer, Senior Production Engineer, shared how the Test Infrastructure team improved Shopify's CI systems. The work reduced the p95 for the core monolith from 45 minutes to 18, letting developers spend less time waiting on builds.
Guilherme Vieira, Senior Developer on the API Patterns team, explained Shopify's rate-limiting system for the GraphQL Admin API. The post explores the limitations of common REST API methods and describes query cost calculations that adapt to client data needs while keeping server load predictable.
Sebastian Ekström, Senior Developer on the Shop team, recounted being first to build an App Clip in React Native, a project surfaced to millions of daily users. Ekström noted the project began with many unknowns and emphasized the value of a developer with native iOS experience, given App Clips' reliance on Apple's tooling.
Still Popular: Older Posts
A few posts from previous years continue to draw significant readership and deserved a spot on the list:
- Building a Data Table Component in React
- Under Deconstruction: The State of Shopify's Monolith
- How to Write Fast Code in Ruby on Rails
