Why Business Systems Need a Different Integration Approach
Modern companies run on a patchwork of specialized business systems—Salesforce, HubSpot, Airtable, NetSuite, Google Sheets, and countless others. These tools are the modular successors to the monolithic enterprise software of the past, and each one targets a specific domain or niche like tax, procurement, or shipment tracking. The trade-off of this modularization is that the systems still need to interoperate, and that burden often falls on the companies using them rather than on the vendors building them.
Because these systems are built on different stacks with different interface styles—from SOAP to FTP to GraphQL—their integration capabilities vary wildly. Some ship with full integration platforms, others with nothing at all. Left to their own devices, teams fill the gap with a hodgepodge of ad hoc scripts running on laptops, manual CSV uploads, third-party add-ons, and whatever out-of-the-box connectors happen to exist. This works at small scale, but as a company grows and the number of point-to-point connections multiplies, the resulting web of integrations becomes unmanageable.
Evaluating an Off-the-Shelf IPaaS
A natural answer is to adopt an Integration Platform as a Service (IPaaS) like Mulesoft, Dell Boomi, or Zapier. The idea of centralizing all integration work in a single, consistent tool is appealing. Shopify took this route first, creating a small team of integration developers to build on Mulesoft, but the evaluation surfaced two significant drawbacks.
Outside the Company's Engineering Gravity
Shopify has thousands of engineers, supported by deep infrastructure, dedicated security teams, internal packages, and a culture of shared best practices. That support system is built around a deliberately narrow set of technologies—Ruby, React, and Go. Mulesoft, by contrast, is a proprietary platform that relies on XML configuration for the Java virtual machine. Building integrations on it meant operating entirely outside Shopify's engineering ecosystem, losing access to internal tooling, training, and peer support.
Blind Spots for Internal Services
Mulesoft's managed cloud runtime is a clear advantage for most users, since it removes infrastructure concerns. But Shopify runs many internal services—like shipment tracking—and infrastructure such as Kafka within its own cloud for security reasons. Integrating those internal systems through Mulesoft would have required the small team to build and maintain its own hosting infrastructure on Shopify's cloud, undercutting the platform's primary benefit.
What initially looked like a cost-saving solution turned out to have more drawbacks than building directly on Shopify's own stack.
A Rails-Based Integration Platform
Shopify's standard for backend development is Ruby on Rails, unless performance demands Go. The decision to build an internal integration platform on Rails follows naturally from that. The harder question is how to use Rails for integration work, since the technical demands differ meaningfully from standard application development.
Keep Logic Minimal
Most applications are authoritative for their own domain: they define the data they accept and validate it. An integration is the opposite—it holds no source of truth. Any validation it introduces is at best a duplicate of the target system's logic, and at worst a source of spurious errors. A required field in a Sorbet Struct that isn't required by the target API will cause records to fail in transit, even when the target would have accepted them.
Make Behavior Transparent
Business systems are highly configurable, and configuration changes often alter their APIs. Airtable, for example, uses column names as JSON keys in its API, so a simple rename in the UI breaks existing integrations. System admins need clear visibility into exactly what each integration sends and receives, both to prevent configuration mistakes and to diagnose them quickly when they occur.
Support Diverse Interfaces
The business systems a large company uses were built at different times by different teams with different design patterns. Integrations must handle FTP, REST, SOAP, JSON, XML, and GraphQL, sometimes all at once. A centralized integration platform must accommodate that full spectrum, not just modern HTTP APIs.
Apply Security Consistently
Integrations routinely handle sensitive data, including personally identifiable information (PII) and compensation details. The platform needs to enforce appropriate security controls across every connection it hosts.
Maximize Reuse
The most reliable and maintainable integrations are small and point-to-point. That design goal conflicts with the need for speed, however, since many point-to-point connections create duplicate code and infrastructure. Building integrations quickly depends on reusing components across connections as much as possible.
From Monolith to Engines: Structuring Integration Code
The first version of the platform kept each integration inside a namespace within the main app directory. That approach quickly became unmanageable: tracing an integration's code meant hunting through controllers, helpers and every other directory to see whether that integration's namespace was present. The obvious answer was to follow the architecture Shopify Core already uses, structuring the application as a modular monolith built from Rails Engines.
Engines fit well because integrations have clear boundaries. Each integration can live in its own engine with shared infrastructure provided by a common engine—without the overhead of packaging shared code into gems or duplicating it. The result is lower development and maintenance cost. It also keeps everything transparent: all code stays in one repository, and it is obvious which code belongs to which integration.
Actions Over Abstractions
Integration work is mostly business logic. Rails offers several abstractions for housing that logic, but most were designed for standalone applications. Adding them to an integration layer tends to obfuscate what the code is actually doing. The team adopted an "Action" pattern that was already being explored at Shopify: a class with a single public perform method, styled after Active Jobs, with no references to other Actions. This gives every data flow a single obvious entry point—look at one Action to understand what a particular integration does.
The trade-off is deliberate code duplication. When two integrations need similar logic, the team accepts copying it rather than coupling the integrations together. Given that integrations should operate independently, some duplication is preferable to shared mutable state.
Hashes as a Transparency Tool
Every integration touches at least two API abstractions of complex systems. Adding another layer of objects on top of those abstractions compounds complexity. Since most payloads are JSON—represented natively in Ruby as hashes—the team works with hashes directly.
The alternative is mapping: JSON to an object, object to another object, then back to JSON. Tracking a transformation through three mappings is far harder than reading one transform function on a hash. Hashes do open the door to typos in string keys, so the team turns frequently used keys into constants. But the transparency gain is worth the caution: the data flowing through the system and exactly how it changes stays visible.
Mocking at the Request Level
Shopify generally defaults to Mocha for mocking, but the integration platform instead uses WebMock. WebMock stubs at the HTTP request level, meaning tests show the URL, query parameters, headers and request body explicitly. That matches the level at which business system APIs are documented, making it straightforward to verify integrations against the docs and understand exactly what data is being sent.
There are exceptions. SOAP requests, for instance, are unreadable as giant XML strings in a test, so Mocha is the better tool there. Tests involving many sequential requests also get unwieldy with WebMock, and the team is building shared factories and prebuilt mocks to improve that experience.
Eliminating Data Storage
A key decision was that the platform would not be the source of truth for any business data. Business data flows in from external systems and passes through the application on its way to its destination. This keeps the system minimal, reduces complexity and limits the security surface—there is simply less data at rest to protect.
Documentation With the Code
Documentation has been a priority since day one: decisions, integrations, client libraries and instructions for building on the platform are all recorded. Early documentation lived in GitHub's wiki, but that made it hard to keep in sync with the code. Moving documentation into the git repository alongside the code means changes can be traced and documentation is updated in the same commit as the code it describes. Shopify's infrastructure makes adding a static site to a repo straightforward.
Leveraging Shopify's Ecosystem
Ruby's mature ecosystem means most business systems have well-maintained gems, so integrating with them often means adding a dependency and credentials rather than writing a client from scratch. Shopify's internal infrastructure provides the rest: dev runtime to stand up applications, established logging/metrics/tracing setups for observability, a developer pipeline for hiring, training teams for onboarding, and security standards to build against.
Current State and Direction
A year after work began, the platform runs 18 integrations. All legacy Mulesoft apps have been migrated off, and teams beyond the original two developers are building on the platform. The design favors quickly and securely building the simple integrations that make up most use cases while keeping maintenance minimal. Work continues on reducing complexity for the harder cases—particularly improving test mock management and the onboarding process—without sacrificing the transparency that makes the platform usable.
