Big Ruby Gems Depending on Small Ones: A Security Gap

RubyGems announced in June 2022 that multi-factor authentication (MFA) would be rolled out gradually to its users. The team behind this effort, Shopify's Ruby Dependency Security team, needed to decide who would be included in the first milestone. Their goal was to include the top 100 gems by downloads, but also to avoid frequent changes to that cohort.

To meet both criteria, they settled on a threshold of 180 million downloads per gem. Any gem crossing that number now requires MFA from its owners. But this design choice raises a question: what about the dependencies of those big gems?

A potential loophole

If a popular gem—one with more than 180 million downloads—depends on a smaller gem below that threshold, an attacker could compromise the smaller package and reach users of the larger one, bypassing the MFA requirement entirely. This seemed unlikely on the surface: surely a dependency, which gets downloaded every time its parent does, would have at least as many downloads as the parent itself.

An investigation was needed to find out if any exceptions existed.

Finding the big gems

The rubygems.org stats page displays the top 100 gems by downloads, but the last gem on page ten sits at 199 million downloads. That would leave out gems in the 180–199 million range, so the stats page alone wouldn't work. Instead, the investigation used the data dumps that rubygems.org publishes daily. These snapshots strip confidential data and can be loaded into a local copy of the rubygems.org database using a script from their repository.

Once the data was in place, the Rails console could run queries against it. The query that worked to get all gems above the 180 million threshold was:

Rubygem.joins(:gem_download).where(gem_download: {count: 180_000_000..}).map(&:name)

This returned 112 big gems.

Checking direct dependencies

Next, the investigation used the rubygems.org API, which returns both a gem's direct dependencies and its download count. By querying each big gem for its dependencies and then checking those dependencies' download counts, a pattern emerged: 13 of the 112 big gems had small gems as direct dependencies.

Two explanations covered most cases:

  1. Some small dependencies are newer than their parent gems and were added later by developers.
  2. Some gems ship with Ruby by default and therefore don't accumulate downloads through RubyGems. racc and rexml are examples.

The API only shows direct dependency relationships though. To get a complete picture of what actually gets installed when a big gem is installed, including transitive dependencies, the investigation turned to Bundler.

Following the full dependency chain

Bundler takes a Gemfile and generates a Gemfile.lock with the complete, resolved list of all needed gems and versions. By programmatically creating a Gemfile containing only one big gem at a time, running bundle lock, and reading the resulting lock file, the investigation could map the entire dependency tree for each big gem.

When the full dependency graph was considered, the results widened considerably: 24 of the 112 big gems rely on small gems somewhere in their install path. That is a substantial fraction of the top tier.

Visualizing the findings

Text dumps of these relationships were hard to read and hid important patterns. For instance, many big gems rely on racc—but is that direct or indirect? A graph would show this clearly.

The graph gem was used with a breadth-first search starting from all big gems, adding edges between gems and their dependencies, and highlighting small gems that sit below the MFA threshold.

The initial graph was cluttered with AWS gems. Removing those produced a cleaner view. The resulting graph tells several concise stories:

  • A cluster of gems revolves around rails, which acts as a clear keystone in Ruby's ecosystem.
  • activesupport receives the most incoming edges from other big gems—a genuinely active support.
  • racc, despite appearing as a small gem in many printouts, is only a direct dependency of nokogiri for the big gems examined. Other connections to racc are indirect.

For the final analysis, a directional graph data structure was implemented from scratch—finding all paths between two nodes wasn't available in any existing Ruby gem. A depth-first search generated a complete printout of every path from a big gem down to a small gem, showing all the routes an attacker could take.

Options for closing the gap

The investigation produced four distinct ways to answer the original question: a direct-dependency printout via the RubyGems API, a full sub-dependency printout via Bundler, the graphical view, and the printout with all dependency paths.

The 24 big gems that depend on small gems remain a policy question. One approach is to do nothing, since MFA will eventually be mandatory for everyone anyway. The other option is to require MFA on these specific gems early, acting as a blocklist to secure the top gems sooner. This would force only a small group of owners to enable MFA a few months ahead of schedule.

Either way, the discovery that large-scale, high-download packages can depend on much smaller ones with fewer protections is an important data point for supply chain security conversations.