Package Management in Go

Go’s Dependency Handling, Explained

Go’s approach to package management diverges sharply from the mainstream. Most actively developed language ecosystems—Ruby with RubyGems, Node with NPM, the JVM with Maven, Rust with Cargo—rely on central repositories paired with a checked-in recipe that describes how to rebuild a project’s dependency tree. Go’s system works differently, and for developers coming from those other languages, the conventions can be disorienting.

One of the most surprising recommendations comes straight from the Go FAQ. It suggests that maintainers of public packages should fork their project if they need to introduce a backwards-incompatible change:

Packages intended for public use should try to maintain backwards compatibility as they evolve. The Go 1 compatibility guidelines are a good reference here: don’t remove exported names, encourage tagged composite literals, and so on. If different functionality is required, add a new name instead of changing an old one. If a complete break is required, create a new package with a new import path.

Having to fork an entire project just to change an API seems extreme. Yet much of the Go literature is apologetic about this design without ever addressing its real trade-offs. To make sense of it, it helps to understand a few core principles that underpin Go’s import and packaging model.

The Workspace Model

Everything in Go’s packaging story revolves around the workspace: any directory pointed to by $GOPATH, with a standard layout:

bin/
pkg/
src/

Your project lives inside a workspace, typically at a path like $GOPATH/src/github.com/brandur/heroku-agent. When you write an import statement, you’re telling the compiler to resolve that path within the workspace, even if the import path looks like it points to a remote host:

import (
    "github.com/brandur/my-dependency"
)

The go get tool fetches dependencies from supported providers and stores them in the workspace so they can be compiled.

Version Control as a Package Registry

It’s often said that Go’s package system is “built on top of distributed version control,” which sounds like an extra layer of robustness over traditional registries. In practice, though, the dominant convention is to host packages on centralized services like GitHub—no more distributed or fault-tolerant than RubyGems. Running go get isn’t fundamentally different from a bundle install --local.

This design doesn’t remove central points of failure entirely, but it does reduce the community’s reliance on a single, expensive-to-maintain central repository. RubyGems and NPM have long depended on largely charitable sponsorship for their existence; that arrangement may not hold forever. Go sidesteps this by letting support for any DVCS provider be added or removed as needed. The only hard requirement is the ability to check out source code into a known location.

Full Import Paths, No Relative Imports

Another feature that surprises newcomers is that relative imports are strongly discouraged and only permitted outside a workspace. For external dependencies that’s reasonable, but even when structuring a project into subpackages, the convention is still to fully qualify every import path:

import (
    "github.com/goraft/raft/protobuf"
)

That syntax leaves questions: does an import reference the master branch on GitHub? Do you have to push subpackage changes upstream before using them from your main package? The former, yes; the latter, no. Again, the workspace is what makes this tenable: import statements always resolve to code already inside the workspace, even if it has diverged from what’s in the origin’s master branch.

Dropping relative imports has clear benefits. Building any package no longer requires understanding a local file hierarchy, which makes paths easier to reason about across projects. It also enforces a consistent convention—you never have to untangle an idiosyncratic project structure to locate a dependency; its position is always given by its import path.

That explicitness is also a boon for open source. When you need to inspect or alter one of your dependencies, you know exactly where to find it.

Vendoring as a Locking Strategy

Importing from master is fine for small experiments, but serious software eventually needs reproducible deployments. The Go FAQ again points to an atypical solution: lock dependencies by copying them into your project.

If you’re using an externally supplied package and worry that it might change in unexpected ways, the simplest solution is to copy it to your local repository. (This is the approach Google takes internally.) Store the copy under a new import path that identifies it as a local copy. For example, you might copy “original.com/pkg” to “you.com/external/original.com/pkg”. Keith Rarick’s goven is one tool to help automate this process.

Vendoring brings its own set of problems. Repositories become cluttered with third-party source, and diffs can be painful to review. But it also makes builds independent of external service availability and eliminates the interdependency-version conflicts that plague other ecosystems.

Simplicity as a Feature

The rationale behind Go’s import system aligns with the principle of simplicity that shapes the rest of the language. Dependencies are resolved through the same version control tools you already use for your own source. Packages live in the same $GOPATH directory structure as your project. There is no versioning—the compiler simply consumes whatever code is on disk. All of it can be resolved and built with the Go toolchain alone, without any extra external tooling.

Reasonable people can argue about the merits of Go’s approach versus others, but it’s hard to dismiss its minimalism gently. After spending hours loosening version constraints in a large Ruby app’s dependencies, or struggling with Maven to resolve a common HTTP library, using a version control system you can reason about easily becomes an attractive alternative, indeed.