All-Rust Turborepo: Port Complete
Turborepo has finished migrating from Go to Rust. The all-Rust version of turbo is now shipping, completing a 15-month effort that moved roughly 70,000 lines of code. The switch lays the groundwork for performance gains, stability improvements, and new feature development.
From the "Go sandwich" to pure Rust
The migration relied on an incremental porting strategy nicknamed the "Go sandwich," where Rust components were embedded within the existing Go codebase with help from Zig. That approach allowed individual pieces—such as file hashing, lockfile analysis, and cache signature verification—to be ported and tested independently.
The sandwich had limits, though. Mixing asynchronous, multi-threaded code across languages proved unworkable. With most of the code already in Rust, the team decided the sandwich had served its purpose. They built what they called the "run outline": an all-Rust version of the run command with most functionality stubbed out, gated behind a feature flag for internal testing.
A better package graph
The first major component added to the run outline was the package graph—a data structure mapping all packages in a monorepo, where nodes are packages and edges are dependency relationships. This lets Turborepo understand the repository structure and avoid duplicate work.
The Rust port improved the graph's type safety. In Go, the workspace root (where the lockfile lives) was designated with a magic string (//), and package names were plain strings, requiring ad-hoc checks. Rust models this with an enum, allowing the package name to be either the root package or a named package:
enum PackageName {
Root,
Other(String)
}
This eliminates an entire class of errors—the compiler forces developers to handle the workspace root case explicitly wherever package names are used.
The prune command
With the package graph in place, the team ported prune, which removes everything from a repository except a single package and its dependencies. Running turbo prune web-app generates an out folder containing only the code and dependencies for that application—particularly valuable for building small Docker images. The port was relatively straightforward since much of the lockfile code had already been ported under the sandwich approach and connected to the existing Go code with minimal iteration.
Hash stability during the transition
Turborepo's cache relies on hashing to determine whether a task has already been run. At the global level, a hash captures changes affecting the entire repository. At the task level, many individual hashes are calculated for specific tasks, each incorporating the global hash. These task hashes act as keys to store task outputs in tar files, both on the local filesystem and in Vercel Remote Cache. When a task hash matches a cached entry, outputs are restored in milliseconds.
The team prioritized hash stability across the language switch—users should not miss cache hits simply because the tool's implementation language changed. They chose Capnproto, a cross-platform, cross-language serialization format defined byte-for-byte, for both Rust and Go. CI was configured to run tasks on both code paths and fail if the hashes ever diverged.
Finding bugs through comparison
The hash comparison immediately surfaced discrepancies. Some were minor, such as incorrect handling of null values in hash inputs. Others were more foundational, involving change detection for packages and proper handling of the --filter flag. These would have been difficult to find through other means, validating the dual-path testing approach.
Dogfooding to release
With enough code ported, the team ran their own CI behind the --experimental-rust-codepath flag, burning down integration test failures until the Rust path matched the Go version exactly—line for line and output for output. After passing all tests, a canary version was published and used in Vercel's internal monorepo. Following 72 hours without reported errors, Turborepo 1.11 shipped.
Lessons learned
Looking back, the team identifies several strategic adjustments they would make:
- Pick one serialization format early. The journey went from JSON to Protocol Buffers and finally to Capnproto. The team realized Capnproto would have been the right choice from the start.
- Ship porting strategies faster. Delays in releasing the Rust shim and Go sandwich meant ported code sometimes couldn't ship to users due to release management bugs. A minimal early release of each strategy would have avoided this.
- Refactor the Go code before porting. The assumption that refactoring could happen alongside the port proved wrong. Cleanup and testing improvements beforehand would have speeded the migration.
- Fully specify core behaviors. Ambiguity around globbing, file watching, and hashing meant the team often couldn't tell whether behaviors in the Go code were intentional or accidental. The Rust port codified undocumented behaviors, and Windows testing gaps were also revealed.
What's next
The all-Rust foundation opens up new possibilities within Turborepo and Turbopack, including tighter integration with the Vercel Developer Experience Platform. Recent additions like Conformance and Code Owners aim to help engineering organizations ship higher quality code as their codebases scale.
