Introducing arborium, a tree-sitter distribution

tree-sitter-highlight provides a callback mechanism for handling injections, but wiring that up correctly, with the right grammar dependencies, is left to the consumer. That's a significant amount of work when you repeat it across dozens of languages.

The alternative — loading a full language server for semantic highlighting — is technically superior but practically disqualifying. An LSP needs all of the source code, all of its dependencies, and the entire standard library sysroot loaded into memory before it can produce annotations. For batch documentation generation, that cost is prohibitive.

A purpose-built distribution

For the past six years, maintaining a private collection of curated grammars has been necessary for any project that needs reliable, offline syntax highlighting across multiple languages. Grammars from the official tree-sitter-grammars organization are high quality and current for mainstream languages like Rust and C++. Less popular languages are a different story: you might find a grammar that looks workable, but was written against an older tree-sitter API, or needs cleanup, regeneration, and removal of rules that cause compilation times to explode.

After collecting 18 grammars and finding the same packaging gaps across them, the solution is to stop solving this per-project and publish a single distribution that bundles the full toolchain: grammars that build cleanly against a modern tree-sitter, plus matching highlights and injections queries for every included language.

That distribution is arborium, now available as a public project.

A crate distribution with 96 grammars built in

Arborium started with a simple observation: people want syntax highlighting for a lot more than Rust. For the 96 languages that were requested, the project maintainers hunted down the best available grammar for each one, vendored it, patched it up, verified the highlight queries, made sure license and attribution were included in the redistribution, and wired it all into a cargo feature flag on the main arborium crate.

The dependency graph goes deeper than just the language itself. If you pull in Svelte, for example, the crate also brings along everything needed to highlight a Svelte component completely: HTML, CSS, and JavaScript.

Like the original tree-sitter crates, these grammar crates can't do much on their own. Arborium provides simple high-level interfaces for highlighting code through the main crate:

This approach glosses over the finer points of tree-sitter's incremental parsing and highlighting, but the more complicated APIs are available if you need them. Everything from the theme — many of which ship built in — to the style of the HTML output is configurable. The default output is modern, compact, and widely supported:

For those who prefer the long-winded format and are confident Brotli compression will make up for it, there's an alternative:

Terminal users can get escape codes in the output too, optionally with a background color, margin, padding, and border to make the highlighted code stand out:

Most importantly, the Rust crates are arranged so they compile through cargo to the wasm32-unknown-unknown target. That was the tricky part — it requires providing just enough libc symbols to keep the grammars happy.

The tree-sitter playground you can get from tree-sitter build --wasm and tree-sitter playground targets wasm32-wasi, which is slightly different. At the end of the day, someone has to supply the system functions, and in Arborium's case, that someone is the project itself. Most of the functions needed are simple (isupper, islower, and so on), with malloc, free, and friends provided by dlmalloc.

Because every one of those crates compiles with a Rust toolchain (invoking a C toolchain along the way) to wasm32-unknown-unknown, they can run in a browser with a little glue.

Angle 1: a script to include

Right now, publishing a crate with documentation that needs highlighting for languages beyond Rust only takes a few steps, documented at arborium.bearcove.eu:

• Create an HTML file in your repository
• Add metadata to your Cargo.toml so the docs.rs build process picks it up

The arborium_docsrs_demo page shows this in action, with sources in the arborium repository. Arborium even detects when it's running on docs.rs and matches the active theme responsively — docs.rs light, docs.rs dark, or Ayu depending on the page. Those themes don't match the maintainer's personal taste, but consistency wins.

This approach works today, with zero extra work for the Rust docs team. They don't have to touch rustdoc, their build pipeline, or their infrastructure. It's an escape hatch that just works, and people have used it for everything from integrating KaTeX to rendering diagrams to front-end tricks.

But the same approach is also a security disaster waiting to happen. It requires JavaScript and WebAssembly, forcing users to download large grammar bundles — sometimes hundreds of kilobytes — just to highlight small code blocks. More critically, it lets third-party JavaScript run in the main context of the docs.rs page. There isn't much to steal there today besides a favorite theme, but that could change. The docs.rs team knows this, and letting arbitrary packages inject scripts is bad practice regardless.

The risk scenario: everyone adopts Arborium as the standard way to highlight docs.rs pages. Years later, a malicious version of the arborium package gets published to NPM and instantly reaches millions of people. Pinning to a specific version would prevent that, but also block important updates. Ideally, all JavaScript on docs.rs pages comes from the docs team itself, limiting the blast radius substantially.

Angle 2: highlights in rustdoc itself

Arborium is ultimately just a set of Rust crates containing C code, both of which are extremely portable. There's no dynamic linking, no plugin folder, no asynchronous loading. Just grammars and code that highlights things. A pull request against rustdoc to add highlighting for other languages follows naturally:

At +537 −11 lines, the PR is small, but it pulls in literal millions of lines of C code from tree-sitter-generated parsers. That raises the question of which grammars to bundle, which is a decision the maintainer is glad not to be making alone.

The comparison above shows a custom rustdoc with all 96 languages compiled in versus the main-branch rustdoc. Binary sizes will inevitably come up in discussion, which leads to the third angle.

Angle 3: highlighting in the docs.rs backend

If bundling hundreds of grammars into every rustdoc build isn't feasible, the alternative is to do the highlighting on the docs.rs server side. arborium-rustdoc is a post-processor specific to rustdoc — it detects code blocks in HTML files and highlights them, then patches the main CSS file to append its styles.

Tested against all dependencies of the facet monorepo, the ~900MB doc folder grew by only 24KB. It's fast, can be built with support for all languages, and avoids the security and bundle size problems of the other two approaches. It can even be sandboxed.

The build and what comes next

The hardest part of the project was the CI setup. Building a small package with GitHub Actions is bearable; orchestrating 2×96 builds plus supporting packages and publishing with provenance to two platforms is punishing. The project relied on donated, beefy CI runners from Depot.dev, without which the maintainer says they would have bailed early. Even with those, the plugin jobs were split into ten tree-themed groups:

Because any CI failure is punishing, most logic was kept out of YAML and put into a cargo-xtask. That's not just for progress bars and nerd font icons — it's also about ensuring every WebAssembly artifact can be loaded in a browser by parsing the bundle and checking its imports with walrus, rather than piping wasm-objdump -x into grep. Blake3 hashes avoid recomputing inputs, chosen partly because the name sounds cool.

Arborium is released under Apache2+MIT, and the hope is that it brings accurate syntax highlighting to the web the way code editors suddenly got better at it years ago. The full details live on the arborium website.

For docs.rs specifically, the realistic recommendation is arborium-rustdoc as a post-processing step: it's fast, supports all languages at build time, and has none of the security or bundle size implications of client-side options.