Restricting a working directory to what matters

The git sparse-checkout command, introduced as experimental in Git 2.25.0, gives you a straightforward way to limit the files in your working directory to only those you need. That can make a meaningful difference in monorepos where teams work on independent microservices or client code but still want the shared history and collaboration benefits of a single repository.

If your repository structure is large enough that operations like git checkout and git status suffer, or the sheer number of files at the root makes the directory hard to manage, restricting the sparse-checkout to relevant directories is often the first step toward reclaiming performance.

Previously, using this feature meant manually editing a file inside the .git directory, changing configuration settings, and running an obscure plumbing command. Recovering from it was even more involved. The current command-line interface is far simpler: init --cone followed by set with the directories to include.

The init subcommand sets the required Git configuration and creates a sparse-checkout file populated with an initial set of patterns. Those patterns restrict the working directory to files at the root level only. The set subcommand then rewrites that file so Git matches the directories you specify. What might be surprising is that it also matches files immediately inside a parent directory of the one you chose, as well as files at the root of any ancestor. For instance, git sparse-checkout set A/B includes files such as A/B/C.txt, A/D.txt, and E.txt.

Disabling the feature is equally straightforward: git sparse-checkout disable returns your working directory to its full state.

A practical monorepo pattern

Consider a repository that holds the code for multiple independent services and clients. While all of these pieces need a shared version control history, each developer rarely touches more than a small fraction of the files. Even in a small example where the whole repository contains roughly 1,557 files, every Git command that writes to or inspects the working directory — like git pull — is paying attention to all of them. Multiply that by three orders of magnitude for a real monorepo and the problem is easy to see.

Running git sparse-checkout init --cone immediately reduces the working directory to just the root-level files. For an Android developer, there’s not much use in all of those directories at once. They can limit the scope to the project they care about:

git clone cd git sparse-checkout init --cone git sparse-checkout set client/android

The team building the identity service has a different need: they need their own code plus the shared code for other microservices. Their bootstrap flow can run the right set command for them:

git sparse-checkout set service/identity service/common

In larger organsations, the natural solution is a root-level helper script such as bootstrap.sh. It can encode the correct sparse-checkout sequence for each team. A team working on the web client might include all the microservice directories in their cone, even if their day-to-day work only touches the browser code:

git sparse-checkout set service web/browser

To inspect the current set of included directories at any time, the list subcommand is available. It returns the directories currently configured in the cone.

Checkout as you clone

There’s another way to take advantage of this: even if you don’t yet use sparse-checkout in an existing repository, you can start fresh by combining git clone --no-checkout with the sparse-checkout commands. That way you avoid ever producing a full working directory on disk.

Adopting this strategy soon reveals two things worth planning ahead. First, the build system needs to tolerate having a subset of directories; second, a helper tool that codifies the typical cones for engineering roles is almost necessary to keep the workflow efficient. A script at the root of the repository—something like bootstrap.sh—is the simplest implementation of that tool. Reducing the coupling between services also helps keep cones small, which directly contributes to faster Git commands.

With a reduced cone, Git commands such as git status, git add, and git checkout spend less time looking at files the developer never intended to touch, often making a large monorepo feel considerably smaller in practice.

This article continues in the next part with a deeper analysis of the “cone mode” behavior and how it makes the feature more performant than the older pattern-matching approach.

Cone mode trades pattern matching for hash lookups

sparse-checkout borrows its pattern syntax from .gitignore. After running git sparse-checkout init without a prior sparse-checkout file, the generated file contains two ordered patterns: one that matches everything at the root, and one that excludes anything inside a folder. Because the second rule is evaluated last, only files directly under the root survive the filter.

Patterns can grow elaborate, such as restricting the working tree to documentation and C headers:

!/*
**/docs/
**/*.h

The cost scales with the complexity. With N patterns and M repository files, every checkout evaluates each pattern against each file — O(N*M) comparisons. For large repositories, this fails quickly. In one test repository with over 1,000 folders to match against a root tree of three million files, leaving 700,000 files populated, the older matching algorithm took five full minutes to complete a checkout.

That quadratic blowup exists because full pattern matching is general purpose. Most users, however, describe their desired tree with simple prefix rules. Recognizing that, cone mode restricts the accepted pattern set so Git can use a different strategy.

How cone mode matches paths

When a user runs a command like git sparse-checkout set service web/browser, Git writes these patterns to the sparse-checkout file:

/*
!/*/
/service/
/web/
!/web/*/
/web/browser/

The ordered rules preserve files at the root, everything under service/, everything in web/browser/, and items immediately below web/. Other paths — client/, say, or web/editor/ — fall out.

Cone mode can recognize only two classes of patterns when parsing the file:

  • Recursive: /<path>/
  • Parent: !/<path>/*/

As Git reads the patterns, it builds two hash sets. A recursive pattern adds its path to the recursive set; a parent pattern removes its path from the recursive set and adds it to the parent set. For a candidate file path like /A/B/C/D.txt, the path is included when:

  • A/B/C is in the parent set,
  • A/B/C is in the recursive set,
  • A/B is in the recursive set, or
  • A is in the recursive set.

Each file requires checking at most as many ancestors as its folder depth d, bringing matching down to O(M + N*d). Files are evaluated in sorted order, which helps further: when a directory beginning matches a recursive rule exactly, Git auto-includes the entire subtree without checking deeper; likewise for a directory outside the cone, which is auto-excluded. Those shortcuts move typical cases closer to O(M + N).

The illustration below shows the slim number of hash lookups required for the web developer’s sparse definition compared to the dense lattice of full pattern matching:

In the test repository that previously took five minutes under the general matcher, cone mode finishes in under one second.

Combining with partial clone

Pairing sparse-checkout with the partial clone feature (--filter=blob:none) means Git transfers only the blobs needed for the selected directories rather than every reachable object. Using the example repository with that flag:

$ git clone --filter=blob:none --no-checkout https://github.com/derrickstolee/sparse-checkout-example
Cloning into 'sparse-checkout-example'...
Receiving objects: 100% (373/373), 75.98 KiB | 2.71 MiB/s, done.
Resolving deltas: 100% (23/23), done.
 
$ cd sparse-checkout-example/

$ git sparse-checkout set --cone
 
$ git checkout main
remote: Enumerating objects: 2, done.
remote: Counting objects: 100% (2/2), done.
remote: Compressing objects: 100% (2/2), done.
remote: Total 3 (delta 0), reused 1 (delta 0), pack-reused 1
Receiving objects: 100% (3/3), 1.41 KiB | 1.41 MiB/s, done.
Already on 'main'
Your branch is up to date with 'origin/main'.

$ git sparse-checkout set client/android
remote: Enumerating objects: 3, done.
remote: Counting objects: 100% (3/3), done.
remote: Compressing objects: 100% (3/3), done.
remote: Total 26 (delta 0), reused 1 (delta 0), pack-reused 23
Receiving objects: 100% (26/26), 985.91 KiB | 13.69 MiB/s, done.

The savings show in object counts. Cloning the complete repository required 373 objects instead of the roughly 2,000 of an unfiltered clone. Initial sparse-checkout fetched only three objects for root files, and populating the client/android directory added 26 more.

Roadmap and availability

Partial clone is still being evaluated at GitHub, and the feature is enabled only on a select set of repositories while it stabilizes. Work on sparse-checkout itself continues; planned updates include a smoother clone-and-sparse onboarding flow, add and remove subcommands to tweak the cone incrementally, a stats subcommand to compare your cone’s size against the full working tree, and allowing git sparse-checkout set to run on a dirty working tree when the update will not disturb existing changes. Feedback and bug reports go to the Git mailing list.