Home/Engineering/Some notes on nix flakes

Engineering

Some notes on nix flakes

JE

Julia EvansJulia Evans

·November 11, 2023

Flakes: A Practical First Look

Nix has been my package manager of choice for about nine months now. For all of that time I ignored flakes, but the constant refrain that they're the best way to use Nix eventually wore me down. What follows are the basic examples I couldn't find anywhere else, plus the problems that nearly sent me back to nix-env -iA.

What a Flake Actually Is

Every explanation of flakes I found framed them in terms of other Nix concepts, which wasn't helpful. What finally clicked was an analogy to Docker container images. Like a Docker image, a flake can:

contain any software you want to install or compile
act as a self-contained dev environment with all dependencies set up
be shared via a flake.nix file that others can build identically

The comparison breaks down quickly, though. There are major differences:

A flake.nix plus flake.lock guarantees identical builds; a Dockerfile does not
Flakes run natively on macOS with no VM or Linux layer
Dependencies are shared efficiently between flakes, and flakes can pick and choose which parts of their dependencies to use
flake.nix is a program in the Nix language, not a list of shell commands
Isolation is done with dynamic linker and rpath tricks, not filesystem overlays, cgroups, or namespaces

What the two share is the core design goal: sharing a dev environment with a single configuration file.

Why Nix Works for Me

People praise Nix for being declarative, reproducible, and functional. My main motivation is simpler: Nix has a lot of pre-compiled binaries for macOS. More than Homebrew does, in my experience, which means fewer source builds. It also means I can build a five-year-old version of Hugo on a Mac without a fight.

Building a Single-Flake Setup

My existing setup was a Homebrew replacement: run nix-env -iA nixpkgs.whatever to install something, done. It worked fine, aside from occasionally breaking randomly. But I thought it would be tidy to have one flake.nix file listing every package I wanted, producing a single directory I could put in my PATH.

The practical advantages are modest: theoretically easier setup on a new machine, and uninstalling becomes deleting a line instead of remembering the right Nix command.

A First Attempt

Starting with Ruby and cowsay, I put together this flake.nix:

{
  inputs.nixpkgs.url = "github:NixOS/nixpkgs/nixpkgs-23.05-darwin";
  outputs = { self, nixpkgs }: {
    devShell.aarch64-darwin = nixpkgs.legacyPackages.aarch64-darwin.mkShell {
      buildInputs = with nixpkgs.legacyPackages.aarch64-darwin; [
        cowsay
        ruby
      ];
    };
  };
}

Here's what I understood about it at the time:

nixpkgs is the central package repository
aarch64-darwin is my architecture, critical for getting the right binary downloads
An "input" is a dependency; I get to choose which parts of it to use
The URL scheme is github:USER/REPO_NAME/TAG_OR_BRANCH_NAME
mkShell enables nix develop, which I abandoned shortly after
The output has to be named devShell.aarch64-darwin or nix develop refuses to run
self and legacyPackages remained mysteries

Then I tried to build it:

$ nix build
error: getting status of '/nix/store/w1v41cyqyx4d7q4g7c8nb50bp9dvjm29-source/flake.nix': No such file or directory

That error is inscrutable. Why does Nix think a path in /nix/store/ should exist?

Problem 1: Git Gets in the Way

The issue: Nix flakes have strange rules about Git.

If the current directory is not a Git repository, everything is fine
If you are in a Git repo and your files are git added, everything is fine
If you're in a Git repo and your flake.nix file is untracked — just created, for instance — Nix completely ignores it

This behavior is documented but easy to miss:

Note that any file that is not tracked by Git is invisible during Nix evaluation

The fix is simple: git add the file.

Enabling Flake Commands

To use nix build and related commands, you need to enable two experimental features: flakes and nix-command. I put experimental-features = nix-command flakes in ~/.config/nix/nix.conf. The alternative is prefixing every command with nix --extra-experimental-features "flakes nix-command".

With features enabled, nix develop gave me a working shell:

$ nix develop
grapefruit:nix bork$ cowsay hi
 ____
< hi >
 ----
        \   ^__^
         \  (oo)\_______
            (__)\       )\/\
                ||----w |

Inside that shell, the PATH was structured one way I didn't expect:

grapefruit:nix bork$ echo $PATH
/nix/store/v5q1bxrqs6hkbsbrpwc81ccyyfpbl8wk-clang-wrapper-11.1.0/bin:/nix/store/x9jmvvxcys4zscff39cnpw0kyfvs80vp-clang-11.1.0/bin:/nix/store/3f1ii2y5fs1w7p0id9mkis0ffvhh1n8w-coreutils-9.1/bin:/nix/store/8ldvi6b3ahnph19vm1s0pyjqrq0qhkvi-cctools-binutils-darwin-wrapper-973.0.1/bin:/nix/store/5kbbxk18fp645r4agnn11bab8afm0ry3-cctools-binutils-darwin-973.0.1/bin:/nix/store/5si884h02nqx3dfcdm5irpf7caihl6f8-cowsay-3.7.0/bin:/nix/store/5bs5q2dw5bl7c4krcviga6yhdrqbvdq6-ruby-3.1.4/bin:/nix/store/3f1ii2y5fs1w7p0id9mkis0ffvhh1n8w-coreutils-9.1/bin

Every dependency gets its own PATH entry: .../cowsay-3.7.0/bin, .../ruby-3.1.4/bin, and so on. Functional, but not what I wanted. I wanted a single directory of symlinks to drop into my normal shell's PATH.

Getting a Symlink Directory with `buildEnv`

A suggestion from the Nix Discord pointed me to buildEnv, which assembles a directory of symlinks from Nix packages:

{
  inputs.nixpkgs.url = "github:NixOS/nixpkgs/nixpkgs-23.05-darwin";
  outputs = { self, nixpkgs }: {
    defaultPackage.aarch64-darwin = nixpkgs.legacyPackages.aarch64-darwin.buildEnv {
      name = "julia-stuff";
      paths = with nixpkgs.legacyPackages.aarch64-darwin; [
        cowsay
        ruby
      ];
      pathsToLink = [ "/share/man" "/share/doc" "/bin" "/lib" ];
      extraOutputsToInstall = [ "man" "doc" ];
    };
  };
}

The result is a result/bin full of links:

$ ls result/bin/
bundle  bundler  cowsay  cowthink  erb  gem  irb  racc  rake  rbs  rdbg  rdoc  ri  ruby  typeprof

That gave me what I could put in my PATH. Next I went about adding every package from my old nix-env -q list.

Problems Adding Packages

Unfree Packages

ngrok is not free, and Nix offered three ways to handle it. Options A and B were most promising:

       c) For `nix-env`, `nix-build`, `nix-shell` or any other Nix command you can add
         { allowUnfree = true; }
       to ~/.config/nixpkgs/config.nix.

Option C's suggestion to add { allowUnfree = true} to ~/.config/nixpkgs/config.nix did nothing for me. Option A worked:

            $ export NIXPKGS_ALLOW_UNFREE=1

        Note: For `nix shell`, `nix build`, `nix develop` or any other Nix 2.4+
        (Flake) command, `--impure` must be passed in order to read this
        environment variable.

Relative Path Flakes

For custom flakes I'd made earlier, I wanted to reference them like this:

      hugoFlake.url = "path:../hugo-0.40";
      paperjamFlake.url = "path:../paperjam";

The first nix build worked. The second produced an inscrutable error. My workaround was to delete flake.lock before every build. There's a lengthy GitHub issue thread about this behavior, but I never found a proper solution.

The "Build Hook" Error

For a while, every nix build ended with:

$ nix build
error:
       … while reading the response from the build hook

       error: unexpected EOF reading a line

After much fruitless poking at flake.nix, the fix turned out to be killing the nix-daemon process. I suspect a botched Nix upgrade, and I don't think this is a common issue.

A Broken Package

Adding zulu for Java produced a complaint about a broken symlink:

$ nix build
error: builder for '/nix/store/4n9c4707iyiwwgi9b8qqx7mshzrvi27r-julia-dev.drv' failed with exit code 2;
       last 1 log lines:
       > error: not a directory: `/nix/store/2vc4kf5i28xcqhn501822aapn0srwsai-zulu-11.62.17/share/man'
       For full logs, run 'nix log /nix/store/4n9c4707iyiwwgi9b8qqx7mshzrvi27r-julia-dev.drv'.
$ ls /nix/store/2vc4kf5i28xcqhn501822aapn0srwsai-zulu-11.62.17/share/ -l
lrwxr-xr-x 29 root 31 Dec  1969 man -> zulu-11.jdk/Contents/Home/man

The zulu package in nixpkgs-23.05 appears to have been broken; it's since been fixed in the unstable branch. I already had Java installed elsewhere, so I removed zulu from the list and moved on.

Wiring It Into My PATH

With the problems solved, I wrote a small script to build my flake and symlink the result to ~/.nix-flake. The rm flake.lock handles the relative-path problem, and NIXPKGS_ALLOW_UNFREE covers ngrok:

#!/bin/bash

set -euo pipefail

export NIXPKGS_ALLOW_UNFREE=1
cd ~/work/nixpkgs/flakes/grapefruit || exit
rm flake.lock
nix build --impure --out-link ~/.nix-flake

Then I put ~/.nix-flake at the front of my PATH.

GC Roots

All the experimentation had eaten about 20GB in /nix/store, so I wanted to run garbage collection. I found two commands — nix-store --gc and nix-collect-garbage — and couldn't tell you the difference. The latter seemed to delete more.

Before collecting, I needed to confirm ~/.nix-flake was a GC root so my packages wouldn't be swept away. Running nix-store --gc --print-roots showed it was there. That command also runs a GC, so it was a slightly dangerous way to check, but it worked out.

Speed Concerns

Installing a small package with nix-env -iA took two seconds:

$ time nix-env -iA nixpkgs.sl
installing 'sl-5.05'
these 2 paths will be fetched (0.41 MiB download, 3.77 MiB unpacked):
  /nix/store/yv1c98m5pncx3i5q7nr7i7mfjkiyii72-ncurses-6.4
  /nix/store/2k78vf30czicjs0dq9x0sj4017ziwxkn-sl-5.05
copying path '/nix/store/yv1c98m5pncx3i5q7nr7i7mfjkiyii72-ncurses-6.4' from 'https://cache.nixos.org'...
copying path '/nix/store/2k78vf30czicjs0dq9x0sj4017ziwxkn-sl-5.05' from 'https://cache.nixos.org'...
building '/nix/store/zadpfs9k1cw5x7iniwwcqd8lb7nnc7bb-user-environment.drv'...

________________________________________________________
Executed in    1.96 secs      fish           external

The same install via flakes took seven seconds, plus editing time:

$ vim ~/work/nixpkgs/flakes/grapefruit/flake.nix
$ time nix-symlink
________________________________________________________
Executed in    7.04 secs    fish           external
   usr time    1.78 secs    0.29 millis    1.78 secs
   sys time    0.51 secs    2.03 millis    0.51 secs

I don't have a fix for that, so I'm living with it for now.

Final Workflow

My routine is now:

Edit flake.nix to add or remove packages
Rerun the nix-symlink script
Periodically run nix-collect-garbage

One last task was setting up the registry:

nix registry add nixpkgs github:NixOS/nixpkgs/nixpkgs-23.05-darwin

That pins nix run nixpkgs#cowsay to the 23.05 nixpkgs version. I wanted to avoid re-downloading the repository, and I'm perfectly happy with pinned versions rather than chasing nixpkgs-unstable.

Where This Leaves Me

My solutions are probably not the best available. But I now have a working setup built on one relatively simple flake.nix and a six-line bash script, with no extra abstraction layers I don't understand. I might try flakey-profile someday, as it looks similarly minimal.

Flakes can manage far more — language package ecosystems, dotfiles, whole system configuration via tools like home-manager. That's not for me right now. I need my setup simple enough to debug when it breaks.

I still find Nix deeply confusing, but once something works, it tends to stay working. Whether that holds true for flakes is still an open question. If it goes badly, I can always go back to nix-env -iA.

JE

Written by

Julia Evans

Writes for Julia Evans. Part of the Tech Report engineering index.

GitHub Bluesky

All articles by Julia Evans →