Compiling C programs without being a C programmer

Every so often I need to build a C or C++ program from source. For years my strategy was: install the dependencies, run make, and if that failed, hunt for a pre-built binary or give up. That approach mostly worked on Linux, but since moving to a Mac, I've had to actually compile things myself more often. Here's what I've learned, using a few real programs as examples: paperjam, sqlite, and qf, a small pager for opening files from rg output.

Getting started: compiler, dependencies, and configure

First, you need a C compiler. On an Ubuntu system, this installs gcc, g++, and make:

sudo apt-get install build-essential

On a Mac, the equivalent is installing the Xcode command line tools.

C has no dependency manager, which means you find and install dependencies yourself. The good news is that C projects usually keep dependencies minimal, and they're often available through your system package manager. The README usually tells you what you need. For paperjam, it says you need the headers for libqpdf and libpaper, plus a2x from AsciiDoc for building man pages. On a Debian-based system, that means installing packages like this:

sudo apt install -y libqpdf-dev libpaper-dev

When a README names a package like libqpdf-dev, assume it's talking about Debian-based distros. On a Mac with Homebrew, it would be brew install qpdf instead.

Some C projects ship with a Makefile; others include a ./configure script instead. Sqlite's source, for instance, has ./configure. This script is part of the autotools system. Run it, it prints a lot of inscrutable output, and then either generates a Makefile or fails because a dependency is missing. That's all you really need to understand about autotools.

Running make

Once you have a Makefile, the next step is make. You can often speed things up with make -j8 to parallelize the build. Expect a flood of compiler warnings; if you didn't write the code, ignore them.

Compiler errors, on the other hand, are usually a sign that a dependency isn't being found correctly. For example, compiling paperjam on a Mac produced an error about qpdf. The library was installed, but the compiler and linker didn't know where to look for it.

The compiler and linker, briefly

Building C programs happens in two stages:

  1. Compiling source code into object files, using gcc or clang.
  2. Linking those object files into the final binary, using ld.

Dependency problems happen when you need to tell these tools where to find the libraries and header files the program uses.

make gets its compiler and linker configuration from environment variables. In paperjam's Makefile, for example, you'll see variables like LDLIBS:

paperjam: $(OBJS)
	$(LD) -o $@ $^ $(LDLIBS)

Whatever you put into LDLIBS gets passed to the linker as command-line arguments. make also has implicit variables it automatically passes to the compiler and linker. One of them, CPPFLAGS, gets passed to the C compiler automatically. In paperjam's case, the Makefile hardcodes CXXFLAGS, so setting CPPFLAGS was the only way to inject compiler flags without editing the Makefile.

There are two ways to pass environment variables to make:

  1. CXXFLAGS=xyz make
  2. make CXXFLAGS=xyz

They differ in precedence: make CXXFLAGS=xyz overrides any value set inside the Makefile; the first form does not.

The flags that fix dependency errors

To fix the paperjam build, I needed to pass two kinds of flags:

  • -I/opt/homebrew/include to the compiler, telling it where header files live (e.g., /opt/homebrew/include/qpdf/QPDF.hh)
  • -L/opt/homebrew/lib -liconv to the linker, telling it where libraries are and which ones to link in (the -l flag means "link this library"; -liconv links iconv, -lm links the math library)

Here's the full incantation that worked:

CPPFLAGS="-I/opt/homebrew/include" LDLIBS="-L/opt/homebrew/lib -liconv" make paperjam

Figuring out those flags involved a lot of confused searching. The clue came from looking at how other packaging systems build the same program. For example, the Nix package for paperjam contains this hint:

  env.NIX_LDFLAGS = lib.optionalString stdenv.hostPlatform.isDarwin "-liconv";

That's basically "pass -liconv to the linker on a Mac." The same file also sets a compile flag related to a "PointerHolder transition," which matched an error I was seeing. Other distros' build files are a great debugging resource even when you can't use their packages directly.

Three small make tricks

Build just one file. If you're in a directory with many tools but only need one, you can target it directly. The qf tool lives in a directory full of other utilities; to build just qf, I ran:

make qf

make $FILENAME works if you know (or can guess) the name of the output binary.

You don't need a Makefile for small programs. For a single-file C program called blah.c, make has implicit rules that just work:

make blah

This expands to cc -o blah blah.c automatically.

Installing the binary. Some Makefiles have an install target (make install), but if you'd rather not trust it with your system layout, you can manually copy the compiled binary to your PATH instead:

cp qf ~/bin

Why this matters

Understanding a little about how C programs are built—header files, compiler flags, linker flags—pays off even if you never write C yourself. Once I got comfortable with these basics, I could even contribute a paperjam package to Homebrew, so future installs are just brew install paperjam. The details differ between packaging systems, but they all ultimately drive the same compilers and linkers.

Two topics I didn't cover: LD_LIBRARY_PATH/DYLD_LIBRARY_PATH (for telling the runtime linker where to find shared libraries) and pkg-config. The first I haven't needed recently, and the second I still don't fully understand. There's always more to learn, but being able to get a C program compiled yourself is a genuinely useful skill.