Booting Ubuntu Under Firecracker: A Day of Image Troubles

Yesterday was spent wrestling with Ubuntu VM images for Firecracker, and it was a day of mixed results. While the eventual goal—a bootable VM—remained elusive, the process yielded several useful lessons about image manipulation, kernel builds, and the quirks of different Ubuntu base images.

Manipulating Disk Images: Sparse Files and Partition Extraction

One approach involved extracting the filesystem directly from the Ubuntu cloud image. This didn't pan out, but it surfaced some practical tricks for working with raw disk images:

  • sfdisk is the tool to inspect a disk image's partition table.
  • Extracting a specific partition is a job for dd, using the partition's start sector and length, e.g., dd if=focal-server-cloudimg-amd64.img.orig.raw of=focal-image.ext4 skip=227328 count=4384735.
  • Linux files can be "sparse," meaning runs of zero bytes aren't physically stored, which is a boon for disk images. The fallocate -d command can convert a regular file into a sparse one.

Kernel Compilation: Surprisingly Quick

Firecracker's setup guide recommends building a custom Linux kernel, which sounded daunting at first. It turned out to be a non-event. A full compile with make -j12 on an AMD Ryzen 5 took roughly five minutes—far less than the anticipated multi-hour slog.

The first build lacked ISO filesystem support, but fixing that was trivial: reconfigure the kernel and rebuild. No deep kernel hacking required.

Base Image Choice: No Clear Winner

The core question remains which Ubuntu base to use for VMs. Two candidates were tried: the official Ubuntu cloud image (from https://cloud-images.ubuntu.com/focal/current/focal-server-cloudimg-amd64.img) and the Docker ubuntu:20.04 image. Both have notable issues.

The cloud image stubbornly refused to boot, hanging at a cryptic A start job is running for /dev/disk/by-label/UEFI step. The Docker image boots, but cloud-init won't run properly inside it, and it's missing fundamental components like init and udev. Since it's not designed for interactive login, it feels incomplete as a base system.

The next step is likely circling back to the cloud image to diagnose its boot failure.