Lab 1.1: Build Firecracker from Source

Background

Before you can read Firecracker seriously, you must be able to build it the way its maintainers do — not with a one-off cargo build, but through tools/devtool, the Docker-based build/test wrapper that pins the exact toolchain, target, and container the CI uses. Getting a clean build is the single most important gate in this whole level: every later lab, every test run, every PR you ever submit starts from a green build, and the build itself teaches you the repo's shape — the workspace crates, the musl target, where the binaries land.

Firecracker builds inside a Docker dev container (public.ecr.aws/firecracker/fcuvm:vNN, tag increments over time — verify on your branch) with your checkout bind-mounted in. The container carries the pinned Rust toolchain, the cross-compilers for x86_64 and aarch64, the static musl libc, and the Python test harness. You do not install a specific Rust version system-wide; tools/devtool does that for you. That is deliberate: a contributor's build must be bit-reproducible against CI, and "works on my machine with my system Rust" is not acceptable.

Why This Lab Matters for Contributors

  • Every PR must pass tools/devtool checkbuild --all and the CI build. If you cannot reproduce the build locally you cannot reproduce a CI failure, and you will waste maintainers' time.
  • The build output teaches you the crate layout — which members exist, which one is the big vmm crate, where firecracker and jailer come from.
  • The musl static-link choice, the pinned channel, and the rust-vmm dependency versions are all facts you will reference for years; you find them here, in rust-toolchain.toml, Cargo.toml, and Cargo.lock. See the rust-vmm overview for why those crates matter.

Prerequisites

  • A Linux host (x86_64 or aarch64) with a working /dev/kvm, Docker, and Git. Firecracker does not build or run anywhere else.
  • You have skimmed the Level 1 overview and read README.md + CONTRIBUTING.md.
# Pre-flight. All three must succeed before you go further.
ls -l /dev/kvm                 # device must exist and be readable/writable
docker info >/dev/null && echo "docker OK"
git --version
uname -m                       # x86_64 or aarch64 — note which; you'll use it in paths

Step-by-Step Tasks

Step 1: Clone the repository

git clone https://github.com/firecracker-microvm/firecracker.git
cd firecracker
# Note the default branch and the latest tag — you may want to build a released version.
git branch --show-current
git tag --sort=-creatordate | head -5

Work on the default branch (main) for this curriculum unless a lab says otherwise. If you check out a tag, remember it — version-sensitive facts in this book must be verified against whatever you have.

Step 2: Inspect the pinned toolchain

The repo pins its Rust channel in rust-toolchain.toml. That pinned version is the project's effective MSRV (minimum supported Rust version) and the version CI uses; never override it.

cat rust-toolchain.toml
# Expect something like:
#   [toolchain]
#   channel = "1.96.0"     ← the exact pin (verify on your branch)
#   targets = ["x86_64-unknown-linux-musl", "aarch64-unknown-linux-musl"]

Note two things: the channel (the Rust version) and the targets — both end in -unknown-linux-musl. Firecracker statically links against musl libc by default, which is part of why the binary is small and self-contained and runs cleanly inside a stripped jailer chroot.

Step 3: Build (debug) with devtool

tools/devtool build is the canonical build. The first run pulls the Docker dev container and does a cold cargo build of the whole workspace — it is slow (many minutes). Subsequent builds are incremental and fast.

tools/devtool build

What this does: starts the fcuvm container with your checkout bind-mounted, then runs cargo build (via tools/release.sh) for your host architecture's musl target in debug mode.

Note: On the very first invocation, devtool may ask to download the container image and may need your user added to the docker group. If tools/devtool complains it cannot talk to the Docker daemon, see Troubleshooting below.

Step 4: Build release

The debug binary is large and slow; CI artifacts and any performance work use --release.

tools/devtool build --release

--release turns on optimizations and is what you boot in Lab 1.3 when you care about the < 125 ms boot claim. You now have both a debug and a release build.

Step 5: Find the binaries

Built artifacts land under build/cargo_target/<target-triple>/<profile>/. The target triple is <arch>-unknown-linux-musl.

ARCH=$(uname -m)
# Debug build:
ls -l build/cargo_target/${ARCH}-unknown-linux-musl/debug/firecracker
# Release build:
ls -l build/cargo_target/${ARCH}-unknown-linux-musl/release/firecracker
# The jailer is built alongside firecracker in the same directory:
ls -l build/cargo_target/${ARCH}-unknown-linux-musl/release/jailer

If you are unsure of your exact triple, let the filesystem tell you:

find build/cargo_target -maxdepth 2 -type d -name release -o -type d -name debug
find build/cargo_target -name firecracker -type f

Step 6: Run --version

Confirm the binary actually runs on your host.

ARCH=$(uname -m)
BIN=build/cargo_target/${ARCH}-unknown-linux-musl/release/firecracker
$BIN --version
# Firecracker v1.x.x
#   ... build/commit info ...

Note there is no "run" subcommand. firecracker --help shows flags like --api-sock, --config-file, --no-api, --seccomp-filter, --enable-pci — you configure a microVM over the API socket (or a config file) and then InstanceStart it. That is the whole CLI surface, and you use it for real in Lab 1.3.

$BIN --help        # read every flag; you will use several this level

Step 7: The jailer

The jailer binary is built by the same workspace build. It is the production isolation barrier — it runs as root, sets up a chroot/cgroups/namespaces, drops privileges, then execs firecracker. You will not use it until Level 9, but confirm it built and reads.

ARCH=$(uname -m)
build/cargo_target/${ARCH}-unknown-linux-musl/release/jailer --help | head -30
# Note flags: --id, --exec-file, --uid, --gid, --chroot-base-dir, --netns, --cgroup, --daemonize

The jailer's design is in docs/jailer.md; the jailer deep dive takes it apart. For now: it is a separate binary that wraps firecracker, and it does not apply seccomp (firecracker does that to itself).

Step 8 (aside): A raw cargo build

You can build without devtool if you already have the pinned toolchain and a musl target installed. This is useful for fast iteration inside the container or with rust-analyzer, but it is not how you produce CI-equivalent artifacts.

# Only if you have rustup with the pinned channel + the musl target:
rustup target add x86_64-unknown-linux-musl
cargo build --release --target x86_64-unknown-linux-musl
# The binary lands in target/ (NOT build/cargo_target/) unless CARGO_TARGET_DIR is set —
# devtool redirects it to build/cargo_target. That difference trips people up.

Tip: Use cargo build (or your IDE's check) for the fast inner loop while reading and editing, but always re-run tools/devtool build and tools/devtool checkbuild --all before you trust a result for a PR. The container is ground truth; your host is not.

Implementation Requirements / Deliverables

  • tools/devtool build (debug) completes with no errors.
  • tools/devtool build --release completes with no errors.
  • You located both the debug and release firecracker binaries by full path and ran --version.
  • You located the jailer binary and read its --help.
  • You recorded the pinned Rust channel from rust-toolchain.toml and the target triple for your host.

Troubleshooting

Docker permission denied

tools/devtool talks to the Docker daemon. If you see permission denied while trying to connect to the Docker daemon socket, your user is not in the docker group.

sudo usermod -aG docker "$USER"
# Log out and back in (or `newgrp docker`) so the group takes effect, then retry.
docker run --rm hello-world   # confirm daemon access without sudo

Running devtool under sudo works but pollutes file ownership in your checkout — fix the group instead.

/dev/kvm not present or not accessible

The build itself does not need KVM, but every run does, and devtool's test path does.

ls -l /dev/kvm
# If missing: you are on a host without hardware virtualization exposed.
#   - Cloud: use a bare-metal instance (AWS *.metal) or enable nested virtualization.
#   - Check you are in the kvm group:  groups | tr ' ' '\n' | grep kvm
sudo setfacl -m u:${USER}:rw /dev/kvm   # grant yourself access if the group route is awkward

musl vs gnu target confusion

The default and CI target is *-unknown-linux-musl (static). If you build for *-gnu (dynamic glibc) you get a different binary that will not match CI artifacts and may surprise you inside a stripped chroot. tools/devtool build -l gnu exists for specific needs, but default to musl.

# Confirm what you actually built links against:
file build/cargo_target/$(uname -m)-unknown-linux-musl/release/firecracker
# "statically linked" is what you want for the musl build.

The first build is extremely slow

The cold build compiles the entire workspace plus all rust-vmm and third-party crates. Tens of minutes on the first run is normal. It is not hung — watch cargo's "Compiling …" output. Give it CPU and RAM; incremental rebuilds afterward are seconds-to-minutes. Do not Ctrl-C and retry repeatedly; that throws away partial progress.

error: failed to run custom build command / linker errors

Usually a missing musl target or a partially-pulled container. Re-run inside devtool (which guarantees the toolchain), and if it persists, force a fresh container:

tools/devtool --help            # see if there is a clean/rebuild flag on your branch
docker image ls | grep fcuvm    # confirm the dev container actually pulled

Expected Output

$ tools/devtool build --release
[Firecracker devtool] Starting build (release, x86_64-unknown-linux-musl) ...
   Compiling vmm v...
   Compiling firecracker v...
   Compiling jailer v...
    Finished release [optimized] target(s) in ...

$ ls build/cargo_target/x86_64-unknown-linux-musl/release/firecracker
build/cargo_target/x86_64-unknown-linux-musl/release/firecracker

$ build/cargo_target/x86_64-unknown-linux-musl/release/firecracker --version
Firecracker v1.x.x

(Exact version, container tag, and crate list vary by branch — verify yours.)

Stretch Goals

  1. Pin down the Rust channel and confirm it is the MSRV. Read rust-toolchain.toml, then grep the docs/CI for any independent MSRV statement and confirm they agree:

    cat rust-toolchain.toml
rg -n -i "msrv|minimum supported rust|rust.?version" README.md CONTRIBUTING.md docs/ .github/

    Write down the version. You will compare it to what rust-vmm crates require.

  2. Find the bundled rust-vmm crate versions. Firecracker consumes kvm-ioctls, kvm-bindings, vm-memory, linux-loader, vm-superio, event-manager, vmm-sys-util, vm-fdt. Find the versions it actually builds against — Cargo.toml gives the requested range, Cargo.lock gives the exact resolved version:

    rg -n "kvm-ioctls|kvm-bindings|vm-memory|linux-loader|vm-superio|event-manager|vmm-sys-util|vm-fdt" Cargo.toml src/*/Cargo.toml
# Exact resolved versions (what is really compiled):
rg -n -A2 '^name = "kvm-ioctls"|^name = "vm-memory"|^name = "linux-loader"' Cargo.lock

    Record kvm-ioctls and kvm-bindings versions specifically — you will use those same crates in Lab 1.4, and matching versions avoids API surprises.

  3. Enumerate the workspace members. Confirm the crate list from the overview against reality:

    rg -n -A30 "\[workspace\]" Cargo.toml | rg "members|src/"
ls src/

    For each member, state in one line what it does. If a crate exists that is not in the overview table, find out why (the workspace evolves).

  4. Build for the other architecture. If you are on x86_64, cross-build the aarch64 target (or vice versa) — devtool's container has both cross-compilers:

    tools/devtool build --release -- --target aarch64-unknown-linux-musl   # syntax may vary; check devtool --help

    You cannot run the foreign binary, but a clean cross-build confirms the toolchain is complete.

Validation / Self-check

You are done when you can answer these without notes:

  1. What does tools/devtool build do that a bare cargo build does not, and why does it matter for a contributor reproducing a CI failure?
  2. What is the exact target triple your build produced, and what does the musl part mean for the resulting binary?
  3. Where on disk are the debug and release firecracker binaries, and how does that path differ from a plain cargo build's output directory?
  4. What pins the Rust toolchain version, what is that version on your branch, and why must you not override it?
  5. What is the jailer binary for, and which security mechanism does it not apply (and who does)?
  6. Which two rust-vmm crates will you reuse directly in Lab 1.4, and what versions does Firecracker resolve them to?

When you can answer all six, proceed to Lab 1.2 — Run the Tests.