Lab 1.3: Boot Your First microVM End to End

Background

You have built Firecracker and run its tests. Now boot a real microVM from your own binary — a real guest Linux kernel, a real root filesystem, a real serial console you log in to. This is the moment the architecture stops being a diagram. Firecracker has no "run this VM" command: you start the process with an API socket, configure the machine by sending JSON over that socket (boot-source, a root drive, machine-config, optionally a network interface), and then issue a single InstanceStart action. The kernel boots, the rootfs mounts, and you get a login prompt.

The whole point of doing this by hand — rather than via firecracker-go-sdk or firectl — is that every curl PUT you send corresponds to a piece of internal machinery you will study in later levels. PUT /boot-source is not magic; it becomes a VmmAction on a channel, parsed by the API thread, applied to VmResources by the VMM thread, and consumed at boot when the kernel image is loaded into guest memory. You will not understand that path today, but you will see its inputs, and the forward-references in this lab tell you exactly where each input goes.

Why This Lab Matters for Contributors

Booting a microVM by hand is the baseline reproduction skill for almost every bug report. When an issue says "my VM won't boot," you reproduce it with exactly these calls. See Level 8.
Each configuration request is the surface of an internal subsystem: boot-source → the boot sequence; drives → virtio-block; network-interfaces → virtio-net and TAP; machine-config → the threading model (vCPU threads). Booting by hand maps the API to the internals.
The --config-file and --no-api alternatives matter for production orchestration; you learn both modes here.

Prerequisites

Lab 1.1 complete: a release firecracker binary you can locate.
A working /dev/kvm you can read and write.

ARCH=$(uname -m)
FC=build/cargo_target/${ARCH}-unknown-linux-musl/release/firecracker
$FC --version && ls -l /dev/kvm

Step-by-Step Tasks

Step 1: Get a guest kernel and a rootfs

docs/getting-started.md is the source of truth — follow it on your branch. The canonical artifacts are CI-built: an uncompressed kernel (vmlinux-X.Y.Z, an ELF that Firecracker loads directly — no bzImage, no BIOS) and an Ubuntu rootfs converted to a flat ext4 image. They live in the spec.ccfc.min S3 bucket; the getting-started doc gives you the current URLs and the conversion command.

# Read the canonical recipe first — URLs and versions change, so do not trust this lab's literals:
rg -n -i "vmlinux|squashfs|ext4|spec.ccfc.min|resources" docs/getting-started.md

ARCH=$(uname -m)
mkdir -p ~/fc-boot && cd ~/fc-boot

# The shape of what getting-started gives you (substitute the CURRENT versions/URLs from the doc):
#   1. A kernel:
#      curl -fLO https://s3.amazonaws.com/spec.ccfc.min/firecracker-ci/.../vmlinux-6.1.x
#   2. A rootfs (often a squashfs you convert to ext4):
#      curl -fLO https://.../ubuntu-24.04.squashfs
#      unsquashfs ubuntu-24.04.squashfs && truncate -s 1G ubuntu-24.04.ext4
#      mkfs.ext4 -d squashfs-root ubuntu-24.04.ext4

ls -l vmlinux-* ubuntu-*.ext4

Note: The kernel must match Firecracker's expectations — an uncompressed vmlinux ELF on x86_64 (Firecracker also accepts a bzImage/PE on aarch64). A compressed bzImage where a vmlinux is expected, or a wrong architecture, is the most common boot failure. Take the artifact getting-started points at; do not improvise a kernel yet.

Step 2: Start the VMM on an API socket

cd ~/fc-boot
ARCH=$(uname -m)
FC=~/firecracker/build/cargo_target/${ARCH}-unknown-linux-musl/release/firecracker

rm -f /tmp/firecracker.socket
sudo $FC --api-sock /tmp/firecracker.socket

The process starts and does nothing but listen on the socket. There is no VM yet — the API thread is up, the VMM thread is waiting, no vCPU threads exist. Leave this running in one terminal; do the curl calls in a second terminal. (The optional --enable-pci flag is discussed in Step 8.)

Step 3: Configure the boot source

API=/tmp/firecracker.socket
sudo curl -X PUT --unix-socket $API \
  --data '{"kernel_image_path":"./vmlinux-6.1.x","boot_args":"console=ttyS0 reboot=k panic=1 pci=off"}' \
  http://localhost/boot-source

What this configures: the kernel ELF to load and the kernel command line. console=ttyS0 routes the kernel's console to the serial port so you can watch the boot; reboot=k panic=1 make the guest halt cleanly on panic; pci=off reflects the no-PCI default. Where it goes: the API thread parses this into a VmmAction; the VMM thread stores it in VmResources; at InstanceStart the boot sequence feeds the path to linux-loader, which parses the ELF's PT_LOAD segments and copies them into guest memory.

# Locate where the boot config eventually lands (verify on your branch):
rg -n -i "boot.?source|kernel_image_path|BootSource" src/vmm/src/vmm_config/ | head

Step 4: Attach the root filesystem

sudo curl -X PUT --unix-socket $API \
  --data '{"drive_id":"rootfs","path_on_host":"./ubuntu-24.04.ext4","is_root_device":true,"is_read_only":false}' \
  http://localhost/drives/rootfs

What this configures: a virtio-block device backed by your ext4 file, marked as the root device (the guest kernel will mount it as /). Where it goes: the VMM thread will, at boot, create a virtio-block device, place it on the MMIO bus at a fixed address, and tell the guest about it via the cmdline (virtio_mmio.device=...). The block device's request handling is the subject of virtio-block and you trace a read down to a host pread in Level 7.

Step 5: Set the machine config

sudo curl -X PUT --unix-socket $API \
  --data '{"vcpu_count":2,"mem_size_mib":1024}' \
  http://localhost/machine-config

What this configures: the number of vCPUs (each becomes its own thread running the KVM_RUN loop) and the guest RAM size (host memory mmap'd and registered with KVM via KVM_SET_USER_MEMORY_REGION). Defaults if you skip this call: 1 vCPU, 128 MiB, SMT off. Where it goes: vcpu_count drives how many vCPU threads build_microvm_for_boot spawns; mem_size_mib sizes the GuestMemoryMmap you will build by hand, in miniature, in Lab 1.4.

Step 6 (optional): Add a network interface with a TAP device

Networking needs a host TAP device the virtio-net device bridges to. Create it first:

# On the host, create and bring up a TAP device (name it tap0):
sudo ip tuntap add tap0 mode tap
sudo ip addr add 172.16.0.1/24 dev tap0
sudo ip link set tap0 up

# Then tell Firecracker about it:
sudo curl -X PUT --unix-socket $API \
  --data '{"iface_id":"net1","guest_mac":"06:00:AC:10:00:02","host_dev_name":"tap0"}' \
  http://localhost/network-interfaces/net1

What this configures: a virtio-net device whose host side is the TAP. Packets the guest sends go out the TAP; packets you route to the TAP arrive in the guest. Where it goes: virtio-net and TAP. You can boot without networking — skip this step if you only want a console login.

Step 7: Start the microVM

sudo curl -X PUT --unix-socket $API \
  --data '{"action_type":"InstanceStart"}' \
  http://localhost/actions

This is the moment everything happens. The VMM thread calls the boot builder: guest memory is registered with KVM, the kernel ELF is loaded and the boot params / zero page are written, devices are created and placed on the bus, the vCPU threads are spawned, and each enters KVM_RUN. Watch the terminal running firecracker — kernel boot messages stream by on the serial console, then:

[    0.123456] Run /sbin/init as init process
...
Ubuntu 24.04 LTS fc-microvm ttyS0
fc-microvm login:

Step 8: Log in and look around

Log in (the getting-started rootfs uses a known user/password — commonly root with no password or a documented credential; check the doc). Inside the guest, confirm the resources you configured:

# Inside the guest:
nproc                 # 2 — the vcpu_count you set
free -m               # ~1024 MiB total — your mem_size_mib
cat /proc/cmdline     # the boot_args you passed, plus what Firecracker appended
ls /dev/vd*           # vda — your virtio-block rootfs
dmesg | grep -i virtio_mmio   # the MMIO device announcements

To shut down cleanly, send Ctrl-Alt-Del (which the partial i8042 catches and turns into a reboot, and with reboot=k panic=1 reboot=... the guest halts):

# From the host (second terminal):
sudo curl -X PUT --unix-socket $API --data '{"action_type":"SendCtrlAltDel"}' http://localhost/actions

Step 9: The config-file / `--no-api` alternative

Sending five curls by hand is fine for learning and reproduction, but production orchestrators often boot from a single JSON config file. Put the same configuration in a file with kebab-case section names and start with --config-file (which also implies the VM starts immediately; combine with --no-api to disable the control socket entirely):

cat > vmconfig.json <<'JSON'
{
  "boot-source": {
    "kernel_image_path": "./vmlinux-6.1.x",
    "boot_args": "console=ttyS0 reboot=k panic=1 pci=off"
  },
  "drives": [
    { "drive_id": "rootfs", "path_on_host": "./ubuntu-24.04.ext4",
      "is_root_device": true, "is_read_only": false }
  ],
  "machine-config": { "vcpu_count": 2, "mem_size_mib": 1024 }
}
JSON

sudo $FC --no-api --config-file ./vmconfig.json
# Boots straight to the login prompt — no socket, no curl, no InstanceStart needed.

The config-file sections (boot-source, drives[], machine-config, network-interfaces[], vsock, balloon, logger, metrics, mmds-config, entropy, …) mirror the API endpoints one-for-one. Confirm the schema on your branch:

rg -n -i "boot-source|machine-config|network-interfaces" src/firecracker/ | head

Note (aside): --enable-pci. By default Firecracker uses virtio-MMIO transport (no PCI enumeration). Recent versions add an optional virtio-PCI transport behind --enable-pci (verify on your branch; note CVE-2026-5747 was a PCI-transport bug fixed in 1.14.4/1.15.1). For Level 1, leave PCI off — MMIO is the default and the model the rest of this curriculum teaches.

Implementation Requirements / Deliverables

You obtained a vmlinux and an ext4 rootfs per docs/getting-started.md.
You booted a microVM to a login prompt over the serial console using the five-call curl sequence, and logged in.
Inside the guest, nproc, free -m, and cat /proc/cmdline reflect the machine-config and boot_args you set.
You booted the same microVM from a single --config-file (with --no-api).
For each of boot-source, drives, machine-config, you can state which thread parses it and which subsystem consumes it at boot.

Troubleshooting

`Failed to open /dev/kvm` / permission denied

The microVM cannot start without KVM access. You ran firecracker without enough privilege, or the device is inaccessible.

ls -l /dev/kvm
groups | tr ' ' '\n' | grep kvm
sudo setfacl -m u:${USER}:rw /dev/kvm   # then you can drop the sudo on firecracker

Socket permission / "address already in use"

A stale socket file from a previous run, or you started two VMMs on the same path.

rm -f /tmp/firecracker.socket
# Ensure no firecracker is already bound to it:
pgrep -a firecracker

If you start firecracker with sudo but curl without it (or vice-versa), the socket's owner may block you. Keep both under the same user/privilege.

Boot hangs with no kernel output

Almost always one of: wrong console arg, wrong kernel format, or a kernel/rootfs mismatch.

No serial output at all: your boot_args lack console=ttyS0, so the kernel logs nowhere you can see. Add it.
Boot source rejected / loader error: you passed a compressed bzImage where a vmlinux ELF is expected. Use the uncompressed kernel from getting-started.
Kernel panics "unable to mount root fs" / "no init found": the rootfs is wrong (not ext4, not the right architecture, or is_root_device not set). Confirm: file ubuntu-*.ext4 and that the rootfs arch matches the kernel arch.

Missing TAP / networking fails but VM boots

If you sent the network-interfaces PUT but never created tap0, the device config is accepted but the guest's NIC has no host backing. Create the TAP (Step 6) before InstanceStart, or skip networking entirely — it is optional for a console login.

`curl` returns 400 with a JSON error

The API validated your request and rejected it. Read the returned fault_message — it names the bad field (e.g. a path that does not exist on the host, or vcpu_count out of range). This validation layer is exactly what you will study and extend in Level 3.

Expected Output

# Terminal 1 (firecracker), after InstanceStart:
[    0.000000] Linux version 6.1.x ...
[    0.234567] Run /sbin/init as init process
...
Ubuntu 24.04 LTS fc-microvm ttyS0
fc-microvm login: root
root@fc-microvm:~# nproc
2
root@fc-microvm:~# free -m | awk '/Mem:/{print $2}'
1003
root@fc-microvm:~# ls /dev/vda
/dev/vda

Stretch Goals

Inspect instance state over the API. Before and after InstanceStart, query the instance:
```
sudo curl --unix-socket $API http://localhost/    # GET / → id, state (Not started / Running), version
```
Watch the state field flip. This GET is served entirely by the API thread.
Boot with the defaults. Skip machine-config entirely and boot. Confirm inside the guest that you got 1 vCPU and 128 MiB — Firecracker's documented defaults.
Add an entropy device. PUT /entropy {} configures a virtio-rng device. Add it, boot, and confirm /dev/hwrng exists in the guest. Cross-reference virtio-rng/entropy.
Capture a boot-time number. Add --log-path (or a logger config) and read the timestamped log to estimate time from InstanceStart to the first userspace process. You will optimize this in the boot-time masterclass.
Drive the guest from the host over the TAP. If you did Step 6, give the guest an IP on 172.16.0.2/24 and ping 172.16.0.1 from inside. You have just used your virtio-net device end to end.

Validation / Self-check

You are done when you can answer these without notes:

Why does Firecracker have no firecracker run vm.img command? How do you actually start a VM?
List the four required curl calls (kernel, rootfs, machine-config implied, start) in order, and say what each one configures.
What kind of kernel image does Firecracker load on x86_64, and why does a bzImage often fail where a vmlinux works?
For machine-config's vcpu_count, what does each count map to at runtime (hint: a thread doing what)?
What is the relationship between network-interfaces and a host TAP device, and what happens if you configure the interface but never create the TAP?
What does --config-file (with --no-api) change about how the VM is configured and started, and why would a production orchestrator prefer the socket?

When you can answer all six, proceed to the flagship project: Lab 1.4 — Build a Minimal KVM VMM.

Open-Source Engineer & Contributor