linux-loader

Firecracker has no BIOS and no bootloader. It boots an uncompressed Linux kernel directly: it parses the kernel image, copies its loadable segments into guest memory, fabricates the boot-time data structures the kernel expects to find (on x86_64, the "zero page" boot_params with an e820 memory map; on aarch64, an FDT), sets the initial vCPU registers, and jumps straight into the kernel's entry point in 64-bit long mode. The crate that does the parsing and structure-writing half of that is rust-vmm's linux-loader. It is the boot sequence deep dive's engine room.

After this chapter you can: explain the KernelLoader trait and its Elf and BzImage implementations; describe the bootparam structs (setup_header, boot_params) and the configurators (LinuxBootConfigurator, PvhBootConfigurator) that write the zero page + e820; build a kernel command line with Cmdline; and find where Firecracker calls each piece in its boot path.

Note: linux-loader knows the Linux boot protocol — the ELF/bzImage formats, the boot_params layout, the e820 conventions. It does not know Firecracker's memory layout, its e820 entries, or its default cmdline. That policy lives in Firecracker's arch/. The division is exactly the rust-vmm pattern: the crate knows the protocol, the VMM supplies the values.

What linux-loader provides

# Confirm the dependency and the pinned version (verify on your branch):
rg -n "linux-loader" Cargo.toml src/vmm/Cargo.toml Cargo.lock
cargo doc -p linux-loader --no-deps --open

docs.rs: docs.rs/linux-loader. Three modules matter:

Loading the kernel: the KernelLoader trait

KernelLoader is the trait that turns a kernel image (a Read + Seek) plus a GuestMemory into "segments copied into guest RAM, here is the entry point."

# Find the loader trait and the two impls:
find ~/.cargo/registry/src -maxdepth 2 -type d -name 'linux-loader-*' \
  -exec rg -n "trait KernelLoader|impl KernelLoader|struct Elf|struct BzImage|fn load\b|KernelLoaderResult" {} +

KernelLoader::load() returns a KernelLoaderResult carrying, crucially, the kernel load address / entry point and (for bzImage) the setup_header it read out of the image. Firecracker takes that entry point and puts it in the vCPU's rip (x86_64) so the very first guest instruction is the kernel's.

#![allow(unused)]
fn main() {
use linux_loader::loader::{KernelLoader, elf::Elf};
use vm_memory::{GuestAddress, GuestMemoryMmap};
use std::fs::File;

fn load_vmlinux(guest_mem: &GuestMemoryMmap)
    -> Result<GuestAddress, linux_loader::loader::Error>
{
    let mut kernel = File::open("vmlinux").expect("open kernel");
    // Parse PT_LOAD segments and copy them into guest RAM.
    // `highmem_start_address` is the lowest address the loader may place at;
    // `None` = no forced load offset.
    let res = Elf::load(
        guest_mem,
        None,                       // kernel_offset
        &mut kernel,
        Some(GuestAddress(0x100000)) // himem start (1 MiB) — FC's HIMEM_START
    )?;
    // res.kernel_load is where it landed; res.kernel_end the top.
    // The entry point goes into the vCPU's rip.
    Ok(res.kernel_load)
}
}

Notice the loader writes into the GuestMemoryMmap from the previous chapter — linux-loader depends on vm-memory and copies segments via write_slice. The crates compose: vm-memory owns the bytes, linux-loader knows what to put in them.

Configuring boot: the zero page and e820

A bare kernel entry point is not enough. The Linux x86 boot protocol says the kernel, on entry, expects rsi to point at a boot_params structure (the "zero page") describing the machine: the e820 memory map, the command-line pointer, the initrd location, and a setup_header with protocol fields. Writing that structure correctly is the configurator module's job.

# The bootparam structs and the configurators:
find ~/.cargo/registry/src -maxdepth 2 -type d -name 'linux-loader-*' \
  -exec rg -n "struct boot_params|struct setup_header|struct e820_entry|trait BootConfigurator|LinuxBootConfigurator|PvhBootConfigurator" {} +

Firecracker fills a boot_params: it sets the magic, the cmdline pointer (CMDLINE_START), the initrd address/size if present, and the e820 map describing its guest-RAM layout (RAM below the MMIO gap, reserved gap, RAM above 4 GiB — see guest memory). Then it calls the configurator to serialize that struct to the zero-page address (ZERO_PAGE_START, conventionally 0x7000 on Firecracker — verify in arch/x86_64/layout.rs).

 guest physical memory (x86_64), as linux-loader + FC arrange it:

  0x00007000  ZERO_PAGE_START   ── boot_params (e820, cmdline ptr, initrd ptr)
  0x00020000  CMDLINE_START     ── "console=ttyS0 reboot=k panic=1 ..."
  0x00100000  HIMEM_START (1MiB)── vmlinux PT_LOAD segments land here
     ...                         high RAM, MMIO gap below 4 GiB, RAM above 4 GiB

 vCPU initial state: rip = e_entry (long mode), rsi = ZERO_PAGE_START

sequenceDiagram
    participant FC as Firecracker (arch/x86_64)
    participant LL as linux-loader
    participant GM as GuestMemoryMmap
    FC->>LL: Elf::load(guest_mem, kernel_file)
    LL->>GM: write_slice(PT_LOAD segments)
    LL-->>FC: KernelLoaderResult{ kernel_load=e_entry }
    FC->>FC: build Cmdline, build e820 entries
    FC->>LL: LinuxBootConfigurator::write_bootparams(boot_params, ZERO_PAGE_START)
    LL->>GM: write_obj(boot_params @ 0x7000)
    FC->>FC: set vcpu regs: rip=e_entry, rsi=0x7000, long mode

PvhBootConfigurator is the alternate path: PVH is a different boot entry (used by some kernels / Cloud Hypervisor more than Firecracker) where the kernel reads a hvm_start_info struct instead of the legacy zero page. The crate supports both; Firecracker primarily uses the standard LinuxBootConfigurator for vmlinux.

The command line: Cmdline

The kernel command line is a C string in guest memory that boot_params points to. linux-loader's Cmdline is a builder that assembles it safely — it checks total length against a capacity, escapes/validates, and can add_virtio_mmio_device entries (the virtio_mmio.device=SIZE@ADDR:IRQ tokens that tell the guest where each virtio-mmio device lives — see virtio-transport-mmio).

#![allow(unused)]
fn main() {
use linux_loader::cmdline::Cmdline;

let mut cmdline = Cmdline::new(4096).expect("cmdline");        // capacity
cmdline.insert_str("console=ttyS0 reboot=k panic=1 pci=off nomodule").unwrap();
// Firecracker appends a virtio-mmio token per device so the guest can find it:
// cmdline.add_virtio_mmio_device(size, GuestAddress(addr), irq, None).unwrap();
let as_cstring = cmdline.as_cstring().unwrap();               // -> write to CMDLINE_START
}

Firecracker builds its default cmdline (roughly reboot=k panic=1 pci=off nomodule 8250.nr_uarts=0, plus console=ttyS0 and the boot args you pass via PUT /boot-source) with this builder, appends a virtio-mmio token per device, then writes the resulting C string to CMDLINE_START and stores that address in boot_params. The exact default varies — read it on your branch.

How Firecracker uses it: the boot path

# Where FC calls the loader, the configurator, and the cmdline builder:
rg -n "linux_loader|KernelLoader|Elf::load|BzImage|load_kernel|kernel_load" src/vmm/src/
rg -n "LinuxBootConfigurator|PvhBootConfigurator|boot_params|configure_system|e820|ZERO_PAGE" src/vmm/src/arch/x86_64/
rg -n "Cmdline|cmdline|add_virtio_mmio_device|CMDLINE_START" src/vmm/src/

aarch64: no zero page, an FDT instead

ARM has no boot_params/e820 zero page. Instead the kernel reads a flattened device tree (FDT/DTB) whose address it finds in register x0. Firecracker builds that with rust-vmm's vm-fdt crate (FdtWriter), not linux-loader's configurator. The kernel itself is an arm64 PE Image loaded via the PE loader.

rg -n "vm_fdt|FdtWriter|create_fdt|DRAM_MEM_START|fn configure_system" src/vmm/src/arch/aarch64/

So on aarch64 the boot-structure crate is vm-fdt; on x86_64 it is linux-loader's configurator. The kernel-loading crate (linux-loader) is shared across both, just with a different loader impl (PE vs ELF). See the boot sequence and Lab R2: Load a Kernel for the full hands-on path.

Reading exercise

# 1. The dependency and pinned version.
rg -n "linux-loader|vm-fdt" Cargo.toml Cargo.lock

# 2. The loader trait and impls for the pinned version.
cargo doc -p linux-loader --no-deps --open

# 3. The bootparam structs and configurators.
find ~/.cargo/registry/src -maxdepth 2 -type d -name 'linux-loader-*' \
  -exec rg -n "trait KernelLoader|LinuxBootConfigurator|struct boot_params|struct e820_entry" {} +

# 4. Firecracker's kernel-load call site.
rg -n "Elf::load|BzImage|load_kernel|KernelLoaderResult" src/vmm/src/

# 5. Firecracker's zero-page / e820 configuration.
rg -n "LinuxBootConfigurator|boot_params|configure_system|e820|ZERO_PAGE_START|CMDLINE_START" src/vmm/src/arch/x86_64/

# 6. The aarch64 FDT path (vm-fdt instead).
rg -n "FdtWriter|create_fdt|vm_fdt" src/vmm/src/arch/aarch64/

Answer:

1. What does the KernelLoader trait do, what does load() return, and what does Firecracker do with the returned entry point?
2. What is the zero page, what three things does boot_params carry, and which configurator writes it for the standard protocol?
3. What is the e820 map, who fills its entries (the crate or Firecracker), and why does Firecracker's RAM layout produce more than one entry?
4. Show the cmdline builder assembling a command line with a virtio-mmio token. Where does the resulting string live in guest memory, and how does the kernel find it?
5. Trace the x86_64 boot path crate-by-crate: which crate parses the kernel, which writes the zero page, which builds the cmdline, and where the initial registers are set.
6. On aarch64, which crate replaces linux-loader's configurator, and what structure does the kernel read instead of the zero page?

Common bugs and symptoms

Next: virtio-queue — the shared virtqueue model. Once the kernel is running it negotiates virtio devices and starts driving rings; this is the crate that defines those rings.