The Serial Console and Legacy Devices

Not everything in a microVM is virtio. Before the guest kernel has probed a single virtio-mmio device, it is already printing boot messages — over a 16550A UART at the classic COM1 port. That UART, a partial i8042 PS/2 controller, an in-kernel PIC/IOAPIC/PIT, and an RTC are the legacy devices: the fixed, architecturally-mandated hardware a PC-class kernel expects to exist whether or not anyone configured it. They do not live on the virtio-mmio bus, they are not described by virtqueues, and most of them are not even emulated in Firecracker's address space. This chapter is about that other device model.

The split matters because it changes where the code lives and how an access is routed. On x86_64 the serial console and i8042 sit on the PIO bus (port I/O), reached by IN/OUT instructions that exit as KVM_EXIT_IO — not the MMIO bus that carries virtio. The interrupt controller and timer are not in Firecracker at all; KVM emulates them in the kernel. After this chapter you will be able to trace a guest's console=ttyS0 write from 0x3f8 all the way to your terminal, explain why that path is bounded for security, describe the deliberately-crippled i8042 that exists only to catch a reboot, and say why KVM_CREATE_IRQCHIP means there is no PIC code to read in userspace.

Note: "Legacy" here is not a slur — it is a category. These devices predate virtio and are part of the x86 platform contract: a Linux kernel will assume a 16550 at COM1 and an i8042 at 0x60/0x64 exist. Firecracker provides the minimum that keeps a kernel happy (a real serial console; just enough i8042 to notice a reset) and offloads the rest (interrupt controller, timer) to KVM. Minimal device = smaller attack surface. Read this chapter as a study in how little a secure VMM can get away with emulating.

Legacy vs virtio: two device models, two buses

# Where the legacy devices live and who wires them up (x86 PIO manager).
rg -n "PortIODeviceManager|devices/legacy|mod legacy|legacy::" src/vmm/src/device_manager/ src/vmm/src/devices/
find src/vmm/src/devices/legacy -type f

Start by drawing the boundary. A virtio device (block, net, vsock) is a Firecracker-authored emulation living behind an MMIO register block, described to the guest by a virtio_mmio.device= cmdline token, and serviced through virtqueues. A legacy device is a fixed-function PC peripheral at a fixed, well-known address that the kernel already knows about. The two are dispatched by different exit reasons and handled by different managers.

Tip: PIO is x86-only. The IN/OUT instructions and the 16-bit port space do not exist on aarch64, so there is no PortIODeviceManager there. On aarch64 the serial console and RTC are MMIO devices, and the interrupt controller is the GIC. Whenever you see PIO, mentally tag it "x86." See the vCPU run loop and VM exits for the exit taxonomy and the MMIO bus and device manager for the MMIO side.

The PortIODeviceManager owns the PIO Bus and inserts the legacy devices at their fixed port ranges. It is the PIO twin of the MMIODeviceManager — same Bus data structure (a sorted address-range → device map), different address space and exit reason.

The 16550 UART: the serial console

# The serial device, its COM1 port, and the vm-superio Serial it wraps.
rg -n "Serial|SerialWrapper|SerialDevice|0x3[fF]8|COM1|vm_superio|com_evt" src/vmm/src/devices/legacy/ src/vmm/src/device_manager/
rg -n "struct Serial|impl .*Serial|fn enqueue_raw_bytes|fn write\b" src/vmm/src/devices/legacy/serial.rs

The serial console is a 16550A UART emulated by the rust-vmm vm-superio crate's Serial type (see vm-superio). Firecracker wraps it in a small adapter (look for a Serial/ SerialWrapper-style struct in devices/legacy/; verify the exact name on your branch) that adds the input eventfd handling and the output destination. It is registered on the PIO bus at the standard COM1 base 0x3f8 (ports 0x3f8–0x3ff).

When the guest kernel boots with console=ttyS0 on its cmdline, every byte it prints is an OUT to a register in that 0x3f8 range. That is a PIO write, so it exits as KVM_EXIT_IO and arrives on the vCPU thread as VcpuExit::IoOut(0x3f8.., data). The run loop hands it to the PIO bus, which routes it to the serial device's write, which feeds the byte into the vm-superio Serial, which emits it to the configured output — your terminal's stdout, a named pipe, or a file, depending on how the API configured the serial.

sequenceDiagram
    participant G as Guest kernel (console=ttyS0)
    participant K as KVM
    participant V as vCPU thread (run loop)
    participant P as PIO Bus
    participant S as Serial (vm-superio 16550)
    participant O as Host output (stdout / pipe / file)
    G->>K: OUT 0x3f8, byte   (write to UART THR)
    K->>V: KVM_EXIT_IO → VcpuExit::IoOut(0x3f8, data)
    V->>P: pio_bus.write(0x3f8, data)
    P->>S: serial.write(offset=0, data)
    S->>O: emit byte to output
    Note over S,O: synchronous, on the vCPU thread — keep it cheap

   ┌───────── x86 PIO port map (legacy) ─────────┐
   │ 0x3f8 .. 0x3ff   COM1  (16550 serial)        │  ← console=ttyS0 lives here
   │ 0x2f8 .. 0x2ff   COM2  (disabled by default) │
   │ 0x060            i8042 data                   │
   │ 0x064            i8042 command/status         │
   │ (0x020/0x0a0 PIC, 0x040 PIT — owned by KVM)   │
   └──────────────────────────────────────────────┘

The whole point: the login prompt, kernel panic dumps, and dmesg output all reach you through this one device. There is no framebuffer, no graphics — the serial console is the console.

Note: Firecracker's default kernel cmdline typically includes 8250.nr_uarts=0, which tells the kernel's 8250 driver to probe zero extra UARTs. The single console UART is still wired by Firecracker; the flag just stops the kernel from poking at additional, non-existent serial ports. Confirm the exact default cmdline on your branch: rg -n "console=ttyS0|8250.nr_uarts|reboot=k|panic=1" src/vmm/src/.

Serial input: host → guest, via the EventManager

# The input side: stdin / a buffer fed into the guest through epoll.
rg -n "stdin|in_buffer|enqueue_raw_bytes|process_serial|EventManager|register|read" src/vmm/src/devices/legacy/serial.rs src/vmm/src/devices/legacy/

Output is driven by the guest (it writes, you see it). Input is the reverse: when you type into the microVM's console, those bytes originate on the host's stdin and must be pushed into the guest. Firecracker does this with the EventManager — it registers the serial input fd (e.g. stdin) with epoll, and when bytes are available the handler reads them and enqueues them into the UART's receive register, which raises a serial interrupt so the guest's tty driver picks them up. See the EventManager for the epoll loop this hangs off of.

So the serial console is bidirectional but asymmetric: output is a synchronous PIO-exit path on the vCPU thread; input is an asynchronous epoll path on the VMM/event thread.

Bounding serial output: a security boundary, not a nicety

# The cap / metering on serial output and its metrics.
rg -n "out_buffer|MAX|flush|SerialOut|metrics|missed_write|FifoFull|loss" src/vmm/src/devices/legacy/serial.rs src/vmm/src/devices/legacy/
rg -n "serial" src/vmm/src/logger/metrics.rs

Here is the part a careless VMM gets wrong. The serial console is host-facing I/O driven entirely by an untrusted guest. A malicious or buggy guest can write to 0x3f8 in a tight loop, as fast as the vCPU can issue OUT instructions. If Firecracker forwarded every byte to the host unconditionally, the guest could:

• Fill the host's stdout pipe/file faster than the reader drains it, blocking the vCPU thread inside the write (the guest is stopped during a VM exit — a blocked write is a stalled guest and wasted host CPU).
• Flood a log/console with unbounded data — a denial-of-service against the host.

So Firecracker bounds the serial output: there is a buffer cap on the output side, and writes beyond it are dropped (and counted in metrics) rather than allowed to block or grow without limit. The invariant is that no amount of guest console traffic can wedge the vCPU thread or exhaust host resources. Treat any guest-driven host I/O the same way — the legacy serial path is the textbook case.

flowchart LR
    G["Guest spams OUT 0x3f8<br/>(untrusted, unbounded)"] --> S["Serial.write()"]
    S --> C{"output buffer<br/>under cap?"}
    C -->|yes| O["emit to host output"]
    C -->|no| D["drop byte + bump<br/>loss metric"]
    O -.->|never block the vCPU thread| G
    D -.->|never block the vCPU thread| G

Warning: When you read or modify the serial output path, the load-bearing property is "untrusted guest cannot wedge or flood the host." Any change that lets a guest write block indefinitely, or removes the cap, is a security regression, not a buffering tweak. The metric exists precisely so that hitting the cap is observable rather than silent.

The partial i8042: just enough to catch a reboot

# The i8042 emulation — note how little of it there is.
rg -n "I8042|i8042|0x60|0x64|kbd|reset_evt|reset|CtrlAltDel" src/vmm/src/devices/legacy/ src/vmm/src/device_manager/
rg -n "struct I8042|fn read\b|fn write\b|reset" src/vmm/src/devices/legacy/i8042.rs

Firecracker emulates an i8042 PS/2 controller — but only enough of it to detect a CPU reset. It is not a keyboard. There is no scancode translation, no real keyboard buffer, no mouse. The i8042 sits at the standard ports 0x60 (data) and 0x64 (command/status), and the only behavior that matters is this: when the guest writes the command that asserts the CPU reset line (a "pulse output line / reset" command on the i8042), Firecracker interprets it as a reboot/shutdown signal and fires a reset eventfd (look for a reset_evt-style field) rather than emulating any keyboard behavior.

This is why the default kernel cmdline includes reboot=k panic=1:

• reboot=k tells the Linux kernel to reboot via the keyboard controller (the i8042) — exactly the path Firecracker watches. Without it, the kernel might try ACPI or a triple-fault reboot that Firecracker's partial i8042 would never see.
• panic=1 reboots one second after a panic, turning a guest panic into a clean shutdown via that same path.

The host-initiated counterpart is the SendCtrlAltDel API action: PUT /actions with action_type: "SendCtrlAltDel" drives the same i8042 reset path, asking the guest to reboot gracefully.

rg -n "SendCtrlAltDel|CtrlAltDel|send_ctrl_alt_del" src/vmm/src/ src/firecracker/

flowchart TD
    subgraph triggers["Two ways the reset fires"]
        A["Guest reboot=k →<br/>OUT 0x64, reset cmd"]
        B["Host: PUT /actions<br/>SendCtrlAltDel"]
    end
    A --> I["i8042.write(0x64, cmd)"]
    B --> I
    I --> C{"reset command?"}
    C -->|yes| R["signal reset_evt"]
    C -->|no| N["acknowledge / ignore<br/>(no real keyboard)"]
    R --> S["reboot / shutdown path"]
    S -.-> X["see signals-shutdown-and-reset.md"]

The reset eventfd is consumed by the shutdown/reset machinery — see signals, shutdown and reset for what happens after the eventfd fires.

Note: The i8042 is the clearest example of Firecracker's "emulate the minimum" philosophy. A full keyboard controller would be dozens of stateful registers and a security liability for a workload that has no keyboard. Firecracker keeps the one behavior a serverless/container guest actually needs — an orderly reboot — and discards the rest.

The interrupt controller and timer: emulated by KVM, not Firecracker

# The irqchip and PIT are created in KVM; there is NO userspace PIC/IOAPIC code to read.
rg -n "create_irq_chip|KVM_CREATE_IRQCHIP|create_pit2|KVM_CREATE_PIT2|setup_irqchip|irqchip" src/vmm/src/ src/vmm/src/arch/

A PC kernel also expects an interrupt controller (the 8259 PIC and the IOAPIC, plus a per-vCPU LAPIC) and a timer (the 8254 PIT). These are legacy in the same architectural sense as the serial port — but Firecracker does not emulate them in userspace at all. Instead it asks KVM to create them in the kernel:

• KVM_CREATE_IRQCHIP creates the in-kernel PIC + IOAPIC + LAPICs.
• KVM_CREATE_PIT2 creates the in-kernel programmable interval timer.

This is a deliberate three-way win — minimal device surface (no PIC/IOAPIC/PIT emulation code in Firecracker to maintain or get wrong), performance (interrupt delivery and timer ticks never round-trip to userspace), and security (less attack surface in the VMM). On aarch64 the equivalent is the GIC, created via KVM_CREATE_DEVICE.

So when you grep the legacy device directory you will find the serial, i8042, and RTC — and you will not find a PIC or PIT, because there is nothing to find. That absence is the design. The full story of how a device's eventfd injects an interrupt through this in-kernel controller is in interrupts and the irqchip.

The RTC

rg -n "Rtc|rtc|pl031|RTC" src/vmm/src/devices/legacy/ src/vmm/src/arch/

vm-superio also provides a real-time clock. On aarch64 the RTC is an ARM pl031 (an MMIO device); on x86 the platform RTC story is more minimal — verify which RTC, at which port/MMIO address, your branch wires with the grep above before asserting anything in a review.

Reading exercise

# 1. The legacy device directory and the PIO manager that wires them (x86).
find src/vmm/src/devices/legacy -type f
rg -n "PortIODeviceManager|register.*pio|insert|0x3[fF]8|0x60|0x64" src/vmm/src/device_manager/

# 2. The serial device and the vm-superio Serial it wraps.
rg -n "Serial|SerialWrapper|vm_superio|enqueue_raw_bytes|com_evt" src/vmm/src/devices/legacy/

# 3. The output bound + loss metrics (the security boundary).
rg -n "out_buffer|MAX|flush|metrics|missed|loss|FifoFull" src/vmm/src/devices/legacy/serial.rs

# 4. The partial i8042 and the reset eventfd.
rg -n "I8042|0x60|0x64|reset|reset_evt|CtrlAltDel" src/vmm/src/devices/legacy/

# 5. The in-kernel irqchip + PIT (note the absence of userspace PIC/PIT).
rg -n "KVM_CREATE_IRQCHIP|create_irq_chip|KVM_CREATE_PIT2|create_pit2|create_gic" src/vmm/src/ src/vmm/src/arch/

# 6. The default cmdline tokens that make all this work.
rg -n "console=ttyS0|8250.nr_uarts|reboot=k|panic=1" src/vmm/src/

# 7. In a booted guest, confirm the hardware the kernel sees:
#    dmesg | grep -i ttyS        # 16550A at 0x3f8
#    cat /proc/interrupts        # no PIT/serial storm of userspace exits

Answer:

1. Which bus carries the serial console on x86, which VM exit reason does a console write produce, and which manager routes it? Why is none of this true on aarch64?
2. Trace one byte of console=ttyS0 output from the guest OUT instruction to your terminal. Name every hop and say which thread it runs on.
3. The serial output is bounded. What two failure modes does the bound prevent, and how is a dropped byte made observable?
4. How much of the i8042 does Firecracker emulate, and what is the only behavior that matters? How does reboot=k connect to it?
5. Which legacy devices does Firecracker emulate, and which does KVM emulate in the kernel? Give the KVM ioctl for each in-kernel one.
6. Serial input and serial output take different paths through Firecracker. Describe both and the thread each runs on.

Common bugs and symptoms

Validation: prove you understand this

1. Draw the PIO-vs-MMIO contrast: address space, triggering instruction, VM exit reason, and the manager that handles each. Place the serial console and a virtio-block device on the correct side.
2. Walk a single console=ttyS0 byte from the guest's OUT 0x3f8 to host stdout, naming the exit reason, the bus, the device, and the thread it all runs on.
3. Explain why the serial output is bounded and what specifically would go wrong if it were not — name both the DoS and the vCPU-stall failure modes, and how loss is surfaced.
4. Describe exactly how much of the i8042 Firecracker emulates and why, and explain the role of reboot=k panic=1 and the SendCtrlAltDel action in the reset path.
5. Explain why there is no PIC, IOAPIC, or PIT emulation code in the Firecracker source, naming the KVM ioctls that replace them and giving the three-part rationale (surface, performance, security).
6. Serial input and output are asymmetric. Contrast the two paths — synchronous PIO exit vs asynchronous epoll — and say which thread each runs on and why.

Next: The Rate Limiter and Token Bucket — how Firecracker bounds virtio block and net throughput with token buckets, the same "an untrusted guest must not exhaust the host" discipline you just saw applied to the serial console.