Lab 13: The Input Encoder (Milestone 8)

Background

Lab 12's frontend sends plain characters. This lab makes every key work: arrows, function keys, modifiers, Ctrl and Alt combinations, paste, mouse, and focus — all mode-aware.

terminal-input is a pure, table-driven crate with no windowing dependency, which means the entire key encoding is unit-testable without a window. That is the design decision that makes this lab tractable.

Why This Lab Matters

  • Input encoding is where the most user-visible compatibility bugs live, and they are all reproducible in a unit test.
  • This is where the terminal's output state (modes) feeds back into the input path — the coupling that surprises people about terminal architecture.

Prerequisites

Predict First

  1. Ctrl+Shift+A. What does the legacy encoding send? Is it distinguishable from Ctrl+A?
  2. On macOS, Option+B. What does winit give you as text? As logical_key?
  3. ?2004 is off and the user pastes three lines into bash. What happens?
  4. A program sets ?1003. How many bytes per second while the mouse moves?

Step 1: The Crate and Its Own Types

cargo new --lib crates/terminal-input --name terminal-input

[dependencies]
terminal-core = { path = "../terminal-core" }   # for the mode flags ONLY
# NOT winit. Enforced in CI:
#   cargo tree -p terminal-input | grep -E 'winit|softbuffer|wgpu'   → empty

#![allow(unused)]
fn main() {
/// This crate defines its OWN key types. terminal-gui translates winit's types
/// into these at the boundary. That is what makes the encoder unit-testable
/// without a window, and swappable to a different windowing library.
#[derive(Copy, Clone, PartialEq, Eq, Debug)]
pub enum Key {
    Char(char),
    Named(NamedKey),
}

#[derive(Copy, Clone, PartialEq, Eq, Debug)]
pub enum NamedKey {
    Enter, Tab, Backspace, Escape, Space, Delete, Insert,
    ArrowUp, ArrowDown, ArrowLeft, ArrowRight,
    Home, End, PageUp, PageDown,
    F1, F2, F3, F4, F5, F6, F7, F8, F9, F10, F11, F12,
    KeypadEnter, Keypad0, /* ... */
}

#[derive(Copy, Clone, PartialEq, Eq, Default, Debug)]
pub struct Modifiers {
    pub shift: bool,
    pub alt: bool,
    pub ctrl: bool,
    pub superkey: bool,
}

impl Modifiers {
    pub fn any(self) -> bool { self.shift || self.alt || self.ctrl || self.superkey }
    /// The CSI modifier parameter: 1 + sum of 1(shift) 2(alt) 4(ctrl) 8(super).
    pub fn param(self) -> u8 {
        1 + self.shift as u8 + 2 * self.alt as u8
          + 4 * self.ctrl as u8 + 8 * self.superkey as u8
    }
}
}

Step 2: The Encoder

#![allow(unused)]
fn main() {
pub fn encode_key(key: Key, mods: Modifiers, modes: &TerminalModes) -> Option<Vec<u8>> {
    match key {
        Key::Named(n) => encode_named(n, mods, modes),
        Key::Char(c) => encode_char(c, mods, modes),
    }
}

fn encode_named(key: NamedKey, mods: Modifiers, modes: &TerminalModes) -> Option<Vec<u8>> {
    use NamedKey::*;
    let m = mods.param();
    Some(match key {
        // Enter sends CR, not LF. The keyboard's RETURN key always did, and
        // ICRNL translates it kernel-side.
        Enter if !mods.any() => vec![0x0d],
        Enter if mods.alt => vec![0x1b, 0x0d],
        Tab if mods.shift => b"\x1b[Z".to_vec(),         // CBT: back-tab
        Tab if !mods.any() => vec![0x09],
        // Backspace is DEL (0x7f); Ctrl+Backspace is BS (0x08). Some setups
        // expect the reverse — make it configurable and document the default.
        Backspace if mods.ctrl => vec![0x08],
        Backspace if mods.alt => vec![0x1b, 0x7f],
        Backspace => vec![0x7f],
        Escape => vec![0x1b],
        Space if mods.ctrl => vec![0x00],               // Ctrl+Space = NUL
        Space => vec![b' '],

        // Arrows: mode-dependent, and MODIFIED arrows always use the CSI form
        // even when DECCKM is set. That asymmetry is xterm's behavior.
        ArrowUp | ArrowDown | ArrowRight | ArrowLeft => {
            let f = match key { ArrowUp => 'A', ArrowDown => 'B',
                                ArrowRight => 'C', _ => 'D' };
            if mods.any() {
                format!("\x1b[1;{m}{f}").into_bytes()
            } else if modes.contains(Mode::APP_CURSOR_KEYS) {
                format!("\x1bO{f}").into_bytes()
            } else {
                format!("\x1b[{f}").into_bytes()
            }
        }

        Home | End => {
            let f = if key == Home { 'H' } else { 'F' };
            if mods.any() { format!("\x1b[1;{m}{f}").into_bytes() }
            else { format!("\x1b[{f}").into_bytes() }
        }

        // CSI ~ keys: the modifier goes AFTER the number.
        Insert => tilde(2, mods), Delete => tilde(3, mods),
        PageUp => tilde(5, mods), PageDown => tilde(6, mods),

        // F1-F4 use SS3; F5-F12 use CSI ~. The gaps (no 16, no 22) are historical.
        F1 | F2 | F3 | F4 => {
            let f = match key { F1 => 'P', F2 => 'Q', F3 => 'R', _ => 'S' };
            if mods.any() { format!("\x1b[1;{m}{f}").into_bytes() }
            else { format!("\x1bO{f}").into_bytes() }
        }
        F5 => tilde(15, mods), F6 => tilde(17, mods), F7 => tilde(18, mods),
        F8 => tilde(19, mods), F9 => tilde(20, mods), F10 => tilde(21, mods),
        F11 => tilde(23, mods), F12 => tilde(24, mods),

        _ => return None,
    })
}

fn tilde(n: u16, mods: Modifiers) -> Vec<u8> {
    if mods.any() { format!("\x1b[{n};{}~", mods.param()).into_bytes() }
    else { format!("\x1b[{n}~").into_bytes() }
}

fn encode_char(c: char, mods: Modifiers, modes: &TerminalModes) -> Option<Vec<u8>> {
    // Ctrl+letter → letter & 0x1f. Note Ctrl+M IS Enter and Ctrl+I IS Tab —
    // not analogous, identical.
    if mods.ctrl {
        if let Some(b) = ctrl_byte(c) {
            return Some(if mods.alt { vec![0x1b, b] } else { vec![b] });
        }
        // No legacy encoding exists (e.g. Ctrl+1). Use modifyOtherKeys if the
        // program enabled it; otherwise drop the key rather than sending
        // something the program will misread.
        if modes.contains(Mode::MODIFY_OTHER_KEYS) {
            return Some(format!("\x1b[27;{};{}~", mods.param(), c as u32).into_bytes());
        }
        return None;
    }
    if mods.alt {
        // ESC prefix, never 8-bit meta: 0xe2 is a valid UTF-8 lead byte.
        let mut v = vec![0x1b];
        v.extend_from_slice(c.encode_utf8(&mut [0u8; 4]).as_bytes());
        return Some(v);
    }
    // Plain characters take the TEXT path in the GUI, not this function —
    // so that layouts, dead keys, and IME work.
    None
}
}

Step 3: The Translation Boundary in the GUI

#![allow(unused)]
fn main() {
// crates/terminal-gui/src/input.rs
// This file is the ONLY place winit's types meet terminal-input's types.

fn translate_key(ev: &winit::event::KeyEvent) -> Option<terminal_input::Key> {
    use winit::keyboard::{Key as WKey, NamedKey as WNamed};
    match &ev.logical_key {
        WKey::Named(n) => translate_named(*n).map(terminal_input::Key::Named),
        WKey::Character(s) => s.chars().next().map(terminal_input::Key::Char),
        _ => None,
    }
}

fn on_key(&mut self, ev: &winit::event::KeyEvent) {
    if ev.state != ElementState::Pressed { return; }   // terminals send on press

    // 1. Named keys and modifier combinations go through the encoder.
    if let Some(key) = translate_key(ev) {
        let is_named = matches!(key, terminal_input::Key::Named(_));
        if is_named || self.mods.ctrl || self.mods.alt {
            if let Some(bytes) = encode_key(key, self.mods, self.terminal.modes()) {
                let _ = self.pty.write_all(&bytes);
                self.scroll_to_bottom();     // typing cancels scrollback view
                return;
            }
            if is_named { return; }          // a named key we do not encode: drop it
        }
    }

    // 2. Plain text. This path handles layouts, dead keys (´+e = é), and IME.
    if let Some(text) = &ev.text {
        // Filter control characters that slipped through as text; we already
        // handled the ones we want above.
        let filtered: String = text.chars().filter(|c| !c.is_control() || *c == '\t').collect();
        if !filtered.is_empty() {
            let _ = self.pty.write_all(filtered.as_bytes());
            self.scroll_to_bottom();
        }
    }
}
}

Step 4: Mouse

#![allow(unused)]
fn main() {
fn on_mouse(&mut self, ev: MouseEvent) {
    // Cell coordinates from pixels. Clamp: a drag can leave the window.
    let col = (ev.x as usize / self.renderer.cell.width).min(self.terminal.cols() - 1);
    let row = (ev.y as usize / self.renderer.cell.height).min(self.terminal.rows() - 1);

    // Shift bypasses mouse reporting so the user can always select text, even in
    // a program that has grabbed the mouse. Every terminal does this; without it
    // you cannot copy from htop.
    if !self.mods.shift {
        if let Some(bytes) = encode_mouse(
            MouseEvent { col, row, ..ev }, self.terminal.modes()) {
            let _ = self.pty.write_all(&bytes);
            return;
        }
    }
    // Not reported: handle locally as selection.
    self.handle_selection(ev, row, col);
}

fn on_wheel(&mut self, delta: f32) {
    if self.terminal.on_alt_screen() {
        // Nothing to scroll on the alternate screen, so translate the wheel into
        // arrow keys — the near-universal convention that makes `less` and `man`
        // scroll with the wheel.
        let (key, n) = if delta > 0.0 { (NamedKey::ArrowUp, 3) } else { (NamedKey::ArrowDown, 3) };
        for _ in 0..n {
            if let Some(b) = encode_key(Key::Named(key), NO_MODS, self.terminal.modes()) {
                let _ = self.pty.write_all(&b);
            }
        }
    } else {
        self.scroll_offset = (self.scroll_offset as i64 - delta as i64 * 3)
            .clamp(0, self.terminal.scrollback_len() as i64) as usize;
        self.terminal.damage_all();
    }
}
}

Step 5: Reset Everything on Exit

#![allow(unused)]
fn main() {
impl Drop for App {
    fn drop(&mut self) {
        // A program that crashed may have left modes set. Our own exit must not
        // add to that — and if we are relaying to an outer terminal (the mux
        // client case) leaving mouse reporting on makes the user's shell emit
        // garbage on every click. This is a rude bug; do not ship it.
        const RESET: &[u8] = b"\x1b[?1000l\x1b[?1002l\x1b[?1003l\x1b[?1006l\
                               \x1b[?2004l\x1b[?1004l\x1b[?25h\x1b[?1049l\x1b[0m";
        let _ = self.pty.write_all(RESET);
    }
}
}

Also install a panic hook that does the same, because Drop does not run on abort.

Expected Output

Run your Lab 1 inspector inside your own GUI terminal — the best possible test:

$ cargo run -p raw-inspector          # running inside terminal-gui
[ 1 byte ]  61   |a|            → 'a'
[ 3 bytes]  1b 5b 41   |^[[A|   → CSI A — Up arrow
[ 1 byte ]  03   |^C|           → ETX (Ctrl+C)
[ 2 bytes]  1b 62   |^[b|       → ESC + b (Alt+B)
[ 3 bytes]  1b 4f 50   |^[OP|   → SS3 P — F1
[ 6 bytes]  1b 5b 31 3b 35 41   → CSI 1;5 A — Ctrl+Up
[ 5 bytes]  1b 5b 31 35 7e      → CSI 15~ — F5
[ 6 bytes]  1b 5b 32 30 30 7e   → CSI 200~ — bracketed paste start

Debugging Steps

Ctrl+C does nothing

You are using the text event, which is empty for Ctrl+C. Use the logical key.

Alt+B produces ∫ on macOS

Option is composing. Use the logical key and add an "Option as Meta" setting.

Arrows work in bash but not in vim

DECCKM is not being consulted, or vim set it and your modes are stale.

Nothing works on a Dvorak/AZERTY layout

You used physical keys for characters. Use the logical key or the text.

Pasting into vim staircases

Bracketed paste not implemented, or vim did not enable it because it did not detect support.

Clicking in htop does nothing

Mouse modes not consulted, or coordinates are 0-based (the protocol is 1-based).

You cannot select text in htop

You have not implemented the Shift bypass.

The shell emits garbage after your terminal exits

Modes not reset on exit.

Experiment

CLAIM. Your terminal's key encoding either matches xterm or it does not, and the difference is exactly measurable.

METHOD. Build a comparison harness:

# 1. In xterm (or Ghostty/kitty), run the inspector and record every key.
cargo run -p raw-inspector 2>&1 | tee /tmp/reference.txt
#    Press, in order: every letter with and without Ctrl and Alt; every arrow
#    with every modifier combination; F1-F12 with and without Shift;
#    Home/End/Insert/Delete/PgUp/PgDn; Tab and Shift+Tab; Enter, Backspace, Escape.

# 2. Do exactly the same inside YOUR terminal.
cargo run -p terminal-gui
#    ...run the inspector inside it, same key sequence
#    tee to /tmp/mine.txt

# 3. Diff.
diff /tmp/reference.txt /tmp/mine.txt

PREDICTION. Before diffing: how many differences do you expect? Which key category will differ most?

RESULT. Every difference is either a bug or a deliberate decision. Write the table: key, reference bytes, your bytes, verdict. Fix the bugs; document the decisions.

Test

#![allow(unused)]
fn main() {
// The encoder is pure, so the whole key table is a data-driven test.
#[test]
fn key_encoding_table() {
    let d = TerminalModes::default();
    let mut app = d; app.insert(Mode::APP_CURSOR_KEYS);

    let cases: &[(Key, Modifiers, &TerminalModes, &[u8])] = &[
        (Key::Named(Enter), NONE, &d, b"\r"),
        (Key::Named(Tab), NONE, &d, b"\t"),
        (Key::Named(Tab), SHIFT, &d, b"\x1b[Z"),
        (Key::Named(Backspace), NONE, &d, b"\x7f"),
        (Key::Named(Backspace), CTRL, &d, b"\x08"),
        (Key::Named(Escape), NONE, &d, b"\x1b"),
        (Key::Named(ArrowUp), NONE, &d, b"\x1b[A"),
        (Key::Named(ArrowUp), NONE, &app, b"\x1bOA"),
        (Key::Named(ArrowUp), CTRL, &d, b"\x1b[1;5A"),
        (Key::Named(ArrowUp), CTRL, &app, b"\x1b[1;5A"),   // CSI even with DECCKM
        (Key::Named(ArrowUp), SHIFT, &d, b"\x1b[1;2A"),
        (Key::Named(ArrowUp), ALT, &d, b"\x1b[1;3A"),
        (Key::Named(Home), NONE, &d, b"\x1b[H"),
        (Key::Named(End), NONE, &d, b"\x1b[F"),
        (Key::Named(Delete), NONE, &d, b"\x1b[3~"),
        (Key::Named(Delete), CTRL, &d, b"\x1b[3;5~"),
        (Key::Named(PageUp), NONE, &d, b"\x1b[5~"),
        (Key::Named(F1), NONE, &d, b"\x1bOP"),
        (Key::Named(F4), NONE, &d, b"\x1bOS"),
        (Key::Named(F5), NONE, &d, b"\x1b[15~"),
        (Key::Named(F12), NONE, &d, b"\x1b[24~"),
        (Key::Named(F5), SHIFT, &d, b"\x1b[15;2~"),
        (Key::Char('c'), CTRL, &d, b"\x03"),
        (Key::Char('a'), CTRL, &d, b"\x01"),
        (Key::Char('['), CTRL, &d, b"\x1b"),
        (Key::Named(Space), CTRL, &d, b"\x00"),
        (Key::Char('b'), ALT, &d, b"\x1bb"),
        (Key::Char('c'), CTRL_ALT, &d, b"\x1b\x03"),
    ];
    for (key, mods, modes, expected) in cases {
        let got = encode_key(*key, *mods, modes);
        assert_eq!(got.as_deref(), Some(*expected),
                   "key={key:?} mods={mods:?} app_cursor={}",
                   modes.contains(Mode::APP_CURSOR_KEYS));
    }
}

#[test]
fn ctrl_shift_a_is_dropped_without_modify_other_keys() {
    // The legacy protocol cannot represent it. Sending a wrong byte is worse
    // than sending nothing — the program would act on the wrong key.
    let d = TerminalModes::default();
    assert_eq!(encode_key(Key::Char('a'), CTRL_SHIFT, &d), Some(vec![0x01]));
    assert_eq!(encode_key(Key::Char('1'), CTRL, &d), None);
}

#[test]
fn modify_other_keys_encodes_the_impossible_combinations() {
    let mut m = TerminalModes::default();
    m.insert(Mode::MODIFY_OTHER_KEYS);
    assert_eq!(encode_key(Key::Char('1'), CTRL, &m).unwrap(), b"\x1b[27;5;49~".to_vec());
}

#[test]
fn terminal_input_has_no_windowing_dependency() {
    // Verified in CI:
    //   cargo tree -p terminal-input | grep -E 'winit|softbuffer|wgpu' → empty
}
}

Challenge Extensions

  1. Implement the kitty keyboard protocol (at least flag 1, disambiguation). Then prove Escape is no longer ambiguous by testing it over an artificially delayed link.
  2. Configurable keybindings from TOML, including a "send these literal bytes" action.
  3. IME support: handle winit's Ime events for CJK input, with a preedit overlay.
  4. Key repeat handling: winit reports repeat: true; decide whether to coalesce under load.
  5. Build the xterm comparison harness from the experiment as a real test that runs in CI against a recorded reference.
  6. Implement ?1004 focus reporting and verify vim reloads changed files on focus.

Deliverables

  • terminal-input with its own key types and no windowing dependency.
  • Every row of the key encoding table implemented and tested.
  • DECCKM honored, including the modified-arrow asymmetry.
  • Ctrl/Alt combinations, including the Ctrl+letter rule and the ESC prefix.
  • modifyOtherKeys support.
  • Bracketed paste with envelope protection.
  • Mouse reporting with the Shift bypass and correct 1-based coordinates.
  • Focus reporting.
  • All modes reset on every exit path, including panic.
  • The xterm comparison table from the experiment, with each difference classified.

Validation / Self-check

  1. Which of physical key / logical key / text do you use for each of: a, Ctrl+C, Alt+B, F5, é?
  2. State the Ctrl-key rule, and name three keys that are identical to control bytes.
  3. Why an ESC prefix rather than 8-bit meta?
  4. Why do modified arrows use the CSI form even under DECCKM?
  5. Give the modifier parameter for Ctrl+Alt+Shift, and show a full sequence using it.
  6. Why is Ctrl+1 dropped rather than encoded, and what changes with modifyOtherKeys?
  7. Why does Shift bypass mouse reporting?
  8. Why does the wheel become arrow keys on the alternate screen?
  9. Why must terminal-input not depend on winit? Name two capabilities it buys.
  10. Which modes must be reset on exit, and which exit paths must do it?

Next: Lab 14 — Selection, Copy, and Paste.