Input Encoding: Keys to Bytes

The input encoder is the mirror of the parser. The parser turns bytes into meaning; the encoder turns meaning into bytes. It is the component that decides that the Up arrow is \x1b[A — except when the program has set DECCKM, when it is \x1bOA.

It is also where the most compatibility bugs live, because there is no single standard: there is the VT100 lineage, the VT220 lineage, xterm's extensions, modifyOtherKeys, the kitty keyboard protocol, and forty years of programs written against whichever one they encountered.

Physical Keys vs. Logical Keys vs. Text

winit (and every modern windowing system) gives you three different things per key press, and using the wrong one is the most common frontend bug.

   Pressing the key labeled "A" on a US keyboard, with Shift:

   PHYSICAL KEY    KeyCode::KeyA
                   ← the physical position. SAME on every layout.
                   ← use for: Ctrl+key shortcuts, WASD-style bindings

   LOGICAL KEY     Key::Character("A")
                   ← what the key means with the current layout+modifiers
                   ← use for: deciding "this is the letter A"

   TEXT            "A"
                   ← the committed text, after IME and dead keys
                   ← use for: inserting characters

The rules for a terminal:

Warning: A frontend that uses only text cannot send Ctrl+C. A frontend that uses only physical keys is broken on Dvorak, AZERTY, and every non-US layout, and cannot do IME at all. You need both, and the dispatch order matters: check named keys first, then modifier combinations, then fall through to text.

#![allow(unused)]
fn main() {
fn on_key(&mut self, event: &KeyEvent) {
    if event.state != ElementState::Pressed { return; }   // terminals send on press only

    // 1. Named keys (arrows, F-keys, Enter, Tab, Backspace, Escape, ...).
    if let Key::Named(named) = &event.logical_key {
        if let Some(bytes) = encode_named(*named, self.mods, self.term.modes()) {
            self.pty.write_all(&bytes);
            return;
        }
    }

    // 2. Ctrl / Alt combinations with a character. These produce no usable text
    //    on most platforms, so we derive the bytes ourselves.
    if let Key::Character(s) = &event.logical_key {
        if self.mods.ctrl || self.mods.alt {
            if let Some(bytes) = encode_char_with_modifiers(s, self.mods, self.term.modes()) {
                self.pty.write_all(&bytes);
                return;
            }
        }
    }

    // 3. Plain text. This path handles layouts, dead keys, and IME output.
    if let Some(text) = &event.text {
        self.pty.write_all(text.as_bytes());
    }
}
}

The Control-Key Rule

   Ctrl + <char>  =  <uppercase char> & 0x1F

   Ctrl+A → 0x01     Ctrl+I → 0x09 (= Tab!)      Ctrl+[  → 0x1b (= Escape!)
   Ctrl+B → 0x02     Ctrl+J → 0x0a (= LF)        Ctrl+\  → 0x1c
   Ctrl+C → 0x03     Ctrl+M → 0x0d (= Enter!)    Ctrl+]  → 0x1d
   Ctrl+D → 0x04     Ctrl+Z → 0x1a               Ctrl+^  → 0x1e
   Ctrl+H → 0x08 (= Backspace)                   Ctrl+_  → 0x1f
   Ctrl+Space / Ctrl+@ → 0x00 (NUL)
   Ctrl+? → 0x7f (DEL)

These are not analogous, they are identical bytes. Ctrl+M and Enter are indistinguishable to the program. That is why vim cannot bind Ctrl+M separately from Enter, and it is a fact worth knowing before someone files it as a bug against your terminal.

#![allow(unused)]
fn main() {
fn ctrl_byte(c: char) -> Option<u8> {
    match c {
        ' ' | '@' => Some(0x00),
        'a'..='z' => Some(c as u8 - b'a' + 1),
        'A'..='Z' => Some(c as u8 - b'A' + 1),
        '[' => Some(0x1b), '\\' => Some(0x1c), ']' => Some(0x1d),
        '^' => Some(0x1e), '_' => Some(0x1f), '?' => Some(0x7f),
        _ => None,
    }
}
}

The Alt/Meta Rule

Two historical conventions:

   ESC PREFIX (the modern default):  Alt+B → 0x1b 0x62
   8-BIT META (legacy):              Alt+B → 0xe2  (0x62 | 0x80)

Use the ESC prefix. 8-bit meta is incompatible with UTF-8 — 0xe2 is a valid UTF-8 lead byte and would be decoded as part of a character.

macOS complication: the Option key is a composing modifier by default (Option+B gives ∫). Every macOS terminal has an "Option as Meta" setting for exactly this reason. Provide it, default it to on for the left Option key, and document the choice.

Special Keys: The Two Lineages

Both are in the wild. Emit what xterm emits — F1–F4 as SS3, F5–F12 as CSI ~, Home/End as CSI H/CSI F — because that is what TERM=xterm-256color promises and what programs' terminfo lookups expect.

Arrow Keys and DECCKM

   DECCKM (mode ?1) RESET — "normal cursor keys":
     Up=CSI A  Down=CSI B  Right=CSI C  Left=CSI D

   DECCKM SET — "application cursor keys":
     Up=SS3 A  Down=SS3 B  Right=SS3 C  Left=SS3 D

The program decides, by setting the mode. The terminal obeys. This is why the same key produces different bytes in bash (normal) and inside vim's insert mode (application) — and it is the concrete reason terminal-input must read terminal-core's mode flags.

#![allow(unused)]
fn main() {
fn encode_arrow(dir: char, mods: Modifiers, modes: &TerminalModes) -> Vec<u8> {
    if mods.any() {
        // MODIFIED arrows always use the CSI form with a modifier parameter,
        // regardless of DECCKM. This asymmetry is xterm's behavior and programs
        // depend on it.
        format!("\x1b[1;{}{}", modifier_param(mods), dir).into_bytes()
    } else if modes.contains(Mode::APP_CURSOR_KEYS) {
        format!("\x1bO{dir}").into_bytes()
    } else {
        format!("\x1b[{dir}").into_bytes()
    }
}
}

Modifier Encoding

   The modifier parameter is  1 + sum of:
     1 = Shift      2 = Alt/Meta      4 = Ctrl      8 = Super/Meta(hyper)

   Shift+Up        → CSI 1;2 A
   Alt+Up          → CSI 1;3 A
   Shift+Alt+Up    → CSI 1;4 A
   Ctrl+Up         → CSI 1;5 A
   Ctrl+Shift+Up   → CSI 1;6 A
   Ctrl+Alt+Up     → CSI 1;7 A
   Ctrl+Alt+Shift+Up → CSI 1;8 A

   For CSI ~ keys the modifier goes after the number:
     Ctrl+Delete   → CSI 3;5 ~
     Shift+F5      → CSI 15;2 ~

#![allow(unused)]
fn main() {
fn modifier_param(m: Modifiers) -> u8 {
    1 + (m.shift as u8) + 2 * (m.alt as u8) + 4 * (m.ctrl as u8) + 8 * (m.superkey as u8)
}
}

Beyond the Legacy Protocol

The legacy encoding cannot represent Ctrl+1, Ctrl+Shift+C, or key release events. Two extensions exist.

modifyOtherKeys (xterm)

   CSI > 4 ; 2 m     enable
   CSI > 4 ; 0 m     disable

   Then keys that have no legacy encoding are sent as:
     CSI 27 ; <modifier> ; <codepoint> ~
   e.g. Ctrl+Shift+A → CSI 27;6;65~

Simple, widely supported. Implement it.

The kitty keyboard protocol

   CSI > <flags> u    push a flag set
   CSI < u            pop
   CSI ? u            query current flags

   flags: 1 = disambiguate escape codes
          2 = report event types (press/repeat/release)
          4 = report alternate keys
          8 = report all keys as escape codes
         16 = report associated text

   Keys are then reported as:
     CSI <unicode-key-code> [; <modifiers> [: <event-type>]] u

Richer: it can report key release, distinguish Escape from an escape sequence unambiguously, and carry arbitrary key combinations. Supported by kitty, foot, WezTerm, Ghostty, and increasingly by Neovim and other TUIs.

Recommended scope: implement the legacy encoding fully, add modifyOtherKeys, and implement kitty flag 1 (disambiguation) if you have the appetite. Full kitty protocol support is a stretch goal.

The Escape-Key Ambiguity

   The user presses Escape:              1b
   The user presses the Up arrow:        1b 5b 41
                                          ↑
                          the same first byte

A program reading from the PTY cannot distinguish them without waiting. The universal solution is a timeout: if nothing follows ESC within ~25–50 ms, it was the Escape key.

Your terminal does not have this problem — you know which key was pressed. But you inherit the consequence: over a slow link, or when a program is busy, Escape feels laggy in vim. This is what kitty flag 1 fixes (Escape becomes CSI 27 u, unambiguously), and it is a good argument for implementing it.

Bracketed Paste

#![allow(unused)]
fn main() {
fn on_paste(&mut self, text: &str) {
    let bytes = terminal_input::encode_paste(text, self.term.modes());
    self.pty.write_all(&bytes);
}
}

See Lab 11 for the envelope-escape protection, which is mandatory.

When the mode is off, consider warning on a multi-line paste — pasting \n into a shell executes a command immediately, and a user pasting from a web page may not intend that. Several terminals now prompt. It is a product decision, but it should be a considered one.

Focus Reporting (?1004)

   Focus gained → CSI I
   Focus lost   → CSI O

Used by vim (to reload changed files) and tmux. Two lines to implement; do it.

The Complete Encoding Table

The full table lives in the appendix and is the specification for Lab 13. Summary:

Experiment

CLAIM. The same key produces different bytes depending on terminal mode, and this is externally observable.

METHOD.

# Run your Lab 1 inspector, and in another window drive the modes.
# (Or add mode toggles to the inspector itself.)

cargo run -p raw-inspector
#   press Up   → 1b 5b 41

# In another shell, targeting the inspector's tty:
printf '\033[?1h' > /dev/pts/N        # DECCKM on
#   press Up   → 1b 4f 41

printf '\033[?1l' > /dev/pts/N
#   press Up   → 1b 5b 41  again

# Now compare `bash` and `vim`:
pty-runner --record keys.cast
#   inside: press Up at the bash prompt, then run vim, press Up, :q
grep -o 'u001b\[?1[hl]' keys.cast

PREDICTION. Does bash set DECCKM? Does vim? Does vim restore it on exit? What happens if it crashes instead?

Test

#![allow(unused)]
fn main() {
#[test]
fn enter_is_cr_not_lf() {
    assert_eq!(encode_named(Named::Enter, NO_MODS, &default_modes()).unwrap(), vec![0x0d]);
}

#[test]
fn backspace_is_del_and_ctrl_backspace_is_bs() {
    assert_eq!(encode_named(Named::Backspace, NO_MODS, &default_modes()).unwrap(), vec![0x7f]);
    assert_eq!(encode_named(Named::Backspace, CTRL, &default_modes()).unwrap(), vec![0x08]);
}

#[test]
fn ctrl_letters_map_to_control_bytes() {
    for (c, b) in [('a', 0x01u8), ('c', 0x03), ('z', 0x1a), ('[', 0x1b), (' ', 0x00)] {
        assert_eq!(encode_char_with_modifiers(&c.to_string(), CTRL, &default_modes()).unwrap(),
                   vec![b], "Ctrl+{c}");
    }
}

#[test]
fn arrows_honor_decckm() {
    let mut modes = default_modes();
    assert_eq!(encode_named(Named::ArrowUp, NO_MODS, &modes).unwrap(), b"\x1b[A".to_vec());
    modes.insert(Mode::APP_CURSOR_KEYS);
    assert_eq!(encode_named(Named::ArrowUp, NO_MODS, &modes).unwrap(), b"\x1bOA".to_vec());
}

#[test]
fn modified_arrows_use_the_csi_form_even_with_decckm_set() {
    // The asymmetry is xterm's behavior; programs depend on it.
    let mut modes = default_modes();
    modes.insert(Mode::APP_CURSOR_KEYS);
    assert_eq!(encode_named(Named::ArrowUp, CTRL, &modes).unwrap(), b"\x1b[1;5A".to_vec());
}

#[test]
fn modifier_parameter_arithmetic() {
    assert_eq!(modifier_param(SHIFT), 2);
    assert_eq!(modifier_param(ALT), 3);
    assert_eq!(modifier_param(CTRL), 5);
    assert_eq!(modifier_param(CTRL | SHIFT), 6);
    assert_eq!(modifier_param(CTRL | ALT | SHIFT), 8);
}

#[test]
fn alt_uses_an_esc_prefix_not_the_high_bit() {
    // 8-bit meta is incompatible with UTF-8: 0xe2 is a valid lead byte.
    assert_eq!(encode_char_with_modifiers("b", ALT, &default_modes()).unwrap(),
               vec![0x1b, b'b']);
}

#[test]
fn function_keys_follow_the_xterm_split() {
    assert_eq!(encode_named(Named::F1, NO_MODS, &default_modes()).unwrap(), b"\x1bOP".to_vec());
    assert_eq!(encode_named(Named::F5, NO_MODS, &default_modes()).unwrap(), b"\x1b[15~".to_vec());
    assert_eq!(encode_named(Named::F12, NO_MODS, &default_modes()).unwrap(), b"\x1b[24~".to_vec());
}

#[test]
fn terminal_input_does_not_depend_on_winit() {
    // A compile-time boundary check, enforced in CI:
    //   cargo tree -p terminal-input | grep -E 'winit|wgpu|softbuffer'  → empty
}
}

Validation / Self-check

1. Name the three things a windowing system gives you per key press, and which to use for what.
2. Why can a text-only frontend not send Ctrl+C? Why is a physical-key-only frontend broken?
3. State the Ctrl-key rule. Which three common keys are identical to control bytes?
4. Why use an ESC prefix rather than 8-bit meta for Alt?
5. What are the two function-key lineages, and which does xterm use for which keys?
6. What is DECCKM, who sets it, and why does terminal-input need terminal-core's modes?
7. Why do modified arrows use the CSI form even when DECCKM is set?
8. Give the modifier parameter for Ctrl+Alt+Shift.
9. What is the Escape ambiguity, how do programs cope, and how does the kitty protocol remove it?
10. What must be stripped from a bracketed-paste payload, and why?
11. Why must terminal-input not depend on winit, and how do you enforce it?

Next: Fonts and Rasterization.