Project 5: The Kitty Keyboard Protocol

1–2 weeks · ●●●○○ · touches input encoding, mode state, and the oldest bug in terminals

The most approachable project in the portfolio, and it fixes a problem that has annoyed people since 1978.

1. The Problem

The legacy keyboard protocol cannot express most of what a keyboard can do.

The kitty keyboard protocol fixes all of it, and it is supported by kitty, foot, WezTerm, Ghostty, rio, and increasingly by Neovim, Helix, and other TUIs.

2. Why It Is Hard

Not the encoding — that is a table. The difficulty is state and compatibility.

3. The Design

The protocol

   CSI > <flags> u     PUSH a flag set onto the stack
   CSI < <n> u         POP n entries (default 1)
   CSI = <flags> ; <mode> u    SET flags (mode 1=set, 2=or, 3=and-not)
   CSI ? u             QUERY  →  the terminal replies  CSI ? <flags> u

   FLAGS (a bitmask):
     1   DISAMBIGUATE ESCAPE CODES   ← the important one
     2   REPORT EVENT TYPES          (press / repeat / release)
     4   REPORT ALTERNATE KEYS       (shifted and base-layout variants)
     8   REPORT ALL KEYS AS ESCAPE CODES
    16   REPORT ASSOCIATED TEXT

   KEY REPORT:
     CSI <unicode-key-code> [: <shifted> [: <base-layout>]]
         [; <modifiers> [: <event-type>]]
         [; <text-codepoints>] u

   modifiers = 1 + shift(1) + alt(2) + ctrl(4) + super(8)
                 + hyper(16) + meta(32) + caps_lock(64) + num_lock(128)
   event-type: 1 = press (default), 2 = repeat, 3 = release

Flag 1, which is 80% of the value

With disambiguate set, keys that had no unambiguous legacy encoding become explicit:

   Escape         1b            →  CSI 27 u        ← NO MORE TIMEOUT
   Ctrl+[         1b            →  CSI 91 ; 5 u    ← distinct from Escape
   Ctrl+I         09            →  CSI 105 ; 5 u   ← distinct from Tab
   Ctrl+M         0d            →  CSI 109 ; 5 u   ← distinct from Enter
   Ctrl+1         (nothing)     →  CSI 49 ; 5 u    ← now expressible
   Ctrl+Shift+A   01            →  CSI 97 ; 6 u    ← distinct from Ctrl+A

Implementing flag 1 alone removes the Escape ambiguity from the protocol. That is a real, noticeable improvement, and it is a few days of work.

The state

#![allow(unused)]
fn main() {
/// A STACK, not a flag. Programs push on entry and pop on exit, so nesting
/// works — which is why `vim` inside `tmux` inside your terminal can each have
/// their own settings.
pub struct KittyKeyboardState {
    /// Bounded: a program that pushes in a loop must not exhaust memory.
    /// kitty's own limit is small; 16 is generous.
    stack: Vec<KittyFlags>,
}

bitflags! {
    #[derive(Copy, Clone, Default, PartialEq, Eq)]
    pub struct KittyFlags: u8 {
        const DISAMBIGUATE     = 0b00001;
        const REPORT_EVENTS    = 0b00010;
        const ALTERNATE_KEYS   = 0b00100;
        const ALL_KEYS_ESCAPED = 0b01000;
        const ASSOCIATED_TEXT  = 0b10000;
    }
}

impl KittyKeyboardState {
    /// The CURRENT flags are the top of the stack, or empty. Empty means
    /// legacy encoding, byte for byte.
    pub fn current(&self) -> KittyFlags {
        self.stack.last().copied().unwrap_or_default()
    }
    pub fn push(&mut self, flags: KittyFlags) {
        const MAX_DEPTH: usize = 16;
        if self.stack.len() >= MAX_DEPTH { self.stack.remove(0); }   // drop the oldest
        self.stack.push(flags);
    }
    pub fn pop(&mut self, n: usize) {
        for _ in 0..n { self.stack.pop(); }
    }
}
}

The encoder

#![allow(unused)]
fn main() {
pub fn encode_key(key: Key, mods: Modifiers, event: KeyEventType,
                  modes: &TerminalModes) -> Option<Vec<u8>> {
    let flags = modes.kitty_keyboard.current();

    // NO flags → the legacy path, byte-identical. This branch must never
    // change behavior, or you break every program that does not opt in.
    if flags.is_empty() {
        if event != KeyEventType::Press { return None; }   // legacy: press only
        return legacy_encode_key(key, mods, modes);
    }

    // Release and repeat are only reported when the program asked.
    if event != KeyEventType::Press && !flags.contains(KittyFlags::REPORT_EVENTS) {
        return None;
    }

    // With only DISAMBIGUATE, keys that HAVE an unambiguous legacy encoding
    // keep it. Escaping everything would be flag 8's job, and doing it here
    // breaks programs that asked only for disambiguation.
    if flags == KittyFlags::DISAMBIGUATE && has_unambiguous_legacy_encoding(key, mods) {
        return legacy_encode_key(key, mods, modes);
    }

    Some(encode_kitty(key, mods, event, flags))
}
}

Warning: That "only DISAMBIGUATE → keep unambiguous legacy encodings" rule is the subtlety of the whole project. A terminal that escapes everything when only flag 1 is set will break vim, which enables flag 1 and still expects plain a to arrive as 0x61. Read kitty's specification on this point specifically; it is precise and the summary above is a summary.

4. Milestones

Milestone 2 is the whole value proposition. Ship it and stop, if you like.

5. The Tests

This project is a table, so it is a table-driven test.

#![allow(unused)]
fn main() {
#[test]
fn no_flags_means_byte_identical_legacy_output() {
    // The compatibility guarantee. Every legacy row must be unchanged.
    let modes = TerminalModes::default();
    for (key, mods, expected) in LEGACY_KEY_TABLE {
        assert_eq!(encode_key(*key, *mods, Press, &modes).as_deref(), Some(*expected),
                   "legacy encoding changed for {key:?}+{mods:?}");
    }
}

#[test]
fn disambiguate_separates_escape_from_ctrl_bracket() {
    // The bug this protocol exists to fix.
    let mut modes = TerminalModes::default();
    modes.kitty_keyboard.push(KittyFlags::DISAMBIGUATE);
    assert_eq!(encode_key(Named(Escape), NONE, Press, &modes).unwrap(), b"\x1b[27u");
    assert_eq!(encode_key(Char('['), CTRL, Press, &modes).unwrap(), b"\x1b[91;5u");
    // Distinct. Under legacy encoding both are 0x1b.
}

#[test]
fn disambiguate_keeps_unambiguous_legacy_encodings() {
    // THE subtle rule. Escaping everything under flag 1 breaks vim.
    let mut modes = TerminalModes::default();
    modes.kitty_keyboard.push(KittyFlags::DISAMBIGUATE);
    assert_eq!(encode_key(Char('a'), NONE, Press, &modes).unwrap(), b"a");
    assert_eq!(encode_key(Named(Enter), NONE, Press, &modes).unwrap(), b"\r");
}

#[test]
fn release_is_reported_only_with_flag_2() {
    let mut modes = TerminalModes::default();
    modes.kitty_keyboard.push(KittyFlags::DISAMBIGUATE);
    assert!(encode_key(Char('a'), NONE, Release, &modes).is_none());
    modes.kitty_keyboard.pop(1);
    modes.kitty_keyboard.push(KittyFlags::DISAMBIGUATE | KittyFlags::REPORT_EVENTS);
    assert_eq!(encode_key(Char('a'), NONE, Release, &modes).unwrap(), b"\x1b[97;1:3u");
}

#[test]
fn the_stack_nests_and_restores() {
    let mut s = KittyKeyboardState::default();
    assert_eq!(s.current(), KittyFlags::empty());
    s.push(KittyFlags::DISAMBIGUATE);                          // tmux
    s.push(KittyFlags::DISAMBIGUATE | KittyFlags::REPORT_EVENTS); // vim
    assert!(s.current().contains(KittyFlags::REPORT_EVENTS));
    s.pop(1);                                                   // vim exits
    assert_eq!(s.current(), KittyFlags::DISAMBIGUATE);
    s.pop(1);                                                   // tmux exits
    assert_eq!(s.current(), KittyFlags::empty());
}

#[test]
fn the_stack_is_bounded() {
    // A program pushing in a loop must not exhaust memory.
    let mut s = KittyKeyboardState::default();
    for _ in 0..100_000 { s.push(KittyFlags::DISAMBIGUATE); }
    assert!(s.depth() <= 16);
}

#[test]
fn query_reports_the_current_flags() {
    let mut t = Terminal::new(24, 80, cfg());
    t.advance(b"\x1b[>5u");         // push DISAMBIGUATE | ALTERNATE_KEYS
    t.advance(b"\x1b[?u");          // query
    assert_eq!(t.take_replies(), b"\x1b[?5u");
}

#[test]
fn modifier_encoding_matches_the_specification() {
    for (mods, param) in [(SHIFT, 2), (ALT, 3), (CTRL, 5), (CTRL|SHIFT, 6),
                          (CTRL|ALT|SHIFT, 8), (SUPER, 9)] {
        assert_eq!(kitty_modifier_param(mods), param);
    }
}
}

6. The Measurement

Two numbers, and one of them is subjective-made-objective.

1. Escape latency. The point of the project.

# Add artificial latency (Linux):
sudo tc qdisc add dev lo root netem delay 200ms
ssh localhost
#   In vim: press Escape then immediately 'i'. Time it, or just feel it.
#   Then enable the protocol and repeat.
sudo tc qdisc del dev lo root netem

Better, measure it: instrument your client to timestamp the Escape key press and the resulting action, and report the delta with and without flag 1, at 0/50/200/500 ms of induced latency. With flag 1 the timeout is gone entirely, so the delta is zero at every latency — that is the result, and it is a nice graph.

2. A compatibility matrix against kitty:

# Press every key combination in kitty's own debug mode, then in yours:
kitty +kitten show_key -m kitty      # kitty's reference output
cargo run -p raw-inspector           # inside your terminal, protocol enabled
diff <(...) <(...)

Every difference is a bug or a documented deviation.

7. Known Traps

Tip: The single most valuable half-day in this project is implementing flag 1 plus the query and nothing else. Programs check the query, discover disambiguation, and stop using timeouts. The Escape lag disappears. Everything after that is refinement.

Deliverables

• The flag stack with push/pop/set/query, bounded, and reset on RIS and exit.
• Flag 1 (disambiguate), with the "keep unambiguous legacy encodings" rule correct.
• Flag 2 (event types), at minimum press and release.
• Legacy output byte-identical when no flags are set — proven by the full legacy table test.
• Nesting verified through your terminal → tmux → vim.
• The latency measurement, with and without, at four induced latencies.
• The compatibility matrix against kitty's show_key.
• Documented precedence between kitty flags, modifyOtherKeys, and DECCKM.

Next: Project 6 — A WebAssembly Terminal Viewer