Why a Multiplexer Needs a Server Process

This chapter answers, precisely: why does tmux need a background server, and why do shells keep running when the client disconnects? The answer is short, mechanical, and follows entirely from the job-control chapter.

The Mechanism, in Full

Without a multiplexer

 ┌────────────────────────────────────────────────────────────────┐
 │  Terminal emulator process                                     │
 │    owns:  PTY master fd  ────────────────┐                     │
 └──────────────────────────────────────────│─────────────────────┘
                                     ═══════│═══ KERNEL ═══
                                    ┌───────▼────────────┐
                                    │  PTY pair          │
                                    │  slave = /dev/pts/7│
                                    └───────┬────────────┘
                                     ═══════│═══ USER SPACE ═══
 ┌──────────────────────────────────────────▼─────────────────────┐
 │  bash — SESSION LEADER, ctty = /dev/pts/7                      │
 │    └── vim, make, ssh ... (its jobs, in their own pgroups)     │
 └────────────────────────────────────────────────────────────────┘

 YOU CLOSE THE WINDOW:
   1. The emulator process exits.
   2. Its fds close, including the PTY master.
   3. The kernel sees the master side gone → the PTY "carrier drops".
   4. The kernel sends SIGHUP to the SESSION LEADER of the session whose
      CONTROLLING TERMINAL this is — i.e. to bash.
   5. bash's default SIGHUP handling: send SIGHUP to each of its jobs, then exit.
   6. Everything dies.

That is not a bug. It is the correct emulation of a serial line being unplugged, which is what SIGHUP — "hang up" — literally means. Your modem hung up; the session is over.

With a multiplexer

 ┌──────────────────┐        ┌────────────────────────────────────────────────┐
 │  mux CLIENT      │◀──────▶│  mux SERVER (a daemon, no controlling terminal) │
 │  owns: a socket  │ socket │    owns:  PTY-B master ─────┐                   │
 │  and the user's  │        │           PTY-C master ──┐  │                   │
 │  own terminal    │        │           Terminal state │  │                   │
 └──────────────────┘        └──────────────────────────│──│───────────────────┘
                                                ════════│══│═══ KERNEL ═══
                                          ┌─────────────▼──▼──────────┐
                                          │  PTY pairs B and C        │
                                          └─────────────┬──┬──────────┘
                                                ════════│══│═══ USER SPACE ═══
                                          ┌─────────────▼──▼──────────┐
                                          │  bash (pane 0), vim (p1)  │
                                          └───────────────────────────┘

 YOU CLOSE THE WINDOW:
   1. The CLIENT process exits.
   2. Its fds close: the socket, and the client's own terminal.
   3. The server's `accept`ed socket reports EOF. The server removes the client.
   4. NOTHING ELSE HAPPENS.
      • No PTY master was closed.
      • No carrier dropped.
      • No SIGHUP was generated.
      • The panes' shells did not notice.
   5. The server keeps reading the pane masters and feeding its per-pane
      Terminals, exactly as before.

One sentence: the shells survive because nobody closed the PTY master.

Why the Server Must Be a Separate Process

Could the server be a thread inside the GUI? No, and the reasons compound:

The last row is the sharpest test: if your architecture cannot create a session with no display attached, you have not built a multiplexer.

Why the Server Must Daemonize Correctly

The server is usually started by a client, from your terminal. If it does not detach properly, it inherits your controlling terminal and dies with it — reintroducing the exact bug it exists to prevent.

#![allow(unused)]
fn main() {
/// Daemonize. Each step exists to sever one tie to the launching terminal.
fn daemonize() -> io::Result<()> {
    // 1. fork and let the parent exit. The child is now an orphan, reparented
    //    to init/systemd, and — crucially — is NOT a process group leader,
    //    which is a precondition for setsid().
    match unsafe { libc::fork() } {
        -1 => return Err(io::Error::last_os_error()),
        0 => {}                                   // child continues
        _ => unsafe { libc::_exit(0) },           // parent exits immediately
    }

    // 2. New session. The child becomes session leader AND DROPS the inherited
    //    controlling terminal. Without this, closing the launching terminal
    //    sends SIGHUP to the server — the very bug we exist to avoid.
    if unsafe { libc::setsid() } < 0 { return Err(io::Error::last_os_error()); }

    // 3. Fork AGAIN. The first child is a session leader, and a session leader
    //    that opens a tty can ACQUIRE it as a controlling terminal. The second
    //    child is not a session leader, so it can never accidentally acquire one.
    match unsafe { libc::fork() } {
        -1 => return Err(io::Error::last_os_error()),
        0 => {}
        _ => unsafe { libc::_exit(0) },
    }

    // 4. Working directory: do not hold a mount busy.
    unsafe { libc::chdir(b"/\0".as_ptr() as *const _) };

    // 5. Redirect 0/1/2 to /dev/null. Leaving them on the old terminal means
    //    a stray println! scribbles on the user's screen, and a write to a
    //    closed terminal gets EIO or SIGPIPE.
    let devnull = unsafe { libc::open(b"/dev/null\0".as_ptr() as *const _, libc::O_RDWR) };
    for fd in 0..=2 { unsafe { libc::dup2(devnull, fd) }; }
    if devnull > 2 { unsafe { libc::close(devnull) }; }

    // 6. SIGHUP: ignore it anyway, as belt and braces.
    unsafe { libc::signal(libc::SIGHUP, libc::SIG_IGN) };
    // SIGPIPE: MANDATORY. A client that disappears mid-write would otherwise
    // kill the server with the default disposition. Handle EPIPE instead.
    unsafe { libc::signal(libc::SIGPIPE, libc::SIG_IGN) };
    Ok(())
}
}

Warning: The double fork is the step people omit and cannot explain. Here is the reason: a session leader with no controlling terminal that later open()s a tty device can acquire it as its controlling terminal (on systems where O_NOCTTY is not used, and even with it in some paths). Forking a second time yields a process that is not a session leader, so that acquisition is impossible. It costs one fork and removes a whole class of "my daemon died when I closed the terminal" bugs.

Warning: Ignoring SIGPIPE is not optional. Its default disposition is to terminate the process. A server that writes to a client which just vanished would die — taking every session with it. Ignore the signal and handle EPIPE from write as "this client is gone."

Why the Server Needs a Terminal Emulator Per Pane

This is the second insight, and it follows from the first.

   You detach. `top` in pane 0 keeps running — nothing told it to stop.
   It writes a screen update every second.

   ┌── Who reads those bytes? ────────────────────────────────────────┐
   │                                                                  │
   │  IF NOBODY READS:                                                │
   │    The PTY output buffer fills (a few KB).                       │
   │    `top` BLOCKS in write().                                      │
   │    It stops updating. Your "persistent session" is frozen.       │
   │                                                                  │
   │  IF SOMEBODY READS BUT ONLY BUFFERS THE RAW BYTES:               │
   │    Detach for an hour → hundreds of MB of buffered output.       │
   │    On reattach, you would replay all of it to reconstruct the    │
   │    screen. Slow, memory-hungry, and WRONG — see below.           │
   │                                                                  │
   │  IF SOMEBODY READS AND PARSES INTO A TERMINAL:                   │
   │    Constant memory: one screen + bounded scrollback per pane.    │
   │    On reattach: repaint from state. Instant, correct.            │
   │    ← THIS. This is why the server contains an emulator.          │
   └──────────────────────────────────────────────────────────────────┘

Why replay is not merely slow — it is wrong

   Pane 0 ran `vim`, which entered the ALTERNATE SCREEN, drew, and exited,
   restoring the primary screen.

   REPLAY of the raw byte log reproduces: enter alt screen → draw → leave.
   Fine, if you replay ALL of it.

   But suppose you bound the log to the last 1 MB, and `vim` ran two hours ago.
   Now the log STARTS in the middle of the alternate screen with a scroll region
   set and application cursor keys enabled — and no ?1049h to establish that
   context. The replayed screen is garbage.

   Parsing into state has no such problem: the state IS the answer, and it is
   the same size whether you were detached for a second or a week.

That argument generalizes: replay reconstructs history; state reconstructs the present. A reattaching client wants the present.

The Client Owns Nothing Durable

A correct client can be kill -9'd at any moment with zero consequence beyond a dropped connection. That is the test.

Tip: Make it a real test. kill -9 the client in CI and assert with ps that the pane children are still alive. It is a three-line test that guards the entire architecture.

Why This Also Explains nohup, disown, and screen

All four tools solve the same SIGHUP problem, at different layers:

Note the difference in kind: the first three cope with the signal; tmux prevents it from existing. That is why tmux also preserves interactivity — the others let a process survive but leave it with no terminal to talk to.

Experiment

CLAIM. The shells survive because the master fd stayed open, and this can be demonstrated directly with lsof.

METHOD.

# 1. Start a real tmux session and find the pieces.
tmux new-session -d -s demo 'sleep 3000'
tmux ls
SERVER=$(pgrep -n tmux)
CHILD=$(pgrep -n sleep)

# 2. Who holds the PTY master? (The server.)
lsof -p $SERVER 2>/dev/null | grep -E 'ptmx|pts'
#    You will see /dev/ptmx (the master) held by the SERVER.

# 3. Who holds the slave? (The child.)
ls -l /proc/$CHILD/fd 2>/dev/null || lsof -p $CHILD | head
#    0,1,2 → /dev/pts/N, and NO ptmx.

# 4. Sessions and controlling terminals:
ps -o pid,ppid,pgid,sid,tty,comm -p $SERVER,$CHILD
#    The SERVER's TTY is "?" — it has no controlling terminal. That is the
#    daemonization working.
#    The CHILD's TTY is a pts — its own, from the mux.

# 5. Attach, then kill the CLIENT hard:
tmux attach -t demo &
sleep 1
pkill -9 -f 'tmux attach'
ps -p $CHILD                  # STILL RUNNING

# 6. Now contrast: kill the SERVER.
kill -9 $SERVER
sleep 1
ps -p $CHILD                  # GONE — the master closed, SIGHUP was delivered

PREDICTION. Before step 6: what is the child's PPID after the server dies? What signal killed it, and who sent it?

RESULT. Record the lsof output showing master and slave in different processes. That output is the mechanism, made visible.

Test

#![allow(unused)]
fn main() {
#[test]
fn killing_the_client_leaves_children_running() {
    // THE architectural test. Three lines, and it guards everything.
    let server = MuxServer::spawn_daemon().unwrap();
    let mut client = MuxClient::attach(&server.socket_path()).unwrap();
    let pane = client.new_pane("sleep 300").unwrap();
    let child_pid = server.pane_child_pid(pane);

    client.kill_hard();                     // SIGKILL: no cleanup at all
    std::thread::sleep(Duration::from_millis(500));

    assert!(process_exists(child_pid), "the pane child must survive its client");
    assert!(server.is_running());
}

#[test]
fn server_has_no_controlling_terminal() {
    let server = MuxServer::spawn_daemon().unwrap();
    // A server with a controlling terminal dies when that terminal closes.
    assert_eq!(controlling_terminal_of(server.pid()), None);
    assert_eq!(session_id_of(server.pid()), server.pid(),
               "the server must be its own session leader");
}

#[test]
fn detached_panes_keep_being_parsed() {
    // Not just "still running" — the server must PARSE their output, or the
    // child blocks in write() once the PTY buffer fills.
    let server = MuxServer::spawn_daemon().unwrap();
    let pane = server.new_pane("bash -c 'for i in $(seq 1 100000); do echo $i; done; sleep 60'");
    std::thread::sleep(Duration::from_secs(2));
    let snap = server.pane_snapshot(pane);
    assert!(snap.contains("100000"), "the server must drain and parse while detached");
}

#[test]
fn server_survives_a_client_vanishing_mid_write() {
    // SIGPIPE's default disposition terminates the process. Ignoring it is
    // mandatory, or one rude client takes down every session.
    let server = MuxServer::spawn_daemon().unwrap();
    let client = MuxClient::attach(&server.socket_path()).unwrap();
    server.new_pane("yes");                  // flood the client
    client.close_socket_abruptly();
    std::thread::sleep(Duration::from_millis(500));
    assert!(server.is_running());
}
}

Validation / Self-check

1. In one sentence, why does tmux keep your shells alive?
2. Trace SIGHUP from closing a terminal window to a job dying: name each signal, its sender, and its receiver.
3. Why can the server not be a thread inside the GUI? Give three reasons, and name the sharpest test.
4. Explain the double fork in daemonization. What does the second one prevent?
5. Why must the server ignore SIGPIPE? What is the default disposition?
6. Why does the server need a terminal emulator per pane? What breaks with each of the three alternatives?
7. Why is replaying a bounded byte log wrong, not merely slow? Give the concrete failure.
8. List what the client owns and what the server owns. What is the test that the split is right?
9. Compare nohup, disown, setsid, and tmux. Which prevents SIGHUP rather than coping with it?
10. Which lsof output would prove your architecture is correct?

Next: The Client/Server Protocol.