Lab 4: The Experiments

Background

Everything in Section 1 so far has been a claim. This lab is where the claims become observations.

Seven experiments, each in the same four-part shape: CLAIM, METHOD, PREDICTION, RESULT. Write the prediction before running the method. Every time. The gap between your prediction and the result is the actual content of this lab; a confirmed prediction teaches you almost nothing, and a wrong one tells you exactly which belief was false.

Keep a file experiments.md in your workspace. It is a required deliverable, and you will re-read it at the capstone.

Prerequisites

• Lab 3 working — several experiments run inside your runner.
• Three terminal windows open. Several experiments need a second and third observer window.

The Tools You Will Use

Note: Where a Linux-only tool has no macOS equivalent, the experiment says so and gives the nearest alternative. On macOS, SIP prevents tracing system binaries — trace your own programs instead, which is more useful anyway.

Experiment 1 — cat and Echo: Who Puts Characters on the Screen?

CLAIM. The kernel's line discipline performs echo. Neither the shell nor the terminal emulator is involved in making typed characters appear.

METHOD.

# 1a. Baseline: canonical mode, echo on.
cat
# type: hello   (do NOT press Enter yet)
# press Enter
# ^D to exit

# 1b. Turn echo off at the kernel level.
stty -echo
cat
# type: hello   (nothing appears)
# press Enter   (now "hello" appears — this is cat's OUTPUT, not echo)
# ^D
stty echo

# 1c. Prove it is per-terminal and lives in the kernel: from a SECOND window,
#     turn off echo on the FIRST window's tty.
#     Window A:
tty                          # → /dev/pts/5
#     Window B (Linux):
stty -echo -F /dev/pts/5
#     Window B (macOS):
stty -echo -f /dev/ttys005
#     Window A: type anything. Nothing appears. Your shell is unchanged; the
#     kernel simply stopped echoing.
#     Window B:
stty echo -F /dev/pts/5      # restore

PREDICTION. In 1b, when you type hello and press Enter: what appears on screen, in what order, and how many times does the word hello appear in total?

RESULT. Record it. Then answer: in 1a, why did hello appear twice on screen — once as you typed and once after Enter?

Tip: This is the experiment that settles the "who echoes" question permanently. If you got it right, you may move quickly through the rest of Section 1.

Experiment 2 — Canonical vs. Raw Mode

CLAIM. In canonical mode, read() does not return until a line terminator; in non-canonical mode it returns as soon as VMIN bytes are available.

METHOD.

# 2a. Canonical: cat prints nothing until Enter.
cat
# type "abc" slowly. Nothing from cat. Press Enter → "abc" appears.
# ^D

# 2b. Non-canonical with VMIN=1: cat prints each character immediately.
stty -icanon min 1 time 0
cat
# type "abc" slowly. Each character appears TWICE — once from kernel echo,
# once from cat writing it back out.
# ^C  (ISIG is still set, so this works)
stty sane

# 2c. Full raw: no echo, no signals.
stty raw -echo
cat
# type "abc" → appears ONCE (cat's output only).
# ^C → does nothing! It is byte 0x03 delivered as data.
# ^D → does nothing either; VEOF is not special in non-canonical mode.
# To escape: press Ctrl+J (a literal LF) then type: stty sane   then Ctrl+J
# (Enter may not work because ICRNL is off.)

# 2d. Observe the flags directly:
stty -a | tr ' ' '\n' | grep -E '^-?(icanon|echo|isig|opost|icrnl|ixon)$'
stty raw -echo
stty -a | tr ' ' '\n' | grep -E '^-?(icanon|echo|isig|opost|icrnl|ixon)$'
stty sane

PREDICTION. Before 2b: how many times will each character appear on screen, and why? Before 2c: what will Ctrl+C do, and what will Ctrl+D do?

RESULT. Record it, including exactly how you escaped from 2c.

Warning: You will get stuck in 2c at least once. That is intended. stty sane followed by Ctrl+J is the escape hatch — memorize it now, because you will need it again in Section 3.

Experiment 3 — Where Does Ctrl+C Go?

CLAIM. The byte 0x03 is converted to SIGINT by the kernel, and delivered to the terminal's foreground process group — not to the session leader, not to whoever is reading.

METHOD.

# Window B — watch continuously (adjust the tty):
watch -n 0.3 'ps -o pid,pgid,sid,tpgid,stat,comm -t pts/5'

# Window A:
echo "--- 1. baseline"
ps -o pid,pgid,sid,tpgid,stat,comm
#   TPGID == the shell's PGID → the shell is in the foreground.

echo "--- 2. run a foreground job"
sleep 100
#   In window B: TPGID has changed to sleep's PGID.
#   Press ^C → `sleep` dies. The SHELL does not.

echo "--- 3. a pipeline is ONE process group"
sleep 100 | cat
#   In window B: BOTH processes share one PGID, and TPGID equals it.
#   ^C kills both.

echo "--- 4. a background job is NOT in the foreground group"
sleep 100 &
ps -o pid,pgid,sid,tpgid,stat,comm
#   sleep's PGID != TPGID.
#   ^C now → the shell's foreground group gets SIGINT; `sleep` survives.
jobs
kill %1

Now trace it at the syscall level (Linux):

sleep 100 &
strace -p $! -e trace=none 2>&1 &          # attach, show only signals
# press ^C in the foreground... nothing.
kill -INT $!                                # explicit: strace shows SIGINT delivered

And prove that raw mode removes the conversion entirely:

# In your Lab 1 raw-inspector: ^C shows as byte 03 and the program lives.
cargo run -p raw-inspector

PREDICTION. In step 4, before pressing ^C: does the backgrounded sleep die? Which process receives SIGINT? What does it do with it?

RESULT. Record it, and write one sentence naming the exact kernel rule.

Experiment 4 — Resize and SIGWINCH

CLAIM. Resizing a window writes struct winsize into the kernel and sends SIGWINCH to the foreground process group. Programs must then ask for the new size; they are not told it.

METHOD.

# 4a. Watch the signal arrive.
trap 'echo "SIGWINCH → $(stty size)"' WINCH
# ...resize the window several times. Each resize prints once.
trap - WINCH

# 4b. Prove the size is stored per-terminal, in the kernel.
tty                                        # → /dev/pts/5
# Window B:
stty size -F /dev/pts/5                    # reports A's size

# 4c. Set the size from OUTSIDE, without touching the window.
# Window A: run `top` (or `vim`).
# Window B:
python3 - <<'EOF'
import fcntl, struct, termios
with open('/dev/pts/5', 'wb') as f:        # ← window A's tty
    fcntl.ioctl(f, termios.TIOCSWINSZ, struct.pack('HHHH', 12, 40, 0, 0))
EOF
# Window A: `top` immediately re-lays out at 12x40, even though the WINDOW
# is still the old size. The program believes the kernel, not the pixels.

# 4d. The two-hop chain in your own runner.
cargo run -p pty-runner
#   inside: watch -n0.3 'stty size'
#   resize the outer window → the inner size follows.
#   Now comment out your SIGWINCH branch, rebuild, and repeat: it does not.

# 4e. What a resize-unaware program looks like. Inside your runner:
#   printf 'x%.0s' $(seq 1 200); echo
#   Resize narrower, then run it again. Note where wrapping happens.

PREDICTION. In 4c: does the terminal window change size? Does top redraw? What happens the next time you resize window A with the mouse — does the forced size persist?

RESULT. Record it, and explain in one sentence why the kernel's winsize and the emulator's actual window can disagree.

Experiment 5 — Job Control with sleep 100

CLAIM. ^Z, bg, and fg are implemented by the shell manipulating process groups and the terminal's foreground group, using SIGTSTP/SIGCONT and tcsetpgrp.

METHOD.

# Window B: keep this running against window A's tty.
watch -n 0.3 'ps -o pid,ppid,pgid,sid,tpgid,stat,comm -t pts/5'

# Window A, one step at a time. After EACH step, read window B before continuing.

sleep 100
#   STAT: S+   (the + means "in the foreground process group")
#   TPGID == sleep's PGID

# ^Z
#   STAT: T    (stopped)
#   TPGID has returned to the shell's PGID  ← the shell called tcsetpgrp
jobs
#   "[1]+  Stopped   sleep 100"

bg
#   STAT: S    (running, but NO '+' — background)
#   TPGID still the shell's
jobs

fg
#   STAT: S+   again
#   TPGID == sleep's PGID again

# ^C → gone.

Then the SIGTTIN demonstration:

cat &
#   Immediately: "[1]+  Stopped (tty input)"
#   The kernel sent SIGTTIN because a BACKGROUND process tried to read the terminal.
ps -o pid,pgid,tpgid,stat,comm
#   STAT: T
fg
#   now it can read. Type something, then ^D.

And SIGTTOU, which is normally invisible:

stty tostop                    # turn ON stop-on-background-write
(sleep 1; echo "from the background") &
#   The job is stopped instead of writing over your prompt.
jobs
fg
stty -tostop                   # restore the default

PREDICTION. Before running cat &: does it (a) read your next keystroke, (b) exit immediately, (c) stop, or (d) error? Before stty tostop: why is that not the default?

RESULT. Record the TPGID value at every step. The sequence of TPGID changes is job control.

Experiment 6 — Pipes vs. PTY

CLAIM. A shell connected by pipes loses interactivity, job control, terminal queries, and signal semantics — even though bytes still flow correctly in both directions.

METHOD.

# 6a. The blunt demonstration.
echo "ls; echo done" | bash
#   Works. No prompt. No colors from ls. No job control message... or is there?

bash -c 'echo $-'                     # option flags: no 'i' → not interactive
echo 'echo $-' | bash                 # same

# 6b. isatty, directly.
test -t 0 && echo "stdin IS a tty" || echo "stdin is NOT a tty"
echo '' | { test -t 0 && echo "stdin IS a tty" || echo "stdin is NOT a tty"; }

# 6c. What programs change when piped.
ls --color=auto | cat                 # colors gone (GNU); try `ls -G | cat` on macOS
ls --color=always | cat | cat -v      # forced: you can SEE the escape sequences

# 6d. Terminal-only ioctls fail.
stty size                             # works
stty size < /dev/null                 # "Inappropriate ioctl for device" (ENOTTY)

# 6e. Interactive programs refuse or misbehave.
echo q | top                          # Linux: "top: failed tty get"  or similar
echo ':q' | vim                       # "Vim: Warning: Input is not from a terminal"
echo | less                           # behaves like `cat`

# 6f. Now compare against a PTY, using `script` as a reference implementation:
script -q -c 'ls --color=auto' /dev/null | cat -v    # colors ARE present
script -q -c 'test -t 0 && echo TTY || echo PIPE' /dev/null

6g — the one you write. Add a --pipes mode to your Lab 2 runner that uses Stdio::piped() instead of a PTY, and try each of the above inside it.

PREDICTION. Write down four specific differences you expect between piped-bash and PTY-bash, before running anything. Most people get one or two.

RESULT. Fill in the full table:

Experiment 7 — The Inspection Tour

CLAIM. Every claim in Section 1 is directly observable with standard tools. You should never have to guess.

METHOD. Start your runner, then from another window investigate it completely.

# Find the pieces.
pgrep -a pty-runner
RUNNER=$(pgrep -n pty-runner)
CHILD=$(pgrep -P $RUNNER)                # bash inside

echo "=== 1. Process relationships ==="
ps -o pid,ppid,pgid,sid,tpgid,stat,tty,comm -p $RUNNER,$CHILD
#   The runner: your OUTER tty. The child: a DIFFERENT pts. Different SIDs.

echo "=== 2. Which fds does the runner hold? ==="
ls -l /proc/$RUNNER/fd                    # Linux
lsof -p $RUNNER                           # both
#   Look for: 0,1,2 → your outer tty;  a higher fd → /dev/ptmx (the master).
#   If you ALSO see /dev/pts/N here, you forgot to close the slave in the parent.

echo "=== 3. Which fds does the child hold? ==="
ls -l /proc/$CHILD/fd
#   0,1,2 → /dev/pts/N, all the same device. And NO ptmx — the child must not
#   hold the master.

echo "=== 4. Who else has this pts open? ==="
lsof /dev/pts/9                           # substitute the real number

echo "=== 5. Line discipline of the inner PTY ==="
stty -a -F /dev/pts/9 | head -5           # Linux
#   Run `vim` inside the runner and re-run this: the flags change dramatically.

echo "=== 6. The controlling terminal relationship ==="
ps -o pid,sid,tty,tpgid -p $CHILD
cat /proc/$CHILD/stat | awk '{print "tty_nr="$7, "tpgid="$8}'   # Linux

echo "=== 7. Syscalls, live ==="
# Linux:
sudo strace -f -p $RUNNER -e trace=read,write,poll,ioctl
#   Press one key in the runner and count the syscalls. You should see FOUR:
#     read(0) → write(master) → read(master) → write(1)
#   That is the echo round trip, made visible.
# macOS:
sudo dtruss -f -p $RUNNER 2>&1 | grep -E 'read|write|kevent|ioctl'

PREDICTION. Before step 7: how many syscalls does your runner make when you press a single ordinary letter key? Write the exact sequence.

RESULT. Record the actual sequence. If it is more than four, explain each extra one.

Tip: Step 7 is the single most valuable observation in Section 1. It is the entire mental model, rendered as syscalls, on your own machine, for one keystroke. Save the output.

Deliverables

• experiments.md with all seven experiments, each containing CLAIM, METHOD, written PREDICTION, RESULT, and — where you were wrong — one sentence naming the false belief.
• The completed pipes-vs-PTY table from Experiment 6.
• The TPGID value at every step of Experiment 5.
• Saved strace/dtruss output from Experiment 7 showing one keystroke's round trip.
• A one-paragraph summary: which prediction were you most wrong about, and what belief caused it?

Validation / Self-check

1. Which component echoes, and what single command proves it?
2. In canonical mode, how many bytes does read() return when you type abc and press Enter?
3. Name every process that receives SIGINT when you press ^C during sleep 100 | cat.
4. Why does a background cat stop immediately? Name the signal and the rule.
5. Why is TOSTOP off by default, and what changes when you enable it?
6. You forced a TIOCSWINSZ from another window. Why did top believe it even though the window did not change?
7. Give four concrete behavioral differences between a piped shell and a PTY shell.
8. In lsof output for a correct PTY runner, which process holds ptmx and which holds pts/N? What does it mean if one process holds both?
9. How many syscalls does one keystroke cost in a naïve relay, and which are they?
10. What does stty -a -F /dev/pts/N show differently while vim is running, and why?

Next: Lab 5 — The Session Recorder, where you make all of this replayable.