The Teaching Method

This chapter describes the shape of every concept chapter and every lab in this curriculum. Read it once so the structure is not a surprise, and come back to it when you are designing your own experiments — because by Section 5 you will be, and the shapes here are the ones that work.


The Nine-Part Concept Treatment

Every concept chapter presents each concept in nine parts, in this order. The order is not decorative: parts 1–3 build the model, part 4 builds the thing, parts 5–7 make the design a choice you made rather than a thing that happened to you, and parts 8–9 connect it to reality.

#PartThe question it answersWhat goes wrong if it is missing
1ConceptWhat is it?You learn a word without a referent.
2ProblemWhat problem does it solve?You implement a mechanism with no idea when not to.
3Mental modelHow should I picture it?You can recite it and cannot debug it.
4ImplementationHow do we build it in Rust?It stays theory.
5AlternativesWhat else could we have done?You mistake one design for the only design.
6DecisionWhy did we choose ours?You cannot defend it in a review, and you cannot revisit it.
7TradeoffsWhat do we gain and lose?You are surprised by the cost later and call it a bug.
8Production concernsWhat goes wrong in a real runtime?You ship the demo version.
9ReferencesWhat should I read?You reinvent 1984.

Parts 5, 6, and 7 are the ones that distinguish this curriculum from a tutorial. A tutorial tells you what to type. An apprenticeship tells you what you are choosing. If a chapter presents a design without alternatives, treat that as a defect in the chapter and go find the alternatives yourself — they exist for everything here.

Note: Not every concept gets all nine parts at full length. A short concept may compress 5–7 into a three-row table. But no concept in this curriculum has zero alternatives, and a chapter that skips them is a chapter that failed.


The Twelve-Part Lab Step

Each implementation step inside a lab contains these twelve parts. When a step omits one, it is because the step genuinely has none — not as a shortcut.

┌──────────────────────────────────────────────────────────────────┐
│  1. The concept being learned        "what is this step for?"    │
│  2. The smallest goal                one sentence                │
│  3. The expected observable behavior what you will SEE           │
│  4. The Rust code                    small enough to read        │
│  5. Line-by-line explanation         of the parts that matter    │
│  6. Commands to run                  copy-pasteable              │
│  7. Expected output                  so you know if it worked    │
│  8. Debugging steps                  for when it did not         │
│  9. One experiment                   a claim you could disprove  │
│ 10. One test                         that survives refactoring   │
│ 11. One challenge extension          to go past the lesson       │
│ 12. A checkpoint question            to verify understanding     │
└──────────────────────────────────────────────────────────────────┘

Avoid large unexplained code dumps. If a code block is longer than about 80 lines it is followed by a walkthrough of the parts that carry meaning. If you find yourself copying a block you cannot annotate, stop and annotate it — the annotation is the exercise.


The Trace: The Signature Element

Every lab contains a section titled ## The Trace. It takes one concrete program and shows it in every representation that exists at that point in the curriculum. This is the mechanism by which the runtime stays observable instead of magical, and it is the thing you should copy into your own projects for the rest of your career.

The Trace grows as the curriculum does:

After labThe Trace shows
1characters → tokens
2characters → tokens → AST
3… → evaluation order → value
9… → bytecode + constant pool
11… → VM stack, one line per instruction
13… → table array/hash split, and which part each key landed in
14… → the open-upvalue list, and the moment each closes
15… → heap census, root set, mark bits, what got swept
19… → the marshaling boundary and the value that reaches Rust
28… → cache state at each access site: monomorphic / polymorphic / megamorphic
30… → Cranelift IR and the guard that protects it

By the capstone, the Trace is a single document that follows one script through all of it. That document is the mandatory end-to-end trace, and it is the strongest evidence that you understand the system.

The rule: reading a Trace in this book is worth about a tenth of producing one from your own code. Every Trace here is reproducible with a command. Run it.


The Predict-First Protocol

You will be asked to predict behavior before revealing the result. This is not a gimmick. Prediction turns a passive read into a test of your model, and the gap between prediction and observation is where learning happens. A confirmed prediction teaches you almost nothing; a wrong one teaches you exactly which belief was false.

  1. Read the question.
  2. Write your prediction down — file, comment, paper. Writing is required. A prediction kept in your head silently rewrites itself when you see the answer. That is hindsight bias and you are not immune to it.
  3. Include your confidence: high / medium / guessing.
  4. Run the experiment.
  5. If you were wrong, write one sentence naming the false belief. Not "I forgot" — the actual belief.

Keep these in docs/learning/predictions.md. You review it at the capstone.

Representative questions

  • What is size_of::<Value>(), and why is it not 9 bytes?
  • local x = x — which x does the initializer read?
  • After local a = {1}; local b = a, how many objects are on the heap?
  • A closure captures a local. The function returns. Where does the value physically live now?
  • You forget to trace table keys in the collector. What is the first symptom, and after how long?
  • while true do end with a 10-million-instruction budget. Where exactly does it stop?
  • A registered Rust function calls back into the VM, which calls the same Rust function. What does the borrow checker say, and is it right?
  • An inline cache is warm. Someone assigns a new field to the table. What must happen?

Warning: The four most commonly mispredicted are: where a captured variable lives after the frame dies; why cycles defeat reference counting specifically (people know the fact and cannot draw it); what (f()) does; and what a JIT guard is guarding. If you predict all four correctly, you may move faster through Sections 4 and 7.


What "Instrumentation" Means Here

Instrumentation is not println! scattered in a loop. It is a deliberate, switchable view of one layer. Every layer gets one, and they are the six subcommands plus five flags:

LayerInstrumentationEnabled by
LexerToken dump with spans and source textember tokens
ParserIndented tree with spansember ast
CompilerDisassembly with constants, line table, and local namesember disassemble
VMip, opcode, stack before/after, frame depthember trace
FramesFrame push/pop with base and return addressember trace --frames
UpvaluesOpen list contents; close eventsember trace --upvalues
HeapCensus by type, bytes, live/dead countsember --stats, collectgarbage("count")
GCBegin/mark/sweep with durations and byte deltas--trace-gc
TablesArray/hash split, rehash events, load factor--trace-tables
CachesPer-site state transitions and miss reasons--trace-ic (§7)
Host boundaryEvery marshal in and out, with types--trace-host (§5)

Four design rules, learned the hard way:

  1. It must not corrupt the thing it observes. Debug output goes to stderr, never stdout — because stdout is what the golden tests compare. A --trace-gc that broke the test suite is a real thing that happens.
  2. It must be switchable at run time, not by recompiling. A flag or an environment variable.
  3. It must be cheap when off. if self.trace { ... } around the formatting, not a formatted string that gets discarded. In the dispatch loop this is measurable; Section 7 measures it.
  4. It must be replayable. Anything you can dump, you can feed back in. The disassembler reads a Chunk; the golden tests read .ember files; the fuzz corpus is committed.

What "One Experiment" Means Here

An experiment has four parts, and it is not an experiment without all four:

CLAIM        A falsifiable statement about the system.
             "Local variable access in the VM does not depend on the number of locals in scope."

METHOD       The exact commands, in order, that would show it.
             "bench a function with 1, 8, and 64 locals reading the last one; compare ns/iter."

PREDICTION   What you expect to observe, written before you run it.

RESULT       What you observed, and — if it differs — which belief was wrong.

Every experiment in this book is written in that shape. When you invent your own, keep it. This is how you will debug a performance problem in production in five years.


What "One Test" Means Here

A test in this curriculum is focused: it asserts one behavior, it names the language rule it is about, and it fails informatively.

#![allow(unused)]
fn main() {
#[test]
fn exponent_is_right_associative() {
    // 2 ^ 3 ^ 2  must parse as  2 ^ (3 ^ 2) = 512, not (2 ^ 3) ^ 2 = 64.
    // Lua 5.4 §3.4.1: `^` is right-associative and binds tighter than unary operators
    // on its LEFT but looser on its RIGHT: -2 ^ 2 == -(2 ^ 2) == -4.
    assert_eq!(eval("return 2 ^ 3 ^ 2"), Value::Float(512.0));
    assert_eq!(eval("return -2 ^ 2"),    Value::Float(-4.0));
}
}

Four properties every test here has:

  1. It names the rule in a comment, with the source of the rule where one exists. Six months later 512.0 is a magic number; "right-associative per §3.4.1" is not.
  2. It asserts one thing. A test that checks the value and the type and the span tells you nothing when it fails.
  3. Its failure message states the rule, not just the numbers.
  4. It survives a refactor, because it goes through the public API — eval(), not parser.parse_binary_internal().

And one property specific to this curriculum:

  1. From Lab 12 onward, every behavioral test runs against both backends. You do not write differential tests as a separate activity; you write normal tests, and the harness runs them twice. That is the design that makes the guarantee free.

For every language feature you implement, the curriculum asks for five things:

  1. The source construct.
  2. The AST it produces.
  3. The bytecode it compiles to.
  4. A focused test.
  5. The corresponding Lua 5.4 behavior, confirmed by actually running lua, and a note in appendix/lua-differences.md if you diverge.

That fifth item is the one people skip, and it is the one that catches misunderstandings. If you cannot make real Lua exhibit the behavior you are implementing, you may have invented it.


What "One Challenge Extension" Means Here

Each lab ends with something past the lesson: a feature the lab did not need, a performance problem the naive version has, or a correctness edge the happy path avoided. Optional, but where the depth compounds. A representative sample:

  • Make the lexer zero-allocation for identifiers and prove it with a benchmark.
  • Add error recovery to the parser so one syntax error does not hide the next five.
  • Implement constant folding in the compiler and measure it on the golden corpus.
  • Make the table's array part shrink on nil assignment, and explain why Lua does not.
  • Implement __gc finalizers, then explain why Ember's default build refuses to.
  • Add a write barrier and turn the collector incremental. (Genuinely hard. That is the point.)

The ADR Discipline

You write fourteen Architecture Decision Records, listed in the appendix. Four headings, under 400 words:

# ADR-00N: <decision, in the imperative>

## Context
What forced the decision. What was known and unknown at the time.

## Options
A, B, C — each with its real cost. A strawman option is worse than no ADR.

## Decision
One sentence, active voice: "We use X."

## Consequences
What this makes easy, what it makes hard, and the observation that would make us revisit.

Three rules:

  1. Write it at the moment of the decision. A reconstructed rationale is fiction, and you will not be able to tell the difference in six months.
  2. Record the options you rejected, with their real costs. The value of an ADR is almost entirely in the rejected options — that is the part you cannot reconstruct.
  3. Never edit a decision; supersede it. A later ADR opens with "supersedes ADR-00N"; the original stays in the directory, unedited. The history is the asset.

Common Failure Modes of Learners (Not of Code)

Failure modeSymptomCorrection
Reaching for a crate too earlylogos or chumsky in Cargo.toml in week 2You skipped the point of Section 1. Delete it, write it by hand, then add it back and diff.
Building three layers before running any"It compiles" as a status reportNothing is done until you have observed it. Run the Trace.
Debugging by guessingRandom edits, recompilesBisect by representation: tokens, AST, bytecode, stack. If you cannot name the layer, that is the real problem.
Deleting the tree walker"It's dead code now"It is the oracle. See ADR-003. This is the single most expensive mistake available in this curriculum.
Optimizing before measuringSuperinstructions in week 9You have no baseline. Any change you make now is unfalsifiable.
Skipping the experiments"I understood it from the text"You did not. The text is the hypothesis; the experiment is the evidence.
Treating the GC as "the hard part to get through"A collector you cannot explain, that mostly worksIt mostly works now. Missing-root bugs surface under memory pressure in production, months later, as impossible-looking corruption.
Reading Section 7 firstBeautiful ideas, nothing to apply them toOptimizations are conclusions drawn from measurements of a working system.
Copying Lua without checking"Lua does it this way" from memoryRun lua. Every claim in this curriculum about Lua's behavior is one you can verify in ten seconds, and you should.

How to Ask Yourself a Debugging Question

When something is wrong, ask in this order. This ordering is the single most transferable skill in the curriculum.

1. WHICH REPRESENTATION IS WRONG?
   → ember tokens | ember ast | ember disassemble | ember trace
     Bisect. Do not read code until you know which layer.

2. COMPILE TIME OR RUN TIME?
   → If the disassembly is wrong, the VM is innocent. If the disassembly is right
     and the answer is wrong, the compiler is innocent.

3. DO THE TWO BACKENDS AGREE?
   → ember run --interp  vs  ember run
     Disagreement localizes the bug to one of two implementations of a KNOWN spec.
     Agreement means your spec itself is wrong — a much more interesting bug.

4. IS IT MEMORY?
   → --trace-gc, and check whether the bug disappears when you disable collection.
     If it does, you have a missing root or a missing edge, and nothing else.

5. IS IT MY BUG OR A MISSING FEATURE?
   → Run the same program under `lua`. If Lua does something different, decide
     deliberately which behavior you want, then write it down in
     appendix/lua-differences.md. An undocumented divergence is a bug either way.

Step 3 is the one this curriculum buys you that most language projects do not have. Protect it.


Validation / Self-check

  1. Name the nine parts of the concept treatment, and say which three make it an apprenticeship rather than a tutorial.
  2. What is the Trace, what makes it mandatory, and how does it grow across the curriculum?
  3. Why must a prediction be written down rather than held in your head?
  4. Give the four parts of an experiment. Which one do people skip?
  5. Name the five things this curriculum asks for with every language feature. Which is most often skipped, and what does skipping it cost?
  6. What are the four rules for instrumentation, and which one is violated by writing --trace-gc output to stdout?
  7. State the three ADR rules. Why is "never edit, always supersede" the important one?
  8. Give the five-step debugging order, and apply it to: "fib(20) returns the right answer under --interp and the wrong one under the VM."

Next: Section 1 — From Characters to Trees. The work starts there.