Section 7: Making It Fast

You have a correct, tested, sandboxed, observable runtime and a benchmark suite with a committed baseline. Now, and only now, you may optimize.

This section is short on techniques and long on discipline. Every change follows the protocol: baseline, hypothesis, change, measurement, tradeoff, decision — recorded either way, including the reverts. Especially the reverts.

It completes Milestone M14.

The Order Is Decided by Your Profile, Not by This Book

Lab 27's opcode profile tells you what to build first. The chapters below are in the order that a typical policy-engine profile suggests, and if yours differs, follow yours.

If your profile says	Build	Chapter
`GET_FIELD` / `GET_INDEX` dominates	inline caches	Inline Caches
`ADD`/`LT`/`MUL` dominate with uniform types	specialized opcodes	Runtime Specialization
`CALL` dominates	call-path work, then a JIT	JIT Architecture
`GET_LOCAL` dominates	superinstructions	Lab 29's challenge

Most people build ADD_INT first because it is the example everyone uses. Most profiles say field access. That gap is the reason Lab 27 exists.

The Three Ideas

Everything in this section is one of three moves, and they compose in one direction:

   1. CACHE THE ANSWER          "this site got the same shape last time"
      inline caches                          ↓
   2. SPECIALIZE THE CODE       "this instruction always sees integers"
      adaptive opcodes                       ↓
   3. COMPILE THE ASSUMPTION    "generate machine code that ASSUMES both"
      a JIT, with guards and deoptimization

   Each layer needs a GUARD, and a way back when the guard fails.
   The way back is what makes it correct. It is also most of the work.

A guard plus a way back is the whole of speculative optimization, and it is why a JIT is much harder than a compiler. Section 7's real subject is not speed; it is how to be fast and still correct when your assumption is wrong.

The Labs

Lab	Title	What you end with
28	Inline Caches	Monomorphic caches on field access, with invalidation
29	Specialized Opcodes	Type feedback rewriting `ADD` → `ADD_INT`, with a guard
30	A Cranelift JIT	`function add(a,b) return a+b end` in native code, with deopt

The Non-Negotiables

Rule	Why
Correctness is not negotiable for speed. Every optimization keeps the differential tests green.	A fast wrong answer is worse than a slow right one.
Every change is measured, both ways.	Half of plausible optimizations do nothing; some are negative.
Reverted optimizations are recorded.	So the next person (you, in March) does not repeat them.
The tree walker is not optimized.	It is the oracle. Speed degrades its value as a reference.
`unsafe` requires the five-part treatment.	Why safe Rust is insufficient, the invariant, the minimized region, tests, and how violating it causes unsoundness.
The JIT is off by default.	It is the only part of Ember with `unsafe`, and a security posture.

The Honest Expectation

Ember will not be as fast as Lua, and nowhere near LuaJIT. Say so:

Lua is a register machine with twenty-five years of tuning, a packed 4-byte encoding, and a hand-optimized dispatch loop.
LuaJIT is a trace compiler with hand-written assembly interpreters per architecture, NaN boxing, and allocation sinking. It is one of the fastest dynamic-language implementations ever built, by one person, over a decade.

The objective here is education, not beating LuaJIT. What you should end with is: a measured speedup you can attribute, a JIT whose architecture you can draw, and the ability to read lj_record.c and know what you are looking at.

Measure your factor against lua in Lab 27's challenge and write it down. It is your honest position, and narrowing it is what this section is for.

Deliverables

Every optimization documented as baseline → hypothesis → change → measurement → tradeoff → decision, in docs/learning/14-performance.md.
At least one optimization reverted, with its measurement recorded.
An inline cache on field access with monomorphic/polymorphic/megamorphic states and a correctness test for invalidation.
At least one specialized opcode driven by type feedback, with a guard and a de-specialization path.
A Cranelift JIT compiling at least function add(a, b) return a + b end, with a guard and a working deoptimization path.
A three-way benchmark: tree interpreter vs VM vs JIT, with the reproducing command.
The differential tests are green at every step.
docs/adr/ADR-014-cranelift.md written.

Common Mistakes in This Section

Mistake	Symptom	Correction
Optimizing without a profile	A faster `ADD` and an unchanged policy benchmark	Lab 27's profile decides the order.
A cache without invalidation	Wrong answers after a table's shape changes	Every cache needs a guard and an invalidation story.
Speculating without a way back	A guard that fails and no path to correctness	The way back is the feature.
Optimizing the tree walker	An oracle you trust less	It is the reference. Leave it alone.
Keeping an unmeasured win	Complexity with no evidence	Revert, and record the revert.
`unsafe` without the five parts	A soundness hazard nobody reviewed	The treatment is not optional.
Believing 2003 dispatch numbers	Threading work that buys nothing on 2020s hardware	Measure `branch-misses / instructions` first.

Section Profile: What a Section 7 Graduate Can Do

Read a profile and decide what to optimize, rather than guessing.
Implement an inline cache with correct invalidation, and explain monomorphic, polymorphic, and megamorphic sites.
Implement type feedback and a specializing instruction rewrite, with a guard.
Explain what a JIT does, why it needs guards, and what deoptimization is for.
Explain why a garbage collector and a JIT must cooperate, and what a stack map is.
Use Cranelift to generate and call machine code from Rust.
Refuse an optimization that has no measurement, including their own.

Next: Inline Caches.

Open-Source Engineer & Contributor