Metatables

Three concepts: the metatable, the lookup path, and operator metamethods.

Metatables are how one data structure becomes an object system, an operator-overloading facility, a proxy mechanism, and a read-only-view mechanism — with no new syntax and about 200 lines of runtime. They are the most language-design-interesting thing in Lua.

Concept 1: The Metatable

1. Concept

Every table may have a metatable: another table whose entries, at specific keys beginning with __, change how the first table behaves.

local Account = {}
Account.__index = Account                        -- lookups that miss go HERE
function Account.new(b) return setmetatable({balance = b}, Account) end
function Account:deposit(n) self.balance = self.balance + n end

local a = Account.new(100)
a:deposit(50)          -- `deposit` is NOT a key of `a`. The metatable found it.
print(a.balance)       -- 150

2. Problem

A language needs objects, inheritance, operator overloading, defaults, proxies, read-only views, and lazy loading. Each of those could be a language feature — six syntaxes, six implementations, six sets of rules. Or one mechanism could produce all of them.

3. Mental model

A metatable is a table of hooks. When an operation on a table would otherwise fail or be undefined — a lookup that misses, an addition with a non-number, a call on a non-function — the runtime checks the metatable for a handler. Metamethods are the "otherwise" branch of every primitive operation.

That framing predicts the whole set: for every operation that can fail, there is a metamethod named after it.

4. Implementation

#![allow(unused)]
fn main() {
pub struct Table {
    // ...
    meta: Option<GcRef<Table>>,      // ← a GC edge; trace it
}

/// The metamethods Ember implements. Interning these names once, at Engine
/// creation, means metamethod lookup is a handle comparison rather than a
/// string hash — which matters, because it happens on every miss.
#[derive(Copy, Clone, PartialEq, Eq)]
pub enum MetaKey {
    Index, NewIndex, Call, ToString, Len, Eq, Lt, Le,
    Add, Sub, Mul, Div, IDiv, Mod, Pow, Unm, Concat,
}
pub struct MetaNames { interned: [GcRef<EmberStr>; MetaKey::COUNT] }
}

Interning the metamethod names at startup is not a micro-optimization. Every failed table lookup consults __index, so the name is hashed on a path that is already the slow path. Pre-interning turns it into a handle compare. Lua goes further and caches a bitmask per metatable of which metamethods are absent (flags in Table), so the common "no metatable behavior for this event" case is a single bit test. That is worth knowing and worth copying if Section 7's benchmarks point here.

5. Alternatives

The A-versus-B distinction is thinner than it looks: __index pointing at another table is prototype delegation. JavaScript's [[Prototype]] is __index with dedicated syntax and no way to change the rest of the protocol. Lua's version is more general and less discoverable — you can build JS's object model in Lua, but not the reverse.

6. Decision

A, with a deliberately partial set of metamethods.

Ember implements __index, __newindex, __call, __tostring, __len, __eq, __lt, __le, and the arithmetic/concat metamethods. It omits __gc (finalizers), __mode (weak tables), __close (to-be-closed variables), __metatable (protection), __pairs, and the bitwise metamethods.

Each omission has a reason, and they are different reasons:

A partial implementation with documented omissions is a design; a partial implementation with silent gaps is a bug. All of these go in appendix/lua-differences.md.

7. Tradeoffs

That last row is a real cost and worth stating to users: in a language with __index, no property access is guaranteed to be cheap or side-effect-free. Section 7's inline caches must account for it, and Section 5's sandbox must count metamethod calls against the instruction budget.

8. Production concerns

• Metatables are a GC edge. Table::meta must be traced. It is one of the four commonly-missed edges.
• A metamethod can run arbitrary script code, from inside what looks like a primitive operation. That means it can allocate, recurse, error, and re-enter the VM. Every metamethod dispatch site must therefore be a safe point — the VM's state must be consistent before calling out.
• setmetatable on a shared table is a global side effect. A script that sets a metatable on a table the host also holds changes the host's view. Section 5's userdata has its own metatable handling for this reason.

9. References

rg -n 'luaT_gettmbyobj|luaT_trybinTM|TM_INDEX|MAXTAGLOOP' ltm.c ltm.h lvm.c

• Lua's ltm.c and ltm.h — the metamethod table, the TM_* enum, and the name cache.
• Lua 5.4 Reference Manual §2.4 ("Metatables and Metamethods") — the normative list, worth reading in full once.
• Programming in Lua, chapters 13 and 16 (object-oriented programming) — the idiomatic patterns built on this mechanism.
• The ECMAScript specification's [[Get]] and [[Set]] internal methods, for option B's version of the same lookup path.

Concept 2: The Lookup Path

1. Concept

t.k is not one operation. It is: a raw lookup, and — only if that misses — a metatable consultation, which may itself be a lookup on another table, recursively.

2. Problem

The recursion must terminate. Two tables whose __index fields point at each other make an infinite chain, and a script can write that in two lines.

3. Mental model

   t.k
    │
    ├─▶ raw get(t, k)          ── found? ──▶ DONE. The fast path, and the common one.
    │        │ miss
    │        ▼
    ├─▶ t has a metatable?     ── no ──▶ return nil
    │        │ yes
    │        ▼
    ├─▶ mt.__index             ── absent ──▶ return nil
    │        │
    │        ├─ is a TABLE?    ──▶ REPEAT the whole process on that table
    │        └─ is a FUNCTION? ──▶ call __index(t, k); its result is the answer
    │
    └─▶ depth limit exceeded ──▶ error "'__index' chain too long; possible loop"

rawget is the fast path and it is the common case. A field that exists is found in one probe; the metatable machinery only runs on a miss. That is what makes the mechanism affordable — and it is why an object system built on __index chains makes method lookup (always a miss on the instance) slower than field lookup.

4. Implementation

#![allow(unused)]
fn main() {
/// Lua 5.4 §2.4. Iterative, not recursive: a __index chain is attacker-controlled
/// and recursion here would be a stack-overflow vector.
const MAX_META_CHAIN: usize = 100;      // Lua's MAXTAGLOOP is 2000

fn index(&mut self, mut obj: Value, key: Value, span: Span) -> Result<Value> {
    for _ in 0..MAX_META_CHAIN {
        let meta = match obj {
            Value::Table(t) => {
                let raw = self.heap.table(t)?.get(key);
                if raw != Value::Nil { return Ok(raw); }     // ← the fast path
                match self.heap.table(t)?.meta {
                    None => return Ok(Value::Nil),           // no metatable → nil, not an error
                    Some(m) => m,
                }
            }
            // Indexing a NON-table is an error unless its TYPE has a metatable.
            // (§5 gives userdata one; strings get one so that s:upper() works.)
            other => match self.type_metatable(other) {
                Some(m) => m,
                None => return Err(self.rt(span, format!(
                    "attempt to index a {} value", other.type_name()))),
            },
        };

        match self.heap.table(meta)?.get(self.names.get(MetaKey::Index)) {
            Value::Nil => return Ok(Value::Nil),
            // A FUNCTION handler: call it. This re-enters the VM.
            f @ (Value::Closure(_) | Value::Native(_)) =>
                return self.call_value(f, vec![obj, key], span)
                           .map(|r| r.into_iter().next().unwrap_or(Value::Nil)),
            // A TABLE handler: repeat the whole process on it. This is
            // delegation, and it is how inheritance works.
            next => obj = next,
        }
    }
    Err(self.rt(span, "'__index' chain too long; possible loop"))
}
}

Four details worth reading twice:

1. The loop, not recursion. MAX_META_CHAIN bounds an attacker-controlled chain. Recursing would put the bound on the Rust stack, where it cannot be caught — the same reasoning as the parser's depth guard.
2. A missing key with no metatable is nil, not an error. Only indexing a non-table errors. Those are different failures with different messages, and conflating them makes article.missing throw when it should return nil.
3. Strings have a shared type metatable, which is how s:upper() works without every string carrying one. Lua does the same, and getmetatable("") returns it.
4. The function branch re-enters the VM, so everything about re-entrancy in Section 5 applies here, one section early. Do not hold a borrow of the heap across that call.

__newindex mirrors it, with one asymmetry that catches everyone:

t[k] = v
--  raw get(t, k) ~= nil  →  assign RAW. __newindex is NOT consulted.
--  raw get(t, k) == nil  →  consult __newindex.

__newindex fires only for keys that do not already exist. That is what makes the "log all writes" proxy pattern work — you keep the real data in a different table so every write is new — and it is why a naive __newindex that assigns into self fires once and then never again.

5–7. Alternatives, decision, tradeoffs

Decision: A, with a limit of 100. Lua's MAXTAGLOOP is 2000; Ember's is lower because a hundred-deep inheritance chain is a bug in any real program, and a lower limit fails faster and more legibly. Document the divergence.

8. Production concerns

• Metamethod calls must be charged to the instruction budget. Otherwise setmetatable({}, {__index = function() ... end}) gives a script an uncounted call on every field access. The budget tick is in the fetch position, so a script metamethod is counted automatically — but a native one is not. Section 5 charges those explicitly.
• __eq is only consulted when raw equality fails and both operands are tables. Getting the order wrong makes t == t call a metamethod: slow, and observably wrong if the metamethod has side effects.
• __lt/__le on mixed types. Lua 5.4 requires both operands to have the same handler, and __le no longer falls back to not (b < a) as it did in 5.3. Check the manual rather than assuming; this changed between versions.
• Errors inside a metamethod must carry a useful traceback. The frame that failed is the metamethod's, but the cause is the field access three frames up. Push a frame for the metamethod call so the traceback shows both.

Concept 3: Operator Metamethods

1–3. Concept, problem, mental model

Every binary operator's error path becomes a metamethod lookup. a + b where either operand is not a number checks __add on a, then on b, and errors only if neither has one.

#![allow(unused)]
fn main() {
fn arith(&mut self, op: BinOp, a: Value, b: Value, span: Span) -> Result<Value> {
    // The FAST PATH is unchanged: two numbers, no metatable involvement.
    if let Ok(v) = arith::binary(op, a, b) { return Ok(v); }

    // The SLOW PATH. Left operand first, then right — Lua's order.
    let key = MetaKey::for_binop(op);
    if let Some(h) = self.metamethod(a, key)? { return self.call1(h, &[a, b], span); }
    if let Some(h) = self.metamethod(b, key)? { return self.call1(h, &[a, b], span); }

    // Neither had a handler: NOW produce the type error, blaming the operand
    // that is not a number — the same message as Lab 4's.
    Err(self.arith_type_error(op, a, b, span))
}
}

Notice that the fast path is untouched. 1 + 2 does not consult a metatable, does not check for one, and costs exactly what it did before Lab 18. That is the property to preserve, and it is why the metamethod check lives in the error branch rather than in front.

4–7. Implementation, alternatives, decision, tradeoffs

The full set Ember implements, with the operation that triggers each:

Decision: implement all of the above, and nothing else. The list is the "otherwise" branch of every operation Ember has. When you add an operation, you add its metamethod in the same commit — that is the rule that keeps the mechanism coherent.

8. Production concerns

• __tostring must be validated. A handler returning a table gives print a non-string to write. Check and error with a message naming the metamethod.
• __call and recursion. A table whose __call is itself is a legal, infinite construct. The call-depth limit catches it; make sure the metamethod path goes through the same do_call that enforces it, rather than a shortcut.
• Ordering of __eq matters for the differential test. The tree walker and the VM must consult metamethods in the same order with the same fallbacks. This and multiple returns are the two places Lab 18 will break your comparison.
• Every metamethod dispatch is a re-entry point. Audit them the way you will audit host functions in Section 5: no live borrows, no half-updated state, ip synced.

9. References

• Lua 5.4 Reference Manual §2.4 — the normative list, with the exact fallback order for each. Read the __lt/__le paragraphs carefully and compare with 5.3's.
• Lua's lvm.c: luaV_finishget, luaV_finishset, luaT_trybinTM, and MAXTAGLOOP.
• Lua's ltm.c: the flags bitmask cache in Table — the "this metatable has no __index" fast-negative check.
• Programming in Lua, chapter 13, for the idiomatic patterns (proxies, read-only tables, defaults, inheritance).

Things to Notice

• Metamethods are the "otherwise" branch of every primitive operation. That framing predicts the whole set without memorizing it.
• The fast path must stay fast. Metatable checks live in the failure branch, never in front.
• __newindex fires only on new keys, which is what makes proxies work and what makes naive proxies fire exactly once.
• __index chains are attacker-controlled, so the traversal is iterative and bounded.
• A metamethod can run arbitrary code inside what looks like a primitive, which makes every dispatch site a safe point and a re-entrancy hazard — Section 5's problem, arriving early.
• One mechanism produces objects, operators, proxies, defaults, and inheritance. That is the best argument in the language for mechanism over policy, and it is why Lua is small.
• Omissions need reasons. __gc, __mode, __close, and __metatable are each left out for a different reason, and writing them down is what makes the subset a design.

Validation / Self-check

1. What is a metatable, and why does the "otherwise branch" framing predict the metamethod set?
2. Walk a.deposit through the lookup path for the Account example, naming each step.
3. Why is the __index traversal iterative and bounded? What is Lua's limit and what is Ember's?
4. When does a missing key error, and when does it return nil? Give both messages.
5. Why does __newindex fire only for new keys? What pattern depends on that, and what bug does it cause?
6. Why does the arithmetic fast path stay untouched by Lab 18?
7. In what order are __add handlers consulted, and when is the type error finally raised?
8. Name four metamethods Ember omits and give a different reason for each.
9. Why is every metamethod dispatch site a re-entrancy hazard, and which section's rules apply?
10. How do __index and JavaScript's [[Prototype]] relate? Which is more general?

Next: Lab 13 — Tables.