/** * THE ASSEMBLER — the table, LINKED into closures instead of interpreted. * * This is ledger row G5 as stated: *"build the grammar reference at run start, * making the swaps on rules / sub-rules (leaves) at that point, then run with NO * logic branching for that option input"*. `exec.ts` builds the same table and * then interprets it: one `switch (code[ip])` over 29 opcodes, executed ONCE PER * ROW — 497,360 rows for `benchmark.less` (`bench/jess/g5-rows.ts`) — each * re-reading its opcode and re-decoding its operands from the `Int32Array`, and * re-testing the same per-parse options. That is the per-node branching the * design exists to remove. * * ## The shape * * Assembly walks the reachable table ONCE and lowers each site to a PIECE: a * closure with its operands already captured as `const`s and its children bound * as DIRECT references to their own pieces. At parse time there is no opcode * read, no operand decode and no switch — a piece is called, and it calls the * pieces it holds. * * The numbers that make this obviously the right trade, measured not assumed: * * 2,241 distinct reachable sites in the less table (`bench/jess/g5-sites.ts`) * 497,360 rows executed for one parse of `benchmark.less` * * Pieces are GRAMMAR-sized; rows are INPUT-sized. Assembly allocates ~2.2k * closures once per process and removes a dispatch plus an operand decode from * ~497k executions — a 222x ratio. Assembly cost is paid once and is not the * metric; it is measured anyway (`bench/jess/g5-ms.ts`, `bench/jess/g5-profile.ts`). * * ## Why this is not "just tuning the switch" * * A 29-case switch on an `Int32Array` load is a jump table whose successor V8 * cannot know. The interpreter loop is ONE basic block with 29 merge edges, so * TurboFan cannot specialise any arm against its caller and every operand stays * an untyped load. Measured on this branch: `exec` reaches TURBOFAN and is * deoptimised back to MAGLEV repeatedly — 100 deopt events in a 20-parse run. * The design was not paying dispatch overhead, it was opting its whole hot path * out of the optimising compiler. * * A piece is created once and its call sites see one shape, so they stay * monomorphic and TurboFan can inline them. That is the structural win, and it is * why the target is codegen's number rather than a fraction of the gap. * * ## Why this keeps the artifact small * * The pieces ship ONCE, here, shared by every grammar and every variant. What a * bundle carries is still the table — DATA. Codegen's 2.10 MB is recognition * machinery inlined bespoke per rule; this stays at the table's 0.56 MB because * the machinery is one copy in the runtime and the variation lives in the * ASSEMBLY, not in duplicated piece bodies. * * ## Options are consumed by SELECTION, not by testing * * The piece set is a SUPERSET; an option set reaches a subset of it. Assembly * walks from the entry rule and instantiates only what it touches, so a piece an * option excludes is never allocated, never linked and costs zero at run — not a * cheap branch, zero. Where a decision is knowable from the option set * (`hostCst` below), assembly picks the piece rather than emitting one that * tests. `scripts/check-invariants.mjs` enforces that no piece body reads a * config field; see `CONFIG_FIELDS` there. * * Semantics are `exec.ts`'s, case for case. That file remains the reference and * the three-way identity sweep gates this against it. */ import type { Combinator, ParseContext } from '../types.ts'; /** * THE COMPLETIONS PROBE, at the terminal fail sites and nowhere else. * * `failAt` (combinators/probe.ts) is called from exactly three places in the * interpreter — `literal.ts`, `regex.ts`, `keywords.ts` — and codegen emits its * mirror (`probeUpdate`) at the same leaf-fail sites. The table recorded * NOTHING, so `completionsAt` on a table artifact saw only the top-level * failure: a swallowed item failure inside a `sepBy` (`'{'` at offset 1 in a * `{ decl; }` grammar) never reached the probe and the item's opener vanished * from the completion set. * * Dormant unless `completionsAt` set `_probe`, so an ordinary parse pays one * property read per terminal miss — the same price codegen pays. */ import { type CompactProgram, type ResolvedTable, type TableProgram, type TableRule } from './program.ts'; /** * THE PIECE SIGNATURE — uniform, narrow, and the same for every one of the 29 * lowerings. * * Three arguments and one return, so every call site in the assembled graph sees * one shape and stays monomorphic. A wider or varying signature (an out-param * object, a per-op return shape) reintroduces the polymorphism this design * exists to remove, and it would show up as exactly the megamorphic call sites * `exec`'s switch already was. * * The end position travels in the assembly-scope `EC.e` slot, as it does in * `exec.ts` and in codegen's `_pfEnd`, rather than in the return value — a * `{ value, end }` pair would be an allocation per row. */ type Piece = (input: string, pos: number, ctx: ParseContext) => unknown; /** * The option set an assembly is specialised for. * * These are the facts `run()` fixes BEFORE the entry is called and that never * change during a parse, so they are resolved by choosing a piece rather than by * a test inside one. Anything that varies DURING a parse (position, the capture * sinks a `node()` opens and closes, the error sink's contents) is runtime and * stays where it is. */ export type RunCfg = { /** Is this parse's host a CST-output host? Fixes `OP_NODE`'s whole shape. */ readonly hostCst: boolean; /** * Does the structural host consume the semantic `children` array? `false` * selects raw-only node collectors. Collapse forces this back to true at the * run boundary because its predicate consumes both child views. */ readonly hostReadsChildren?: boolean; /** * Structural node types whose host consumes trivia. Evaluated while an * assembly is built, once per node site, never from the node execution path. * Absent means the backwards-compatible default: capture every structural * node's trivia. */ readonly hostCaptureTrivia?: ((type: string) => boolean) | undefined; /** `ctx.trackLines` — decides whether the trivia leaf swap is legal at all. */ readonly trackLines: boolean; /** * `ctx._tolerant` — is THIS parse allowed to recover? * * FIXED FOR THE LIFETIME OF A PARSE, which is what makes it legal here rather * than a per-piece test. Every writer of `ctx._tolerant` sets it BEFORE the * parse begins and never during — `compile.ts:119` (`parseWithErrors`), * `functional/run.ts:399`, `functional/doc.ts:29`, `combinators/completions.ts:40`. * The single mid-parse mutation is `recovery/scan.ts:36`, which CLEARS it for * the duration of a sentinel probe and restores it in `finally`; a sentinel is * a `firstSetSentinel` char-class combinator (or an `orSentinel` of two), never * an assembled rule, so no piece runs under the cleared value. Even if one did, * the selection would be the RIGHT one: a probe must not recover, and it would * get the strict assembly. * * Contrast `cstCaptureActive`, which a previous lane proposed for the same * treatment and which is per-NODE state — keying on that would have been * incorrect, not merely redundant. */ readonly tolerant: boolean; /** * `ctx._grammarCoverage` — is THIS parse counting grammar coverage? * * FIXED FOR THE LIFETIME OF A PARSE, checked the same way `tolerant` was. There * are exactly two writers — `createGrammarInstrumentationContext` (coverage.ts), * which builds the field into a FRESH context object before any parse, and * `functional/run.ts:403`, which installs it on the context it is about to run * — and no reader anywhere mutates or clears it, mid-parse or otherwise. * `createParseContext` initialises it to `undefined` once, at construction. * * So it is per-PARSE, unlike `ctx._cstBuf`, which a previous lane proposed * keying on and which `beginCstNodeCapture`/`endCstNodeCapture` replace per * NODE — that selection would have been incorrect, not merely redundant. * * An ordinary table has no `OP_COV` rows at all, so this bit selects a DIFFERENT * assembly only for a table encoded with a coverage plan. For every other table * the two assemblies are identical work and the extra bit costs one cache slot. */ readonly coverage: boolean; /** * `ctx._probe !== undefined` — is THIS parse feeding a completions probe? * * FIXED FOR THE LIFETIME OF A PARSE, on exactly the evidence `tolerant` is * held to. The two writers both install it BEFORE the parse begins and never * during — `combinators/grammar.ts:213` (built in `run()`'s prologue when * `recover` is set) and `combinators/completions.ts:38`. The single mid-parse * mutation is `recovery/scan.ts:29`, which CLEARS it for the duration of a * sentinel probe and restores it in `finally`; a sentinel is a * `firstSetSentinel` char-class combinator, never an assembled rule, so no * piece runs under the cleared value. Even if one did the selection would be * the RIGHT one — a sentinel probe must fail fast, and it would get the * strict assembly. * * It is here for `OP_GATE`, which is the only piece that reads it on a * SUCCESS path. The six leaf sites read it after their own `return`, on the * failure path only, and are deliberately left alone: selecting a second body * per literal length to remove a failure-path test would double the runtime's * literal bodies to buy nothing measurable. */ readonly probe: boolean; }; /** The cfg key an assembly is cached under. */ export declare function cfgKey(c: RunCfg): number; export type Assembly = { /** One entry piece per rule name, already linked. */ readonly pieces: Readonly>; readonly end: () => number; /** * Per-parse reset. What `exec.ts`'s `begin` DECIDED here (`trackLines`, the * host mode) is exactly what assembly resolved, so all that is left is * clearing the installed scanner and latching the host value. */ readonly begin: (ctx: ParseContext) => void; /** Close an invocation and restore a suspended re-entrant frame, if any. */ readonly finish: () => void; readonly scanSkip: readonly (readonly Combinator[])[]; /** * The sites this option set actually REACHED. A strict subset of the table's * reachable set whenever an option excludes anything, and the assertion * `test/unit/table-assemble.test.ts` makes on that. */ readonly reached: ReadonlySet; /** * WHY THIS ASSEMBLY IS RUNNING CLOSURES, when it is. * * `undefined` means the emitted engine (`emit-assembly.ts`) built this * assembly. A string names the construct it refused. It is a field rather * than a log line because a grammar that quietly drops to the closure path * is a permanently slow path nobody would ever find — the same failure * `encode.ts:1208-1213` refuses to allow for `OP_LIVE`. */ readonly emitRefusal: string | undefined; }; /** * Link one resolved table, for one option set, into a graph of closures. * * ONE walk. Each site is lowered at most once and memoised by its code offset, * so a subtree shared by two parents is one piece with two references to it. */ export declare function assemble(t: ResolvedTable, prog: TableProgram, cfg: RunCfg): Assembly; /** * Assemblies for one resolved table, one per option set, built on demand. * * The option set is not known when the rule map is created — `ctx.build`, * `ctx.trackLines` and `ctx._tolerant` arrive with the parse — so the entry * computes the scalar key and takes the assembly for it, building it the first * time that combination is seen. That is the "assembled at run start" half of G5: * only the option combinations a process actually uses, each holding the pieces * its options reach. A process that never parses tolerantly never builds, and * never runs, a single recovery piece. */ export declare class AssemblyCache { private readonly t; private readonly prog; private readonly byCfg; /** * Monomorphic predicate inline cache. An assembly depends on the predicate * FUNCTION and the scalar cfg key, not on the host object that carries it, so * retaining the host would add neither correctness nor reuse. Stable parsing * is one identity comparison plus an array index; replacing the predicate * replaces this specialisation without retaining either host. */ private hostPredicate; private hostAssemblies; constructor(prog: TableProgram, resolved?: ResolvedTable); for(cfg: RunCfg): Assembly; /** * The assembly for the option set this `ctx` implies. * * ─── WHY THIS IS NOT A VIOLATION OF "NO OPTION READS AT PARSE TIME" ───────── * * The rule that shaped 0.47 is that consulting an option PER RULE or PER * COMBINATOR is a fail, and the reason it is a fail is that such a consult * scales with the input. This one does not, and the previous defence of it — * "cheap, allocation-free, only once" — was the wrong argument, because * cheapness is not the criterion. The right argument is that the consult is * IRREDUCIBLE, and it is worth stating precisely so nobody re-opens it: * * A table entry has the artifact signature `(input, pos, ctx)`, shared with * codegen. All scalar option bits live on that `ctx`, and every one of them is * supplied PER CALL by the caller — `run()` takes `tolerant`, `build` and * `instrumentation` as options on an artifact it was handed; * `combinators/grammar.ts` sets `trackLines` on scope entry; * `completionsAt` installs `_probe`. So the option set is not knowable * before the call, and a selection that cannot happen before the call must * happen at it. * * That is not a hole in G5, it is G5's own first clause: "quickly building the * grammar reference ON RUN START, making some swaps on rules or sub-rules, and * then the run actually runs with no logic branching for that option input" * (`notes/TABLE-DRIVER.md`). This IS the run-start step. What the criterion * forbids is the second sentence, and past this call there is no option read * anywhere — `scripts/check-invariants.mjs` INV-6 decides that mechanically. * * MEASURED, not asserted: exactly ONE call per entry invocation, including * `benchmark.less` at 106,802 bytes. Eliminating it would require binding the * option set to the ARTIFACT rather than to the CALL, which is a change to the * public run API (the map would have to hand back a cfg-keyed family and * `run()` index it), not a change to this file. * * It allocates nothing: the key is packed from the ctx's own bits by `cfgKeyOf`, * which takes them as arguments precisely so no `RunCfg` need exist here — that * object is built only on the miss that builds an assembly. A host trivia * predicate additionally selects its identity-specialised inline cache * because a function cannot be represented by a scalar bit. * * DO NOT CACHE THE RESULT ACROSS CALLS. Keying it on anything but the `ctx`'s * own option bits is how `tableRules` handed a strict parse the PREVIOUS * parse's tolerant assembly (`test/unit/table-assemble.test.ts`). */ forCtx(ctx: ParseContext): Assembly; } /** * The ASSEMBLED rule map — the same artifact contract as `tableRules`, run * through linked closures instead of the bytecode interpreter. * * THE ONE config read is HERE, at the boundary, once per entry invocation: * `AssemblyCache.forCtx` turns the `ctx` into a scalar option set and takes the * assembly built for it. Everything past that point is pieces, and no piece body * reads an option — `scripts/check-invariants.mjs` INV-6 asserts it. `forCtx` * carries the argument for why that read is irreducible rather than merely cheap. */ export declare function tableRules(source: TableProgram | CompactProgram, artifactMetadata?: Readonly>): Record; export {}; //# sourceMappingURL=assemble.d.ts.map