# Pipeline overview, generator + retrieval + engines (mode: modify pipeline internals)

## Protocol layers, v1.0 Candidate terms

Two protocol layers coexist (ADR-0059, `docs/ops/spec/spec-a2ui-v1-conformance.md`):
the shipping dialect this pipeline emits (Layer A, `packages/gen-ui/a2ui/`) and
the A2UI v1.0 Candidate stack (Layer B, `packages/genui/`) reached
through `packages/genui/wire-bridge/`. **[amended 2026-08-29, ADR-0096]**
`adiahealth/gen-ui-system` is absorbed first-party under `packages/genui/`
and the standalone repo is archived, Layer B is in-repo source, not a
vendored dependency; `VENDOR.json` and the sync mechanism are gone (see
this same note on siblings `data-model-reactivity.md` and
`surface-lifecycle.md`). Candidate terminology is
**renderer/agent**, never client/server: `callableFrom` values are
`rendererOnly`/`agentOnly`/`rendererOrAgent`; the wire function kinds are
`callRendererFunction`/`callAgentFunction` +
`rendererFunctionResponse`/`agentFunctionResponse`; the MIME type is
`application/a2ui+json`; catalog resolution is strict (component `catalogId` →
surface `catalogId` → error, no registry default). The producer's
`wireFormat` flag (`packages/genui/adia-producer/exit-gate.js`) **[flipped
2026-08-30, PR #2412]** now defaults to `'v1'`, not `'dialect'`, ADR-0072
ratified the flip (its precondition, genui-system#51's re-verification,
re-scoped and satisfied via ADR-0096/PR #2401) and the flip itself has since
executed. Documents authored here stay dialect-shaped; the bridge owns the
translation, never hand-write Candidate envelopes from this skill's
surfaces.

**Catalogs are opt-out scopes, not a taxonomy.** A2UI v1.0 lets a renderer
mix catalogs within one surface: `createSurface.catalogId` is the default,
and any component may carry its own `catalogId` to override it
(a2ui.org/specification/v1.0-a2ui/, "Catalog Reference"). So the useful way
to split a catalog is by what an app OMITS, never by component kind, Buttons
and Inputs are never omitted together, whole feature areas are. This is the
ratified basis of the five-catalog partition (gh#2211, Kim 2026-08-28):
`adia.core` / `adia.navigation` / `adia.data` / `adia.agent` / `adia.shells`,
every one a derived view over the yaml `category` axis and package path,
never a hand list. Two facts that trip a re-derivation: the `category` axis
had no `data` bucket for tables/charts (they declared `agent`; the partition's
pre-step re-categorizes five components), and `packages/web-modules/**` is a
path seam, not a category (its sidecars mostly declare `layout`/`container`).
The partition lives on the Layer B side only; the dialect catalog stays one
file per ADR-0059 §1. The 44 L1 harvested widgets are a pattern library, not
catalog members.

site-a2ui (the build-time HTML→A2UI docs-site transpile) was RULED fit as
the **dialect side's regression corpus**, not a v1.0 conformance bed
(ADR-0068): it exercised the dialect renderer, the ADR-0061 lifecycle path,
and the engine transpiler at real-content scale in production, but never
touched the producer, the bridge, or the `wireFormat` flag. That fitness
ruling was load-bearing on `wireFormat` defaulting to `'dialect'`, with a
**Named expiry**: the flag-flip ADR that makes `'v1'` the shipping default
had to re-rule site-a2ui's fitness (re-point vs retirement-by-attrition).

**[resolved, ADR-0072, retired 2026-08-31]** That trigger fired and was
ruled: retirement, not re-point (Decision 2). Passive attrition never
netted the promoted set down (355 pages and growing at last measurement, regen work kept adding rows faster than breakage retired them), so the
operator converted it to an active drain (gh#2410): the mechanism, builder, ledger, artifact tree, gates, hook, is gone. The docs site now
renders every route through the legacy template path only; site-a2ui is
no longer part of this pipeline. The dialect side's regression-corpus
coverage site-a2ui once provided (real-content-scale exercise of the
dialect renderer, the ADR-0061 lifecycle path, and the engine transpiler)
has no standing replacement, see ADR-0072 Decision 2 / gh#2410 for the
closure record.

All paths repo-relative. Specs worth reading before structural changes:
`.claude/docs/specs/a2ui-v1.0-catalog-guide.md` (protocol + catalog format),
`.claude/docs/specs/genui-multiturn-architecture.md` (state cache, refiner,
op format), `.claude/docs/specs/genui-chunk-marker.md` (chunk attributes),
`.claude/docs/conventions/gen-ui-pipeline.md` (harvester wiring + embedding
lifecycle), `.claude/docs/specs/package-architecture.md` (package relations).

## The pipeline in one diagram

```text
intent → retrieval (chunk / composition search)
       → strategy engine (zettel | chunk-zettel | free-form | monolithic)
       → composer (plan → A2UI JSON)
       → validator + render + anti-pattern scan
```

Every change touches exactly one stage; identify which before patching. History
for any constant or decision lives in git and PR descriptions
(`git log -S STRONG_MATCH_THRESHOLD -- packages/gen-ui/a2ui`).

## Key files (verified 2026-07)

### Engine orchestration

| File | Role |
| --- | --- |
| `packages/gen-ui/engine/compose/core/generator.js` | `generate_ui` orchestrator, instant / pro / thinking / stream modes; multi-turn via `executionId` |
| `packages/gen-ui/engine/compose/strategies/registry.js` | Engine registry, `registerEngine(name, factory)`. Reserved names: `monolithic`, `monolithic-instant`, `monolithic-pro`, `monolithic-thinking`, `zettel`, `chunk-zettel`, `free-form` |
| `packages/gen-ui/engine/compose/strategies/zettel/` | Zettel + chunk-zettel engines, see [strategy-engines](strategy-engines.md) for the per-file map |
| `packages/gen-ui/engine/compose/strategies/free-form-composer/` | Free-form engine (`index.js`, `system-prompt.js`, `transpile.js`) |
| `packages/gen-ui/engine/compose/strategies/_shared/chunk-loader.js` | Shared chunk loading for engines |
| `packages/gen-ui/engine/compose/shared/validate-and-repair.js` | Shared closed-loop validate→repair, adopted by every LLM-calling engine (thinking/pro fully; free-form validate-only), see [strategy-engines](strategy-engines.md) §Closed-loop validation |
| `packages/gen-ui/engine/compose/transpiler/transpiler.js` | HTML → A2UI transpile pass (used by harvester + convert_html) |

### Corpus + retrieval

| File | Role |
| --- | --- |
| `packages/gen-ui/engine/corpus/scripts/chunk-library.js` | Chunk catalog API, `getChunk()`, `searchChunks()` (keyword), `searchChunksAsync()` (keyword + cosine), `listChunksByKind()`, `lookupChunksByPrimary()`. Reads `corpus/chunks/` + `_index.json` (moved from `packages/gen-ui/a2ui/` under ADR-0048's package split, corrected 2026-08-16) |
| `packages/gen-ui/engine/compose/strategies/zettel/composition-library.js` | Composition loader + `searchAll()` scoring (normalizes harvested chunks to composition shape) |
| `scripts/build/harvest-chunks.mjs` | `[data-chunk]` boundary walker over `site/pages/`, `apps/`, `playgrounds/`, `catalog/`, writes `corpus/chunks/<name>.json` + `_index.json`. Run via `npm run harvest:chunks` |
| `packages/gen-ui/engine/retrieval/intent/intent-categorizer.js` | Free-text intent → UI-category taxonomy |
| `packages/gen-ui/engine/retrieval/feedback/feedback-analyzer.js` | Aggregates JSONL feedback (`corpus/feedback/*.jsonl`); promotion + gap candidates |
| `packages/gen-ui/engine/retrieval/feedback/gap-registry.js` | Persistent gap tracking → `packages/gen-ui/engine/corpus/gaps/registry.json` |
| `packages/gen-ui/engine/retrieval/anti-patterns.js` | The `check_anti_patterns` rule source |
| `packages/gen-ui/engine/compose/core/reference.js` | Thin wrappers over retrieval exports (`searchBlocks`, `searchBlocksSemantic`, …) |

### LLM bridge + MCP

| File | Role |
| --- | --- |
| `packages/llm/core/llm-bridge.js` | `createAdapter()`, real LLM or stub fallback |
| `scripts/load-env.mjs` | Shared .env loader for Node scripts |
| `packages/gen-ui/mcp/gen-ui/server.js` + `packages/gen-ui/mcp/gen-ui/tools/*.js` | MCP stdio server + tool registrations, see [mcp-tool-reference](mcp-tool-reference.md) |

## Critical rules

1. **Relative imports in `packages/llm/core/*.js`**, never `@llm/` Vite aliases;
   they don't resolve in Node and break published consumers.
2. **`load-env.mjs` before any a2ui import in Node**, without it,
   `createAdapter()` silently returns `StubLLMAdapter` (canned 6-component
   card). Feedback or diagnosis on stub output is noise.
3. **Metadata IS the search index.** Descriptions + keywords are what retrieval
   matches. Enriching descriptions from structure (headings, labels, button
   text) took meaningful-description rate 40% → 95%. When search degrades,
   inspect chunk metadata before touching thresholds.
4. **Instant-mode gate lives in `monolithic/generate-instant.js`**, words ≥3
   chars pass; `GATE_STOPS` filters boilerplate. Grep the set before concluding
   "the gate rejects valid intents".
5. **A2UI describes LAYOUT, not behavior.** Generation emits component trees +
   props + slot bindings; never JS or per-canvas CSS. Behavior delegates to
   traits or pre-built apps. "Make the generator emit JS/CSS" is a won't-fix.
   **This is a GENERATION-pipeline claim, not the protocol's outer bound**
   (ADR-0022 amendment, 2026-08-24): the protocol itself, as consumed by the
   renderer and wire bridge, now carries a ratified CSS channel, `UpdateStylesMessage`/`RemoveStylesMessage`
   (`packages/gen-ui/a2ui/a2ui.schema.json:279-305`, renderer
   `#updateStyles`/`#removeStyles` at `renderer.js:113-114,821-879`), a
   first-class part of the protocol, not a carve-out. What stays true
   verbatim: compose/zettel synthesis itself still never emits
   `updateStyles`, so this rule's generation claim is unchanged; only the
   closed "the protocol never carries CSS at all" claim was falsified. JS
   remains fully out of scope for both the protocol and generation.
6. **Renderer guards `textContent` against container wipe**, `packages/gen-ui/a2ui/renderer.js` whitelists pure-text leaves
   (`TEXT_TAG_OK`); everything else routes through the `text=` attribute so
   slotted children survive. Preserve this when touching the renderer.
7. **Registry ↔ catalog parity.** A component in the runtime registry but
   missing from catalog schemas silently drops from generated compositions, `npm run check:registry-catalog-coherence` guards it; run it after catalog
   changes. `packages/gen-ui/a2ui/registry.js` is Class R (ADR-0069,
   gh#3055): never hand-edit it, a new component enters through its yaml
   `component:`/`tag:` fields, an alias or native-element mapping through
   `packages/gen-ui/a2ui/registry.exceptions.json`, then
   `node scripts/build/a2ui-registry.mjs` (derived-resync regenerates it on
   main; `check:a2ui-registry` is the advisory freshness gate,
   `check:a2ui-registry:validate` the blocking validity gate).
8. **Multi-turn emits A2UI `updateComponents` messages, not new compositions.**
   The chunk-refiner mutates the binding plan via four ops (`rebindSlot` /
   `appendToSlot` / `removeFromSlot` / `replacePage`); state chains via
   `parent_state_id`. Spec: `genui-multiturn-architecture.md`.
9. **Issue telemetry goes through `ctx.issueAccumulator`** (suppressed when
   `ctx.evalMode` is true so evals stay clean). New engine failure paths plumb
   the accumulator; never write records to disk directly. Durable issue
   tracking belongs in GitHub issues / PR descriptions, not the engine store.
10. **A new LLM-calling engine adopts `validate-and-repair.js` as its final
    stage**, full schema/catalog/anti-pattern conformance, orthogonal to
    whatever narrower plan/grounding validation the engine already runs
    (chunk-zettel's slot bindings, free-form's ingredient grounding). It
    is NOT automatic: a raw-HTML-wrapper engine (chunk-zettel today, see
    TKT-0009) gets zero value from it, check the engine's actual output
    shape is a real component graph before wiring it in.
11. **Escalation across engine tiers is the dispatcher's call, never the
    engine's own**, `registry.js`'s adapter wrappers (not
    `generate-instant.js`/`generate-pro.js` themselves) decide whether a
    hard-fail escalates to a stronger tier, capped at one hop. See
    `generateInstantAdapter`'s comment for the worked example.
12. **`harvest-chunks.mjs` validates every templated chunk at admission
    time** (validate-only, via the same shared module), report-only by
    default; `--strict` refuses to write on any invalid chunk but is NOT
    the default for `npm run harvest:chunks` (TKT-0010: 27% of the corpus
    currently fails, mostly a schema-generation gap around `data-*`/`span`
    attributes, not corpus-content defects, see the ticket before
    assuming a chunk is actually broken).
13. **Every generated catalog schema carries three synthesized universal
    props today** (`slot`/`hidden`/`ariaLive`, none declared in any yaml
    SoT); ADR-0097 rules a fourth, `traits`, but that part is decided-not-
    yet-shipped (gh#2513), see `primitive-authoring/references/
    yaml-contract.md` §Synthesized universal props for the full contract.
14. **Provider "extended thinking" is a strategy-level opt-in, not a
    global default** (gh#3516, LLD-0033). PR #3511 (gh#3477) made
    `{ thinking, thinkingBudget }` reachable through
    `AdiaUILLMBridge.complete()`/`stream()`, default off; two call sites
    opt in for real, each behind its own fixed-constant budget rather than
    a caller-threaded option: `generate-thinking.js`'s own generate call
    (`THINKING_BUDGET_MONOLITHIC_THINKING`) and `free-form-composer/
    index.js`'s ingredient-picker call, both its primary pick and its own
    paraphrase-retry (`THINKING_BUDGET_FREE_FORM`), both starting at the
    bridge's own `DEFAULT_THINKING_BUDGET`, lowered to 4000 (gh#3516 slice
    3, conductor ruling 2026-09-08) after an n=5 measurement at 10000
    showed no measurable quality gain with real cost. `auto` inherits
    through the free-form picker call the moment it escalates that far -
    `monolithic-thinking` itself is never reachable via `auto`'s own
    escalation ladder. Every other LLM call site (`generate-pro.js`'s four
    branches, zettel's locator/modifier/synthesizer, the shared
    `validate-and-repair.js` repair loop) stays thinking-off deliberately,
    a narrow first pass pending real eval numbers - do not assume a new
    engine inherits thinking by proximity to one that has it.
    `StubLLMAdapter` accepts and records `thinking`/`thinkingBudget` on
    its own `calls` log for strategy-level test assertions.
    `eval-diff.mjs` gained `--mode` (`instant`/`pro`/`thinking`), scoped
    to `--engine mcp` only, so `--engine mcp --mode thinking` exercises
    the opted-in path independently of the router's own default. See
    `docs/ops/lld/lld-0033-strategy-thinking-opt-in.md` for the full
    decision record and the conductor's D1-D5 rulings.

## Test + run commands (all verified in root package.json)

```bash
npm run test:a2ui              # smoke checks, no LLM (22/22, +1 skipped OK)
npm run test:a2ui:full         # + thinking mode (calls API)
npm run test:evals             # 5-dimension quality evals (mcp/evals/evals.json); --save-baseline via test:evals:baseline
npm run generate "login form"  # instant-mode CLI (mcp/scripts/generate.mjs)
npm run smoke:engines          # all registered engines
npm run smoke:register-engine  # in-process registration (11/11)
npm run eval:diff -- --engine zettel        # floors in SKILL.md
npm run harvest:chunks         # re-harvest corpus (dry: harvest:chunks:dry)
npm run smoke:chunks           # stub-LLM chunk smoke (re-harvests as side effect!)
npm run eval:chunk-synthesis   # hold-out intents against real LLM
npm run build:embeddings:chunks
npm run smoke:refine && npm run smoke:state-cache && npm run smoke:issues
npm run eval:refine-synthesis  # multi-turn refinement quality
npm run feedback:report && npm run feedback:promote
```