# @ai-matrx/content-ir

The framework-independent Content IR kernel used by AI Matrx applications. It parses streamed or complete structured content into canonical envelopes, manages incremental parse sessions, converts kind schemas, and performs structural validation.

```bash
pnpm add @ai-matrx/content-ir
```

## Entry points

The root export is the full surface and always will be. Since 0.10.0 each internal area is also addressable on its own — prefer these, so a consumer that only reads wire contracts never pulls the parser (or `ajv`) into its graph.

| Import | What it holds |
|---|---|
| `@ai-matrx/content-ir` | everything below |
| `@ai-matrx/content-ir/core` | IR types + tree, JSON tokenizer, kind parser, snapshots, normalize, fingerprint, discriminator, envelope read/value/cache, region, schema structure |
| `@ai-matrx/content-ir/session` | `ParseSession` and the one-writer-per-identity registry |
| `@ai-matrx/content-ir/registry` | kind registry types, storage transform, the dual gate, `createKindValidator` over a host `SchemaSourcePort` |
| `@ai-matrx/content-ir/convert` | kind ⇄ JSON Schema, the OpenAI structured-output converter |
| `@ai-matrx/content-ir/wire` | partial-kind events, runtime wrapper readers, emit payload |
| `@ai-matrx/content-ir/directives` | Kind Directives: slug grammar, the ONE decoder |
| `@ai-matrx/content-ir/source` | offset-preserving source tokenizer, splice save + anchor-mapping change set, the single-dollar math rule |

```ts
import { createKindStreamParser, envelopeFromCompleteValue } from "@ai-matrx/content-ir/core";
import { validateStructuralLeg } from "@ai-matrx/content-ir/registry";
import {
  readPartialKindEvent,
  makePartialKindStalenessGate,
  rehydrateNodeOutcome,
  rehydrateRunResult,
  withRootKind,
} from "@ai-matrx/content-ir/wire";
```

## Reading an envelope: `kindState`, never `kind` alone

`root.kind` is **preserved** when a payload fails its schema — a broken `flashcard_set` stays a `flashcard_set` so you can say what broke. `root.status` reports only that parsing finished. The validity signal is `root.kindState`:

- `"resolved"` — the only state a typed consumer may act on.
- `"unverified"` — the kind has no registered schema, so nothing was ever checked; it still routes to its component.
- `"raw"` / `"pending_*"` / `"speculative"` — do not consume as typed data.

The package has no browser, React runtime, application store, database client, routing, or renderer dependency. `ajv` is its only runtime dependency. Host-specific registry fetching, component compilation, rendering, and error capture remain adapters around this kernel.

## `/source` streaming mode: what "settled" promises

`tokenizeSource(text)` reads stored text; `tokenizeSource(prefix, { streaming: true })`
reads a message still arriving. In streaming mode anything the next bytes could
still change is held back as plain prose (an open last line, an unclosed code
span, a `$$` that can still pair, a footnote whose continuation is undecided,
lines a box-drawing tree still forming below may claim as its root label, a
paragraph cut short by a block that may still dissolve, …),
so **an island reported `complete` is the island the finished text has** — with
ONE documented exception:

- **A late orphan reasoning closer.** Models do emit closer-only reasoning (the
  `<thinking>` opener lost upstream). When a `</thinking>` / `</think>` /
  `</reasoning>` arrives with no opener, the one-shot reading folds everything
  before it into a thinking region — retroactively. No streaming reader can
  know that in advance short of holding ALL prose provisional until the stream
  ends, which costs more than it saves (chair ruling, 2026-09-26; common-docs
  `projects/rich-content-unification/REGISTER.md`, RC-B3 rulings).

**Consumer contract:** when a stream completes, re-read the finished text in
one-shot mode (`tokenizeSource(text)`, no `streaming`) and show THAT — the
final screen always equals the one-shot result, which is also what a reload
shows. Stored text never goes through streaming mode, so none of this can
change a stored byte. (matrx-frontend's chat renderer does the same for its
own live accumulator: `components/mardown-display/chat-markdown/settle-stream-blocks.ts`.)

## Release gate

```bash
pnpm --filter @ai-matrx/content-ir typecheck
pnpm --filter @ai-matrx/content-ir test
pnpm --filter @ai-matrx/content-ir check:package
```

The gate runs the parser's shared fixtures and parity vectors, validates the packed declarations and exports, installs the tarball in an empty project, and imports the public API.