# @ai-matrx/records

THE headless client of the AI Matrx record store: the Table / Field / Rule /
Record / Relation / Home types, the closed vocabularies and refusal codes read
from the store ITSELF, the typed calls over the store's doors and nothing else,
the merge-field declaration, a validation mirror that predicts only what the
store's own triggers refuse, and a result envelope that never throws.

The package ships the types root, the headless `/core` and the React binding
`/react`. `@ai-matrx/records-ui` renders it; nothing else talks to the store.

**The one rule this package lives by:** the client mirrors the store's contract
and never re-implements a rule the store enforces. Where the store refuses, this
package surfaces the store's own sentence — never a translation of it.

```ts
import {
  type Table, type Field, type StoredRecord, // the shapes
  type ValueEnvelope,                        // ver · src · actor · on_behalf_of · at · absent · alternates · dated
  PARITY_FIELD_TYPES,                        // the 13 field types, read from custom.parity_field_types()
  ACTOR_VOCABULARY,                          // exactly: user, agent, system
  ABSENCE_REASONS,                           // exactly: never asked, none, refused, conflicting
  STORE_DOORS, STORE_REFUSAL_CODES,          // GENERATED from pg_proc on the live store
} from "@ai-matrx/records";
```

## Quick start — our record store inside your own system

Everything a company needs to list, create, edit, archive and watch records from its own screens or
scripts, with none of ours. One package; each section loads only when imported.

```sh
npm install @ai-matrx/records @supabase/supabase-js
```

```ts
import { createClient } from "@supabase/supabase-js";
import { createRecords, sessionAuth } from "@ai-matrx/records/setup";
import { ensureTable } from "@ai-matrx/records/core";

// 1. Sign in. The database address is https://db.matrxserver.com; the publishable key is on your
//    AI Matrx account page. Any supabase-js sign-in works (password, magic link, SSO, server cookies).
const supabase = createClient("https://db.matrxserver.com", PUBLISHABLE_KEY);
await supabase.auth.signInWithPassword({ email, password });

// 2. One call. You choose the organization new records are saved in; reads span all of yours.
const matrx = await createRecords({ auth: sessionAuth(supabase), organizationId: (orgs) => orgs[0] ?? null });

// 3. A table (found by its slug, or made the first time), then records.
const table = await ensureTable(matrx.records, { name: "Service callbacks", slug: "service_callbacks",
  fields: [{ key: "caller", label: "Caller", type: "text" }, { key: "phone", label: "Phone", type: "phone" }] });
if (!table.ok) throw new Error(table.error.message);
const tableId = table.data;                 // the table id

const created = await matrx.records.recordWrite({ table_id: tableId, data: { caller: "Marcus Delgado", phone: "+14155550178" } });
const page    = await matrx.records.list({ table_id: tableId, limit: 50 });                  // { rows, total }
const record_id = created.ok ? created.data : "";
const read    = await matrx.records.recordRead({ record_id, withVersion: true });          // .data.document, .data.version
const expectedVersion = read.ok ? (read.data.version ?? undefined) : undefined;
const edited  = await matrx.records.recordUpdate({ record_id, patch: { phone: "+14155550199" }, expectedVersion }); // answers the new version
// Always send expectedVersion: a colleague's newer edit is then refused (`stale_write`, field by field), never overwritten.
const archived = await matrx.records.recordDelete({ record_id });                            // soft: recordRestore brings it back

// 4. Live: other people's changes to this table (this setup's own writes are not echoed back).
const stop = matrx.watch(tableId, { records: (ids) => console.log("changed", ids) });

// 5. Search across every table you can see; platform records (contacts…) through matrx.entities.
const hits = await matrx.search({ search: "Delgado" });

stop(); matrx.close(); // a script closes the live socket before it exits
```

Every call answers `{ ok: true, data } | { ok: false, error }` — the store's own sentence, never a
throw. React hosts add `@ai-matrx/records/react` or `@ai-matrx/records/data/react` hooks on the same
client; `@ai-matrx/records-ui` is our optional set of screens. Organization API keys plug into `auth`
as a second mode (planned); today `auth` is a signed-in session.

| Section | What |
|---|---|
| `records/setup` | `createRecords`, `sessionAuth` — the one call above |
| `records/core` | the typed client over every door (tables, records, versions, views, forms, totals) |
| `records/realtime` | the live-changes port setup uses |
| `records/search` | `searchRecords`, `listStoreRecordCandidates` |
| `records/forms` | the form document, embeds, which read a public address does, `typedAnswersFor` |
| `records/versions` | the version rule: `useRecordVersions`, version-carrying write doors, `askOnce`, `refusal` |
| `records/entity`, `records/entity/react` | platform records (contacts, projects, tasks) |
| `records/data`, `records/data/react` | one data port and TanStack hooks over custom tables and entities |
| `records/react` | provider + hooks over the client |

## /core — the typed client over the doors

```ts
import { createClient } from "@supabase/supabase-js";
import { createRecordsClient, ensureTable, supabaseDataSource } from "@ai-matrx/records/core";

const supabase = createClient(url, publishableKey);  // the client that carries this person's session
const records = createRecordsClient({
  dataSource: supabaseDataSource(supabase),  // binds the store's schema for you
  actor: { actor: "user" },                  // who is acting: "user", "agent" or "system"
  organizationId: orgId,                     // where writes go; the store is keyed (organization_id, id)
  onError: (e) => report(e),                 // OPTIONAL — the default screams to console
});
// Reads of tables and records span `organizationIds` when you pass them (every organization the
// person belongs to); typed tables read every one of them by default, without being told.
// (This block is compiled with the package: src/__tests__/readme-quick-start-compiles.tsx.)

// A page's own small table: found by its slug, or made the first time (one transaction).
const log = await ensureTable(records, { name: "Front desk callback log", slug: "front_desk_callback_log",
  fields: [{ key: "caller_phone", label: "Phone", type: "phone", unique: true }, { key: "reason", label: "Reason", type: "long_text" }] });
const one = await records.tableFind({ slug: "front_desk_callback_log" });          // the Table, or null
const id  = await records.recordUpsert({ table_id, key: "caller_phone", data: { caller_phone, reason } }); // write-or-update
// With expectedVersion it is an EDIT: no live record of that key → stale_write, never a new record.

const written = await records.recordWrite({ table_id, data: { title: "Acme", owner: personId } });
const rows    = await records.list({ table_id, limit: 50 });
const read    = await records.recordRead({ record_id, withComputed: true });
const moved   = await records.recordUpdate({ record_id, patch: { title: "Acme Inc" }, expectedVersion: 3 });

if (!moved.ok && moved.error.code === "stale_write") {
  // The store's own words, plus the conflict field by field.
  moved.error.message;  // "Someone else changed this record while you were working on it. …"
  staleWriteDetail(moved.error); // { expected_version, current_version, contested_fields }
}
```

### `ensureTable` never changes a column — it says how each one differs

A table that already exists keeps every column exactly as stored (a retype is a conversion a person
decides). What the spec asked for differently is answered, never ignored:

```ts
const log = await ensureTable(records, spec);
// { ok: true, data: tableId, homeId, created: false,
//   drift: [{ field: "caller_phone", aspect: "type",  stored: "text",   declared: "phone" },
//           { field: "patient_name", aspect: "label", stored: "Patient", declared: "Patient full name" }] }
```

`drift` is `FieldDrift[]` — `{ field, aspect, stored, declared }`, `aspect` one of `type` (parity type,
else behavior), `format`, `label`, `multi`, `required`, `unique`, or `spec` (that column's spec could
not be read; `declared` is the store's sentence). It is `[]` on a birth and when nothing differs, and
`null` only from a store too old to compare. The declared side is the spec as the store WOULD store
it, so `long_text` is never a false `type` drift. This shape is the contract a release-time drift
check reads (`parseFieldDrift` types a raw answer).

### A table the app keeps

```ts
const outputs = await ensureTable(records, {
  name: "Exercise flashcard outputs", slug: "agent_output_exercise_flashcard",
  platformOwned: true, keptFor: "agent_output", kind: "exercise_flashcard",
  homeId: sharedOutputsHome,                         // a NEW table goes in this Home; none is made
  fields: [{ key: "body_region", label: "Body region", type: "select", optionsTableId }],
});
```

`keptFor` is part of the address: a spec carrying it finds only a table kept for that word, and a spec
without it never finds an app's table — a person's table and an app's table of one slug are different
tables. `platformOwned`, `keptFor`, `kind`, `draftsFor` and `draftsStamp` are stored on the table it
makes. `homeId` must be a live Home of this organization the caller may change, or nothing is made.
`optionsTableId` reuses an existing choice list; `declaration` passes any other `custom.field_declare`
key through as written.

### When two tables share a slug, `tableFind` and `ensureTable` may differ

A slug is not unique (older organizations hold a few duplicates). `tableFind` answers the oldest
table of that slug **the caller may see**. `ensureTable` takes the oldest table of that slug
**overall** and, if the caller may not know that one, refuses with the read door's own words rather
than making a second table. So for a hidden oldest duplicate, `tableFind` returns a newer table while
`ensureTable` refuses; both always agree when the caller can see the oldest one.

Nothing here throws, and nothing here invents an answer. A door the store does
not have yet is answered `door_absent`, naming the door AND the lane that owes
it — `fieldPropose`, `tablePropose` and `mergeFieldResolve`
refuse today rather than returning an empty list that would read like "there is
nothing".

## Three ways to make a table (each under a minute, all the same store)

Every way ends in the same place: an ordinary record-store Table, with the store's own doors
deciding access, history and archive. Pick the one that matches where you are standing.

**1. From code — declare it, it exists on first use, the rows are typed.** No codegen: `RowOf` /
`InsertOf` come from the declaration, so a wrong choice, a missing required column or a misspelled
column fails `tsc`.

```ts
import { defineTypedTable, f } from "@ai-matrx/records/typed-table";
import { useTypedTable } from "@ai-matrx/records/react";

export const vendorContacts = defineTypedTable({
  name: "Vendor contacts", slug: "vendor_contacts", scope: "organization", kept_for: "purchasing", key: "vendor",
  fields: {
    vendor: f.text({ required: true, unique: true }),
    contact_name: f.text(),
    category: f.select(["Medical supplies", "Lab services", "Facilities", "IT"] as const),
    monthly_spend: f.currency("USD"),
  },
});
// in a component under <RecordsProvider>: the table is made in the active organization on mount
const { rows, upsert } = useTypedTable(vendorContacts);
await upsert({ vendor: "Quest Diagnostics", category: "Lab services", monthly_spend: 1850 });
```

**2. From structured JSON — REST v1 or the AI Matrx MCP `tables` tool** (`POST /api/v1/tables`
with `X-Organization-Id`; MCP `tables {action:"create_table", organization_id, name, columns}`):

```json
{ "name": "Vendor contacts", "columns": [
  { "name": "Vendor", "type": "text", "required": true, "unique": true },
  { "name": "Category", "type": "choice", "choices": ["Medical supplies", "Lab services"] },
  { "name": "Monthly spend", "type": "currency" } ] }
```

**3. From SQL — `POST /api/v1/sql` or the MCP `sql` tool**, one statement at a time, written as
for Postgres: `CREATE TABLE vendor_contacts (vendor text not null unique, category text check
(category in ('Medical supplies','Lab services')), monthly_spend numeric)`, then `INSERT …`,
`SELECT … WHERE … ORDER BY … LIMIT`, `UPDATE … WHERE`, `DELETE … WHERE` (archives), `ALTER TABLE
… ADD COLUMN`. The subset and every refusal: aidream `aidream/services/table_api/FEATURE.md`.

**A table that already exists** (made in the app, or by way 2 or 3) gets a typed definition with
`pnpm tables:types <table id | slug> [--organization <id>] --out features/x/vendor-contacts.ts`,
which emits the `columns` map plus `…Row` / `…Insert` types from the live store.

## /typed-table — tables the platform keeps for its own features

A feature that needs a small table of its own declares it once in code; each organization gets its
own copy on first use, and the row type comes from the declaration.

**The ten-minute path**

```ts
// features/decisions/decision-picks.typed-table.ts — imports only @ai-matrx/records/typed-table
import { defineTypedTable, f } from "@ai-matrx/records/typed-table";

export const decisionPicks = defineTypedTable({
  name: "UI decision picks", slug: "ui_decision_picks", kept_for: "decisions",
  scope: "person", owner: "lane:PLATFORM-APP-DATA",       // owner is optional
  key: "decision_id",                                       // must be a unique field
  fields: {
    decision_id: f.text({ label: "Decision", unique: true, required: true }),
    choice: f.select(["a", "b", "neither"] as const, { label: "Choice" }),
    decided_at: f.datetime({ label: "Decided" }),
  },
});

// in a component under <RecordsProvider>
const { rows, upsert, remove, loading, error, status } = useTypedTable(decisionPicks);
await upsert({ decision_id: "header-density", choice: "a" });   // choice: "maybe" does not compile
```

- `rows` are `RowOf<typeof decisionPicks>[]` from every organization the person belongs to — the
  client's `organizationIds` when the host passed them, else the store's own list (`iam.my_orgs()`);
  the caller passes nothing. `{ organizationId }` narrows to one (that is also where writes go).
  Without it, writes go to the active organization (the client's `organizationId`).
- `total` is the store's count of the rows this person may see; `truncated` is true when `rows`
  stopped at 10,000 (`APP_ROWS_CEILING`) — a read is never cut silently.
- `status` is `"ok"`, `"drifted"` (the stored columns differ from the code — reported, never
  applied; a stored column the code no longer declares is drift `extra`), `"archived"` (the store
  refuses to make it again: `error` says so, naming the table, and `upsert` answers the same
  sentence — restore it from Archived tables) or `null` while unknown or refused. `error` is the
  store's own sentence; on a store without the lane-12 doors it is `door_absent`.
- Each typed table is found by its slug AND `kept_for`: a person's own table of the same slug is a
  different table and never answers. Every typed table of an organization lives in one Home,
  "Platform tables".
- **The code depends on it:** `ensureTypedTable` marks each copy `code_depends: true` (plus
  `app_table.declared_in` from the definition's `declared_in`). The store then refuses archiving,
  renaming (slug or `kept_for`), moving or unmarking it through every door; the one way through is
  `archiveTypedTableDeliberately(client, { tableId, typedSlug, reason })` — a person types the slug,
  an agent is refused.
- Headless: `ensureTypedTable(client, def, { organizationId })`, `checkTypedTable(client, def)`
  (`status`, `drift`, `rowCount`), `listAppRows`, `upsertAppRow`, `removeAppRow` (the store's soft
  delete). The store refuses a value under a key the definition never declared (23514, by name).
- **Graduation** (contract: common-docs `projects/data-doctrine-adoption/v6/CONTRACT-PLATFORM-APP-DATA-GRADUATION.md`):
  a definition adds `graduatedTo: { token: "<schema>.<table>", map: { <field>: <column> | "custom_fields" | "association" } }`
  once its standard entity table exists (made through `platform.create_entity_table`). Every declared
  field must be mapped (relations to `"association"`); the spec hash does not move. `pnpm tables:graduate
  <module> <export> [--target clone|production] [--dry-run]` copies every live and archived row of every
  organization's copy (id, organization, author, timestamps, version, archived state, `shown_to`,
  `visibility` kept; undeclared keys → `custom_fields`; store bookkeeping + `legacy_id` → `metadata`;
  relation values → `platform.associations`), one transaction per organization, `ON CONFLICT (id) DO
  NOTHING`, refuses on any count disagreement, then archives each live copy through
  `custom.table_archive_deliberately` ("graduated to <token>"). Default target: the clone. The pure
  parts are exported: `graduationProblems`, `graduatedRow`.
- Builders: `text longText email phone url number integer decimal currency(unit) percent checkbox
  date datetime select multiSelect relation(otherDef) member attachment entityReference(kinds)` and
  the read-only `formula autonumber createdTime modifiedTime` (absent from `InsertOf`).

**The gate** (Arman's ruling of 2026-10-02, verbatim in common-docs
`systems/data/custom-data/DECISIONS.md`, "When the platform may keep its own data in the store"):

- **Allowed:** trials, features few people use, unproven adoption, short-term stopgaps, small data
  — including platform features whose rows belong to organizations (bookings, intake lists).
- **Forbidden:** anything most or all people use (tags), a clearly long-term feature, data that
  will barely change shape, or a lot of data.
- **Size, not time:** no reason or review date is recorded; a table is checked at 50,000 rows
  across all organizations and again every further 10,000 (`checkTypedTable`'s `rowCount` feeds that
  check; the acknowledgement step is not built yet).

**Scope, honestly:** `organization` rows are seen by members of the row's organization, and every
member can add rows (the copy is shared with members at editor level when it is made). `person`
rows start "Only me": they are **hidden from everyone else's lists** — the store's list predicate
drops them, so another member's read returns only her own rows — but they are **not locked**: a
member who has a row's id can still open it. A store-level lock is the store's Confidential level,
which is separate and needs the owner's approval per table; never put anything that must be private
in a `person` table. `global` is one copy in the platform organization, written only by server code;
`ensureTypedTable` refuses it unless that organization is named explicitly.

## The mirror — what the store is about to say, said half a second earlier

```ts
import { predictWriteRefusals } from "@ai-matrx/records/core";

const problems = predictWriteRefusals({ fields, document, recordType });
// [{ field_key: "amount", sqlstate: "23514", predicted_by: "custom.validate_values",
//    message: "Amount has to be at least 1" }]
```

This is NOT validation — the store validates, inside the write door, and cannot
be bypassed. The mirror only says what the store is about to say, in the store's
exact sentence, and it says NOTHING where it cannot see (option membership,
relation targets, anything needing a query). Silence means "ask the store".

## /react — bind once, then hooks

```tsx
import {
  RecordsProvider, useTables, useFields, useRecords, useRecord, useRecordMutation,
} from "@ai-matrx/records/react";

<RecordsProvider config={{ dataSource: supabaseDataSource(supabase), actor: { actor: "user" }, organizationId: orgId }}>
  {children}
</RecordsProvider>;
// A config without `actor` is refused by name ("RecordsProvider needs config.actor …").

const { tables } = useTables();
const { fields } = useFields(table_id);
const { rows, total, live, liveOffReason } = useRecords({ table_id, pageSize: 50 });
const { write, update, remove, conflict } = useRecordMutation();
```

`live` is never quietly false: when realtime is not bound, `liveOffReason` says
why in a sentence a screen can show.

## /entity — platform records (contacts, projects, tasks) through the same port
Platform records (contacts, projects, tasks…) through the same `DataPort` an Applet reads a custom
table with. A thin typed client over the store's entity doors, and a host `DataPort` for entity
aliases, and token-first React hooks over records' one data engine (`@ai-matrx/records/data/react`).

**Reads and writes.** `create` saves into the active organization the host names (none → refused
`organization_required`), `update` sends the version the person saw (a stale one is refused with the
store's sentence), `archive` / `restore` move a row to the trash and back. **Lists are live** when the host passes `realtime: createEntityRealtime({ manager })`
(and `organization`, the active one): other people's changes arrive as `rows:<alias>` events naming the ids.

```ts
import { createEntityDataPort } from "@ai-matrx/records/entity";
import { DataPortProvider, hostEvents, useDataRows } from "@ai-matrx/records/data/react";

const port = createEntityDataPort({
  dataSource: supabase, // the signed-in person's supabase-js client
  aliases: { contacts: { kind: "entity", token: "party", organizationId } },
});

// in React:
// <DataPortProvider port={port} events={hostEvents(port)} portKey="applet-1" queryClient={qc}>
//   const { rows, hasMore, loadMore, error } = useDataRows("contacts", { org: organizationId, sort: [{ column: "display_name" }] });
```

In a React app, name the type and let the engine do the rest:

```tsx
import { EntityDataProvider, useEntityList, useEntityWrite } from "@ai-matrx/records/entity/react";

// once, high: <EntityDataProvider dataSource={supabase} activeOrganizationId={activeOrg}>…
const contacts = useEntityList("party", { where: [{ column: "last_name", op: "eq", value: "Ellis" }] });
const write = useEntityWrite("party");
await write.update(contacts.rows[0]._id, { first_name: "Ada" }); // drawn at once, rolled back on refusal
```

### Entries

| Entry | What |
|---|---|
| `@ai-matrx/records/entity` | `createEntityClient({ dataSource })` → `list`, `get`, `columns`, `insert`, `update`, `archive`, `unarchive`; `toWire`; `mapEntityError`; `createEntityDataPort({ client \| dataSource, aliases, activeOrganizationId })` → `HostDataPort` + `describe`; `createEntityRealtime` |
| `@ai-matrx/records/entity/react` | `EntityDataProvider`, `useEntityList`, `useEntityColumns`, `useEntityType`, `useEntityRow`, `useEntityWrite` |

### Behavior worth knowing

- The lane defaults to the type's own landing lane, decided by the store; `org` is a filter and never a lock.
- Paging is keyset: the cursor is the store's `next_after`.
- A filter or sort the store does not offer yet is refused by name with the store's sentence, never emulated in the browser.
- Custom fields read from `row._custom` and filter as `_custom.<key>`; a field the person may not read arrives `null`.
- A total that stopped at the store's count cap is reported as `null` (unknown), never as a wrong number.

Local mechanics: `src/entity/FEATURE.md`. Formerly the package `@ai-matrx/entity-data` (deprecated; same API, renamed imports: `/core` and `/port` → `records/entity`, `/react` → `records/entity/react`).

## Drill-down — declared definitions

A server drill (`platform.drill_describe` / `platform.drill_ask`) answers a standard table from
the registry, or a **declared definition**: a `*.drill.ts` file (`declareDrillDefinition({...})`,
shape in `scripts/drill-definitions/ai_usage.drill.ts`) compiled to one function per definition,
`platform.drill_def__<key>()`. A Dimension may carry `level` (breakouts, show, attributes,
records); the UI side is `useDrill` in `@ai-matrx/records-ui` and the table in
`@ai-matrx/design-system` (its README § Drill-down).

| Command (in this package) | Does |
|---|---|
| `pnpm drill:sync --write --lane <LANE>` | writes a campaign file (+ inverse) for every declaration whose live body differs |
| `pnpm drill:sync --write --lane <LANE> --only <key,key>` | the same, for ONLY your definitions — never a peer's half-done one |
| `pnpm drill:sync --check` | the census: every declaration sound and equal to its live body (exit 1 otherwise) |
| `pnpm drill:sync --check --only <key,key>` | the census of your definitions only |
| `... --target clone` | reads the nightly copy instead of production |

`--only` naming a key no file declares fails, so a typo never passes as "nothing to do".

## The generated half

`src/store.generated.ts` is written by `pnpm records:generate` from the live
store's own catalog — the vocabularies from the store's vocabulary functions,
the field types from `custom.parity_field_types()`, the doors from `pg_proc`,
the kernel ids from the kernel rows, the tokens from `platform.entity_types`.
`pnpm check:store-registry` exits non-zero the moment this package and the store
disagree, which is how a door another lane added shows up as a red check rather
than as a runtime surprise.

## The suite

`pnpm test` runs against the live rehearsal branch. Every case is one
transaction that always ends in `rollback`: inside it the suite opens the
product switch, grants only what its own half needs, and sets
`request.jwt.claims` to the campaign's test identity (`test@test.com`) — so every
call goes through the real doors, the real triggers and the real guards, as a
real identity, and the branch is byte-identical afterwards (the last test
re-reads it from a fresh connection and proves it). The red half is the same
calls without that standing, asserted by the store's message text.

## Commands

| Command | What it does |
|---|---|
| `pnpm records:generate` | Rewrite `src/store.generated.ts` from the live store |
| `pnpm check:store-registry` | Fail when this package and the store disagree |
| `pnpm test` | The real-door suite |
| `pnpm build` | tsup → `dist` (esm + cjs + d.ts), then the client boundary stamp |
| `pnpm check:package` | build · exports agreement · publint · tarball canary |