/** * @copyright Sister Software * @license AGPL-3.0 * @author Teffen Ellis, et al. * * PIX1 placetype-pair index (placetype-pair-prior arc). A pure-JS, browser-safe lookup * from a folded (child, parent) place-name pair to the `ComponentTag` the pair resolves to (e.g. * "shoreditch" under "london" → `dependent_locality`) — the retrieval-augmented complement to the * encoder's own judgment, following the same PCB1 single-file writer+reader pattern as * `postcode-binary-resolver.ts` so the layout can never drift between the two ends. * * This file owns BOTH ends of the format — `serializePairIndex` (run in Node by the shard-build * tooling) and `PairIndexResolver` (run in the browser and server alike) — with zero Node imports * in the reader path. * * Binary layout (little-endian): magic "PIX1" (4 bytes) u32 headerLen, headerLen bytes of * UTF-8-encoded JSON (`PairIndexHeader`) u32 pairCount, then pairCount records of: * * ``` * u16 childLen, child utf8[childLen], u16 parentLen, parent utf8[parentLen], u8 tagIdx, u8 parentTagIdx * ``` * * sorted by (child, parent) UTF-16 code-unit order. `tagIdx` and `parentTagIdx` BOTH index the * header's `tagTable` — the copy of `COMPONENT_TAGS` embedded at serialize time (schema 3; u8 caps * at 256 tags, asserted at serialize — the table is nowhere near that today), so the binary is * self-describing and immune to reordering of the runtime tag union. Normative outside-contributor * spec: `docs/engineering/reference/pix1.ksy` (kept honest by the layout-conformance test). * * A record is a TYPED EDGE, and both ends are recorded. Schema 2 wrote only the child's tag and the * decode side derived the parent's from `WESTERN_PARENT_OF` — which cannot express the edges the * builders actually extract (the US WOF source emits a `dependent_locality` under a BOROUGH, itself * a `dependent_locality`; containment says a `dependent_locality`'s only parent is `locality`). One * byte per pair buys the source's own answer instead of a re-derived guess. * * This departs from PCB1's fixed-width key table on purpose: postcodes are bounded (~7 ASCII * chars), but place names vary widely in byte length, so a fixed-width key would either truncate * long names or waste space padding short ones. A `u16`-length-prefixed UTF-8 string per field * costs 2 extra bytes per pair — irrelevant at the ~20k-entry scale this index targets — in * exchange for exact byte-for-byte names. `probe()` is Map-backed (built once in the constructor) * rather than binary search, for the same reason: variable-width records make positional * `record[i]` addressing awkward, and 20k entries is small enough that the Map's O(n) build cost * and memory footprint are non-issues. * * `child`/`parent` are expected to already be folded (NFKC-lowered, punctuation-stripped — see * `normalizeFSTToken` in `fst-prior.ts`) by the caller; `foldVersion` in the header records which * fold the entries were built against, so a consumer can detect a stale index if the fold changes. * * Duplicate-tolerance is explicitly NOT a serializer concern: `serializePairIndex` asserts its * input is already deduped by (child, parent) and throws otherwise. Building the shard is where * duplicates should be resolved (e.g. picking the higher-confidence tag) — silently last-write-wins * or first-write-wins at serialize time would hide a shard-build bug. */ import { type ComponentTag } from "@mailwoman/core/types"; /** * Schema 3 (2026-08-04): every record carries a second tag byte — the PARENT's `ComponentTag` — so a pair asserts the * WHOLE typed edge rather than half of it. Schema 2 (the tag table moving into the header, same day) is refused rather * than read: a v2 record stops after `tagIdx`, so decoding one as v3 would swallow the NEXT record's `childLen` as a * parent tag. Both breaks are deliberate (operator-ruled): the release pipeline rebuilds pair indexes anyway * (`copy-weights` → `gazetteer pair-index`), and a tolerant fallback would keep a wrong-by-construction artifact * alive. * * Why the parent tag is RECORDED and not derived. The #46 preregistration derived it from `WESTERN_PARENT_OF`, on the * argument that containment already owns the fact. It does not: containment maps a child tag to the parents the TREE * BUILDER will accept, which is a different question from what the extraction actually observed. The US borough source * emits (`Park Slope`, `Brooklyn`) — a `dependent_locality` under a `dependent_locality` — and containment's * `dependent_locality: ["locality"]` can never say that. Deriving also spends the bias across every allowed parent when * the set has more than one, so a `locality` child biased `subregion`/`region`/`country` alike and moved nothing. */ export declare const KNOWN_SCHEMA_VERSION = 3; export interface PairIndexEntry { /** * Folded child place name (e.g. a dependent_locality or locality candidate). */ child: string; /** * Folded parent place name the child was observed under. */ parent: string; /** * The `ComponentTag` this (child, parent) pair resolves the CHILD to. */ tag: ComponentTag; /** * The `ComponentTag` the same pair resolves the PARENT to — the other half of the asserted edge (schema 3). Required: * {@link serializePairIndex} refuses an entry that omits it or names something outside `COMPONENT_TAGS`. A builder * that cannot state its parent's tag from its source's own semantics must not guess one — see * `mailwoman/gazetteer-pipeline/borough-pairs.ts` for the worked case (the WOF parent row's placetype, projected * through `PLACETYPE_PROJECTION`). */ parentTag: ComponentTag; } /** * What a probe hit returns: the whole typed edge. Deliberately an object rather than the bare child tag — returning * half of a two-ended assertion is exactly the defect schema 3 exists to close, and a caller that only wants the child * reads `.tag` visibly rather than silently getting a half-answer. * * Instances are INTERNED per resolver (there are a handful of distinct (tag, parentTag) combinations across even the * 199k-entry FR artifact), so the probe map costs one pointer per entry, not one object per entry. */ export interface PairEdge { readonly tag: ComponentTag; readonly parentTag: ComponentTag; } export interface PairIndexHeader { /** * ISO country code this shard was built for. */ country: string; /** * The soft-prior bias magnitude a probe hit should contribute (consumer-interpreted). */ delta: number; schemaVersion: 3; /** * The tag universe `tagIdx` and `parentTagIdx` index into, embedded at serialize time (a copy of `COMPONENT_TAGS` as * of the build). Makes the binary self-describing — an outside reader decodes tags with no mailwoman import — and * decouples every shipped artifact from the ORDER of the runtime tag union. The reader resolves each record's name * against the runtime's known tags and throws on a referenced unknown; unknown names no record references are * tolerated, so a binary built after the union grows still loads on an older reader as long as the new tag is * unused. */ tagTable: string[]; /** * Which fold (`normalizeFSTToken`-style normalization) the entries were built against. */ foldVersion: 1; /** * MD5s of the source file(s) this shard was built from, for provenance. */ sourceMD5s: string[]; /** * ISO date the shard was built. */ buildDate: string; /** * OPTIONAL per-country transition-bonus magnitude (TRANSITION-BETA build, 2026-07-24): on a pair hit, the prior emits * a position-scoped decoder adjustment of `+transitionBeta` on every transition INTO `B-` at the child span's * first piece — the path-fusion recovery lever the task-8 transition-level probe measured (β=5: 13/17 comma-free GB * misses recovered, zero measured collateral on 47 correct rows + 200 venue-confound rows). ABSENT = no transition * term at all (today's emission-only behavior) — backward compatible (old binaries lack the field and keep working) * AND forward compatible (old readers parse the header JSON and simply never consult the extra key; optional fields * ride the JSON header without a schema bump). Calibrated per country like `delta`: the GB artifact ships 5; the NZ * artifact deliberately ships WITHOUT it (unmeasured there, and comma-free NZ is already at 99.2%). */ transitionBeta?: number; /** * OPTIONAL per-country WHOLE-EDGE bias magnitude (#46, default-on 2026-08-04): on a pair hit, the prior ALSO writes * `+parentDelta` onto the record's `parentTag` over the parent window, not just `+delta` onto the child. ABSENT = no * parent bias at all (the child-only behaviour every artifact carried before this) — absence-tolerant in the same * sense as {@link transitionBeta}, and absence means OFF, never 0-as-a-default. * * Calibrated per country, and only where it was MEASURED. `us`/`gb`/`nz`/`fr` ship 5 — the smallest δ that saturates * bar B-2's brooklyn-class sub-board, flat from there through 20 * (`docs/records/evals/2026-08-04-pix1-whole-edge-verdict.md`). `de`/`in`/`es`/`it` ship WITHOUT it: no board has * graded the parent side there, and the D-rule's answer to an unmeasured locale is a per-locale gate, not an * inherited magnitude. * * Overridable at decode: `PlacetypePairPriorOpts.parentDelta` (which `MAILWOMAN_PAIR_PARENT_DELTA` feeds) wins over * the header, so an eval can sweep δ without rebuilding artifacts. */ parentDelta?: number; } /** * The caller-supplied half of {@link PairIndexHeader}: everything except the two format-owned fields (`schemaVersion`, * `tagTable`), which {@link serializePairIndex} stamps itself — the format version is the serializer's fact, not the * builder's claim. */ export type PairIndexHeaderInput = Omit; /** * Serialize (header, entries) into the PIX1 flat binary. Entries are sorted by (child, parent) so the format is * deterministic regardless of input order. Run in Node; consumed by {@link PairIndexResolver}. * * Throws if `entries` contains a duplicate (child, parent) pair (dedupe upstream — see the file-header note on why this * isn't silently resolved here), if a child/parent string exceeds the u16 length prefix (65,535 UTF-8 bytes — no real * place name approaches this), or if an entry's `tag` / `parentTag` is missing or is not a `ComponentTag`. The * `parentTag` check is not defensive noise: a builder that cannot state its parent's tag from its source's semantics * must fail loudly here rather than have a plausible-looking default written into a shipped artifact. */ export declare function serializePairIndex(header: PairIndexHeaderInput, entries: readonly PairIndexEntry[]): Uint8Array; /** * Read just the magic + header block (no entry parsing, no Map build) — the same validation the constructor does * (bad-magic throw, future-schema throw) but stops the instant the header JSON is decoded. Lets a caller inspect * `country`/`delta`/`sourceMD5s` etc. before paying for the full entry parse — e.g. * `NeuralAddressClassifier.loadFromWeights`'s hard country gate (`classifier.ts`) reads this FIRST and only constructs * a `PairIndexResolver` (which walks every entry to build the probe `Map`) when the header's country matches the * resolved locale; a mismatch skips construction entirely rather than paying the full parse just to discard the * result. */ export declare function peekPairIndexHeader(bytes: Uint8Array): PairIndexHeader; /** * Pure-JS, browser-safe reader over the PIX1 flat binary. Builds a `Map` once in the constructor * (cheap at the ~20k-entry scale this index targets) so `probe()` is O(1). The `PairEdge` values are interned across * records — the FR artifact's 199k entries share a single frozen object — so the second tag byte costs the map no extra * allocation. */ export declare class PairIndexResolver { #private; readonly header: PairIndexHeader; constructor(bytes: Uint8Array); /** * Look up the typed edge a folded (child, parent) pair asserts, or `undefined` if the index has no entry for it. * Returns BOTH tags — a caller that only wants the child's reads `.tag`. See {@link PairEdge} for why this is not the * bare child tag. */ probe(childFolded: string, parentFolded: string): PairEdge | undefined; /** * Exposes the calibrated delta bias magnitude so the resolver conforms to {@link PairIndexLike} and can be passed * directly. */ get delta(): number; /** * Exposes the header's ISO country code so the resolver conforms to {@link PairIndexLike}. Consumed by the * placetype-pair prior's segment path to pick the country-specific trailing-postcode shape it strips before folding a * parent-candidate segment key (see `placetype-pair-prior.ts`'s `segmentParentPostcodeShape`). */ get country(): string; /** * Exposes the optional transition-bonus magnitude (see {@link PairIndexHeader.transitionBeta}) so the resolver * conforms to {@link PairIndexLike}. `undefined` on a binary built without the field — the prior then emits no * transition adjustments (the pre-TRANSITION-BETA behavior, exactly). */ get transitionBeta(): number | undefined; /** * Exposes the optional whole-edge parent-bias magnitude (see {@link PairIndexHeader.parentDelta}) so the resolver * conforms to {@link PairIndexLike}. `undefined` on an artifact built without it — the prior then writes no parent * bias at all, which is the pre-#46 behaviour exactly. */ get parentDelta(): number | undefined; } /** * Minimal subset of `PairIndexResolver` a prior module consumes — structural typing so callers depend on the shape, not * the class (the `query-shape-prior.ts` "…Like" convention). `delta` is optional because a hand-built test double may * omit it; a real index's header carries the authoritative value. `transitionBeta` is optional in BOTH senses: a test * double may omit it, and a real header legitimately lacks it (see {@link PairIndexHeader.transitionBeta} — absent means * no transition term, not a default). */ export interface PairIndexLike { probe(child: string, parent: string): PairEdge | undefined; readonly delta?: number; readonly transitionBeta?: number; /** * The header's whole-edge parent-bias magnitude (see {@link PairIndexHeader.parentDelta}). Optional in both senses, * like `transitionBeta`: a hand-built double may omit it, and a real header legitimately lacks it (an unmeasured * locale ships without one). An explicit `PlacetypePairPriorOpts.parentDelta` overrides whatever this says. */ readonly parentDelta?: number; /** * The index header's ISO country code (lowercase, e.g. `"gb"`/`"nz"`). Optional for the same two reasons as `delta` * and `transitionBeta`: a hand-built test double may omit it, and it drives an OPTIONAL behavior — the segment path's * country-aware trailing-postcode strip (see `placetype-pair-prior.ts`'s `segmentParentPostcodeShape`). Absent, or a * country with no known postcode shape → no strip (byte-stable). */ readonly country?: string; } //# sourceMappingURL=pair-index-resolver.d.ts.map