/** * @copyright Sister Software * @license AGPL-3.0 * @author Teffen Ellis, et al. * * Boundary-instability synthesizer (#375 — the highest-impact parser lever). The failure taxonomy * * The within-token-punctuation decomposition (#702) found one failure FAMILY surfacing under many * names: the model mis-places token boundaries between adjacent components when the boundary is * ambiguous or unmarked. This generator emits diverse BIO-labeled rows that put the gold boundary * exactly where the model wobbles, so a retrain learns the boundary from context, not the * lexeme. * * The four token-aligned stress shapes, all in BASE LOCALES (US/FR/DE) so the shard never * introduces tokens the base corpus lacks (the #511 base-consistency lint flagged an earlier * AU-bearing draft: AU 4-digit postcodes collide with US house numbers, and AU localities are * absent from the US/FR/DE base — a real contradiction). Each component is a whitespace-separated * token run, so `alignRow` labels it cleanly: * * 1. `street-eats-affix` — multi-word street + suffix (`Country Club Rd` → street + street_suffix), * the #1 wobble: the model keeps the suffix in the street. * 2. `comma-less-city-state` — no comma between street / locality / region (`100 Main St Springfield * IL 62701`), the #694 family: concatenated input loses the segmentation cue. US-only (US * zips are base-consistent; the boundary is locale-agnostic). * 3. `fr-prefix` — FR street-type prefix split from the name (`Rue Jean-Baptiste Lebas` → * street_prefix * * - Street), postcode-first order. * 4. `house-number-after-street` — FR/DE number-follows-street (`Neuve-des-Capucines 5` → street + * house_number), the model absorbs the number into the street. * * Two BALANCING shapes (added 2026-06-18 after the v1.6.0 probes). The first pass at weight 1.0 * lifted the boundaries but over-fit a NARROW distribution — every row was a clean, full, * structured address — so the model regressed on out-of-distribution real rows (held-out US * locality 66.3→58.2%). These two widen the distribution the shard teaches over, per the * diagnosis (scripts/eval/locality-regression-probe): 5. `bare-locality` — locality with NO * street (`Public Library, Lisbon ND`, `75003 Paris`), the ship-blocker: 84% of the v1.6.0 * locality regression was DROPPED locality on bare "City, STATE" rows, because every other shape * placed a street before the city. Bare / comma-less / postcode'd / venue-prefixed forms, US + * FR. 6. `house-number-before-street` — the confounding mirror of #4 (same FR vocab, number * BEFORE the street). A balanced before:after mix breaks the positional shortcut behind the #4 * order-bias. * * EXCLUDED: the region+postcode glue (`NY14201` — sub-token, no punctuation to split) and the * AU/NZ/UK slash unit-convention (`4/2A` → unit+house_number). The slash labels cleanly (the * tokenizer splits `/`) and is the worst within-token class — but it inherently requires non-base * AU/NZ/UK locales, which contradict the US/FR/DE base (the lint catch). It belongs in a * separately-scoped AU/NZ/UK boundary-coverage shard that ALSO adds AU base coverage, not in this * base-locale shard. `synthesizers/boundary-stress.test.ts` proves the alignments. */ import type { CanonicalRow } from "@mailwoman/corpus/types"; export type BoundaryStressTemplate = "street-eats-affix" | "comma-less-city-state" | "fr-prefix" | "house-number-after-street" | "bare-locality" | "house-number-before-street"; export interface BoundaryStressBaseTuple { locality: string; region: string; postcode: string; country: string; } export interface BoundaryStressSynthesisOpts { random?: () => number; /** * Force a specific shape (tests + balanced shard composition). */ forceTemplate?: BoundaryStressTemplate; } export interface SynthesizedBoundaryStressRow { raw: string; components: CanonicalRow["components"]; locale: string; template: BoundaryStressTemplate; } /** * Synthesize one boundary-stress row. `base` is optional — when omitted, a locale-appropriate tuple is drawn from the * internal pools (so the generator is self-contained; a build script can pass real tuples for scale + diversity). Every * component value is a verbatim substring of `raw`, so `alignRow` locates + BIO-labels it. */ export declare function synthesizeBoundaryStressRow(base: BoundaryStressBaseTuple | undefined, opts?: BoundaryStressSynthesisOpts): SynthesizedBoundaryStressRow; //# sourceMappingURL=boundary-stress.d.ts.map