/** * @copyright Sister Software * @license AGPL-3.0 * @author Teffen Ellis, et al. * @file Rewrite a parquet file's `source` column from the retired spelling to the current one. * * A recipe-output `source` is a wire identifier. Every row of every corpus carrying it stores the value. * Every training config that weights it keys on it. Every model card trained on it quotes it. The * retired `synth-*` spelling labels a row fabricated where most of these rows are attested records a * recipe rendered, spliced or fragmented. `RECIPE_SOURCES` in `#recipes/sources` holds both spellings and * the operation each recipe performed. * * The rewrite runs over an assembly's staging files before `overlay-manifest` computes each file's * `sha256`, so the manifest describes the renamed bytes. A corpus already assembled is never rewritten in * place. Its manifest records digests over the bytes it has. The configs that target it keep the * spelling it stores. * * **What the verification establishes.** The check compares row count, distinct `source` set and an md5 * over the ordered `source_id` column before and after. Together, those checks establish that the rewrite * changed the `source` column and the row order of no other column. They do not establish that the * output's other columns are unchanged. A full-file digest cannot show that either because the rewrite * re-encodes the file. */ import { type PathBuilderLike } from "path-ts"; /** * What one file's rewrite established. */ export interface SourceNameRewrite { parquet: string; rows: number; /** * The mapping applied, keyed by the retired spelling. * * A file whose rows already use current spellings is reported with an empty map and left untouched. */ renamed: Record; /** * The md5 over the ordered `source_id` column, identical before and after. */ sourceIDDigest: string; } /** * Rewrite one parquet file's `source` column in place, through a temporary output. * * @throws When a row has a `source` the table does not name under either spelling. * An unknown value is refused rather than preserved, because preserving it would leave one * file holding both spellings and no later reader could tell which rows were rewritten. * @throws When the row count, the distinct `source` count or the ordered-`source_id` * digest differs after the rewrite. */ export declare function rewriteSourceNames(parquetPath: PathBuilderLike): Promise; //# sourceMappingURL=source-names.d.ts.map