/** * Exodus migration engine. * * `runExodusMigrate()` performs the actual data migration from legacy DBs * to the consolidated dual-scope `cleo.db`. Key invariants: * * - Source DBs are opened **read-only** via `openCleoDbSnapshot` (AC4). * - Source files are backed up to the staging dir before any writes (AC5). * - Import is wrapped in `BEGIN … COMMIT` per source; partial failure leaves * the target DB untouched (AC6). * - Idempotency keys are propagated where the source row has them; generated * where it does not (AC7). * - The staging journal is written atomically before each table copy so a * crash can be resumed (AC5). * * ## Type coercion — epoch INTEGER → ISO-8601 TEXT (ROOT CAUSE 1 fix — T11546) * * Many legacy tables store timestamps as INTEGER epoch values (seconds or * milliseconds). The consolidated schema declares these columns as `text` with * a CHECK constraint: `CHECK ("col" IS NULL OR "col" GLOB '[0-9][0-9][0-9][0-9]-[0-9][0-9]-[0-9][0-9]*')`. * * When a source INTEGER value is inserted into a target TEXT+GLOB column, SQLite * coerces the integer to its decimal string representation (e.g. `"1717200000"`), * which fails the GLOB check. `INSERT OR IGNORE` then SILENTLY DROPS the entire * row. Result: all conduit_messages, brain_observations, etc. → 0 rows copied * while migrate reports `success: true, rowsCopied: 0`. * * Fix (two parts): * (a) Per-column value transform: `detectEpochToIsoColumns()` reads the target * table DDL from `sqlite_master` to identify columns with an ISO GLOB * CHECK constraint. For those columns, if the source type affinity is * INTEGER, the SELECT expression applies `strftime('%Y-%m-%dT%H:%M:%fZ', * col, 'unixepoch')` (seconds) or the `/1000.0` ms variant depending on * the per-source/table heuristic. * (b) No-swallow assertion: after the bulk INSERT OR IGNORE, the actual * `changes` count is compared against the source row count. Any shortfall * is surfaced as a hard table-level error (not a silent success). * PK/UNIQUE conflicts on resume are tolerated (they are expected and safe * to ignore); unexpected shortfalls are flagged with a detailed error. * * ## ATTACH-once-per-source design (P0 fix — T11531) * * Each legacy source DB is ATTACHed to the target handle ONCE using a unique * per-source alias, all tables from that source are copied under the single * attachment, and then the alias is DETACHed. The ATTACH and DETACH are * performed **outside** the BEGIN/COMMIT transaction block because SQLite * forbids DETACH inside an active multi-statement transaction. The INSERT * statements themselves are issued inside the transaction for atomicity (AC6). * * Prior implementation called ATTACH/DETACH per-table inside an open * transaction — DETACH silently failed (SQLite restriction), the alias stayed * attached, and every subsequent table for the same source threw * "database _exodus_src_ is already in use", causing ~80 % data loss. * * ## FK-defer bulk copy (ROOT CAUSE 1 fix — T11533) * * Legacy tasks.db has self-referential FKs (`tasks.parent_id → tasks.id`), * cross-table FKs, and ~18 tables with FK relationships. When FK enforcement * is ON and tables are copied in arbitrary order (children before parents), * each child INSERT fails with FOREIGN KEY constraint error (SQLite errcode 787) * and the table is silently skipped — causing ~114K rows of data loss. * * Fix: set `PRAGMA foreign_keys = OFF` on the target connection before any * bulk INSERT, then after all tables are committed run `PRAGMA foreign_key_check` * to validate referential integrity. Genuine orphan rows surface as verify * failures; child-before-parent ordering artifacts are NOT dropped. * `PRAGMA foreign_keys = ON` is restored after the check. * * ## Name-mapping (ROOT CAUSE 1 fix — T11532) * * Legacy source DBs use UNPREFIXED table names (`tasks`, `messages`, `skills`, * …) while the consolidated `cleo.db` uses DOMAIN-PREFIXED names * (`tasks_tasks`, `conduit_messages`, `skills_skills`, …). The * `resolveConsolidatedTableName()` function from `table-name-map.ts` performs * the deterministic legacy→consolidated mapping before every copy. Tables with * no consolidated home emit an explicit WARN journal entry rather than being * silently discarded. * * ## Column-drift tolerance (ROOT CAUSE 2 fix — T11532, hardened T11533) * * When the source and target schemas differ (consolidated schema added/changed * columns vs legacy), the copy uses the INTERSECTION of source and target * column names. New target-only columns take their schema defaults; old * source-only columns are dropped. This is implemented by introspecting both * schemas via `PRAGMA table_info` and building an explicit column list. * * **NOT NULL / no-default hazard (T11533)**: target-only NOT NULL columns * WITHOUT a schema default caused `INSERT OR IGNORE` to silently drop rows * whose source value was NULL for that column (constraint violation → IGNORE). * The fix: for each intersection column that is NOT NULL in the target AND * has no `dflt_value`, the SELECT clause wraps the source reference in * `COALESCE(src_col, type_default)` so no row is silently dropped. * A type_default of `''` is used for TEXT affinity and `0` for numeric. * * ## Advisory file lock (AC4) * * The source DB files are opened read-only via `openCleoDbSnapshot` which * calls `new DatabaseSync(path, { readOnly: true })`. Node's SQLite binding * opens with `SQLITE_OPEN_READONLY`, which prevents any writes from this * process. We additionally write a `.lock` sentinel file next to each source * DB for the duration of the migration so that other CLEO processes can detect * an in-progress exodus and refuse to write. * * @task T11248 (E5 · SG-DB-SUBSTRATE-V2) * @task T11531 (P0 attach-leak fix) * @task T11532 (P0 name-mapping + column-drift + verify-rowid fix) * @task T11533 (P0 FK-defer + NOT NULL coalesce + signaldock-global map + nexus hash fix) * @task T11547 (P0 enum normalization — 7,421 rows recovered) * @task T11548 (P0 final enum coverage — 285 remaining rows zero-loss) * @task T11549 (P0 zero-loss final mile — brain_decisions enums + seconds-epoch coercion + * agent_credentials/brain_release_links tables) * @saga T11242 */ import type { ExodusMigrateResult, ExodusPlan } from './types.js'; /** * Invalidate the migrate journal after an aborted/rolled-back migration so a * post-abort retry RE-COPIES every table from scratch (T11572). * * ## Why this is required for retry correctness * * `runExodusMigrate` is resumable (AC5): after each table copy it writes a * journal entry with `status: 'done'`, and a subsequent run SKIPS any table * already marked `done`. That resume optimisation is correct only while the * consolidated rows it copied still EXIST. When the parity gate aborts, the * exodus-on-open hook truncates the consolidated tables back to EMPTY * (`rollbackConsolidatedToEmpty`) — but the on-disk journal still says every * table is `done`. The next open therefore re-triggers (consolidated empty), * runs `runExodusMigrate` again, finds every table `done` in the journal, copies * NOTHING, re-verifies the still-empty target, and re-aborts — a permanent loop * that pays the full migrate+verify cost on every open. * * Deleting the journal file forces `runExodusMigrate` to `initJournal()` afresh, * so the retry actually re-copies. Idempotent and best-effort: a missing journal * (or a missing staging dir) is a no-op. The staging backups are intentionally * LEFT in place (they are the legacy source-of-truth snapshots — harmless to * keep, valuable for forensics); only the progress journal is cleared. * * @param stagingDir - The staging directory whose `exodus-journal.json` to clear. * @returns `true` if a journal file was removed, `false` if there was nothing to * remove. * * @task T11572 (abort/rollback must invalidate journal so retry re-copies) * @epic T11249 (E6) * @saga T11242 */ export declare function clearExodusJournal(stagingDir: string): boolean; /** * Run the exodus migration. * * @param plan - Pre-flight plan from `buildExodusPlan()`. * @param forceCrossVersion - Skip the schema-version guard (AC9). * @param onProgress - Optional progress callback called after each table. * * @returns {@link ExodusMigrateResult} * * @task T11248 (AC4, AC5, AC6, AC7, AC9) * @task T11531 (P0 attach-leak fix) */ export declare function runExodusMigrate(plan: ExodusPlan, forceCrossVersion?: boolean, onProgress?: (msg: string) => void): Promise; //# sourceMappingURL=migrate.d.ts.map