/** * Periodic compaction for the LanceDB-backed search indexes (change: bulletproof-background-index). * * Every incremental update is a `delete` + `add`, and LanceDB is append-only and versioned: each * one leaves the previous version and its data fragments on disk. Nothing reclaimed them. A full * `analyze` happens to clean up — it rebuilds the directory from scratch — but a machine that just * leaves the watcher running never gets there, which is exactly the "always on in the background" * case this tool is meant for. * * Measured on a real working repository whose SOURCE is about 9 MB, after a few weeks of watching: * * | index | on disk | live rows | versions | after compaction | * |-------------------|---------|-----------|----------|------------------| * | `text-line-index` | 401 MB | 234,267 | 953 | 36 MB | * | `vector-index` | 62 MB | 8,372 | 952 | 14 MB | * * Roughly 90% of what the tool had written was reclaimable garbage, and it grows without bound. * * Two deliberate choices: * * - Compaction runs on a COUNTER, not every update. It costs a few hundred milliseconds on the * indexes above, which is fine occasionally and is not fine on every keystroke-save. * - Old versions are kept for {@link VERSION_GRACE_MS} rather than dropped outright. A reader * that opened the table a moment ago is still pinned to the version it opened; `cleanupOlderThan: * new Date()` would delete the files out from under it. The current version is never removed by * LanceDB regardless, so the grace window only protects concurrent readers. */ /** Minimal view of the LanceDB table surface this module needs. */ interface CompactableTable { optimize(options?: { cleanupOlderThan?: Date; deleteUnverified?: boolean; }): Promise; } /** * Record one incremental update against `dbPath`, and compact if enough have accumulated. * * Best-effort by construction: compaction is a space optimization, never a correctness * requirement, so a failure is swallowed and the counter still resets. The alternative — letting * an optimize error surface — would turn a disk-space chore into a failed file save. */ export declare function noteUpdateAndMaybeCompact(dbPath: string, table: CompactableTable): Promise; /** Test-only: lower the threshold so a small fixture reaches it. Returns the previous value. */ export declare function _setCompactEveryForTesting(n: number): number; /** Test-only: shrink the reader-safety window so a fresh fixture can be reclaimed. */ export declare function _setVersionGraceMsForTesting(n: number): number; /** Test-only: forget the counters so cases cannot leak into each other. */ export declare function _resetCompactionCountersForTesting(): void; export {}; //# sourceMappingURL=index-compaction.d.ts.map