# Changelog

## 0.8.0 (2026-05-04)

### Features

- **`dryRun(sql)` on the DuckDB context.** New pre-execution validator that runs `DESCRIBE (<sql>)` to parse + bind + plan a SELECT without executing it, returning `{ ok, columns }` on success or `{ ok: false, error }` on a binder error. Used by widget-flow's pre-render gate so missing-column / wrong-UNNEST-shape / type-mismatch errors are caught + sent to fix-query before the full scan, instead of after the result lands. Also exported as the standalone helper `dryRunQuery(conn, sql)` and the `DryRunResult` / `DryRunOk` / `DryRunErr` types.

## 0.7.0 (2026-04-24)

### Features

- **Smart XLSX layout detection (Stage 1).** The ingest worker no longer blindly picks sheet 0 + row 0 on XLSX/XLS files. A new `xlsx-detect` module analyzes every sheet and scores candidate layouts on density, sheet-name hints (`data|raw|records` vs `cover|toc|notes|readme`), hidden-sheet penalty, header-row heuristics (type divergence between header row and the 5 rows below, banner-gap bonus, short-string column check), data-range detection (walk down until an empty row, a trailing-total row, or a filled-ratio drop), and column-type consistency. The worker now auto-picks the winning sheet + header row + data range and passes the range to `sheet_to_json`, so files with title banners, offset data ranges, or a cover page as sheet 0 ingest correctly instead of silently producing garbage. Clean single-sheet-A1-no-merges exports hit a fast path and skip detection overhead. No protocol changes visible to hosts; phase event now briefly passes through `"detecting"` between `"reading"` and `"parsing"`.

### Changes

- `FilePhaseMessage.phase` (and the mirror in `IngestProgressEvent`) widened from `"reading" | "parsing"` to `"reading" | "detecting" | "parsing"`. Hosts doing exhaustive switches over ingestion phases need a new arm for `"detecting"`.

## 0.6.1 (2026-04-22)

### Fixes

- **`defaultRowCap` no longer breaks non-SELECT queries.** 0.6.0 appended `LIMIT N` to every query routed through `runQuery`, including `SHOW TABLES`, `DESCRIBE`, `PRAGMA`, `CREATE`, `INSERT`, `COPY`, and `EXPLAIN` — all of which reject a trailing LIMIT (`Parser Error: syntax error at or near "LIMIT"`). The cap now only fires when the first non-comment keyword is `SELECT` or `WITH`. Also skips when the query ends in `OFFSET N` (DuckDB rejects `OFFSET N LIMIT M`).

## 0.6.0 (2026-04-22)

### Features

- **`defaultRowCap` prop on `<DuckvizDBProvider>`** — when set to a positive number, every `runQuery(sql)` call appends `LIMIT <cap>` unless the SQL already ends in a `LIMIT N` clause. Single chokepoint for protecting the tab from unbounded LLM-generated SELECTs. Defaults to `0` (disabled). The check is conservative — only a trailing `LIMIT N [OFFSET N]` is detected, so nested CTE limits still get capped at the outer level.
- **Ingest stall detection.** The ingest worker now posts an `ingest-stalled` message (path, table, batch index, waited-ms) when a batch acknowledgement does not arrive within 15 s. Surfaces via `IngestProgressEvent` from `run-ingest` so hosts can warn the user instead of staring at a frozen progress bar. Does not abort the ingest — it keeps waiting.

## 0.5.2 (2026-04-21)

### Features

- **`getPersistedStateSize()`** — new export from `@duckviz/db/internal` that returns `{ blobs, keyval, totalBytes }`. Host apps can use it to verify a clear actually landed (both stores empty) before navigating, instead of trusting that the clear resolved cleanly.

### Fixes

- **"Clear All Data" no longer requires two clicks.** The 0.5.1 fix made each `tx()` close its db handle on `oncomplete`, but `clearAllPersistedState()` was still `Promise.all([idbClear(STORE_BLOBS), idbClear(STORE_KEYVAL)])` — and each `idbClear` awaited the **request's** `onsuccess`, which fires BEFORE the **transaction's** `oncomplete`. So `await clearAllPersistedState()` returned with both IDBDatabase handles still open, and the next line (`await deleteAllPersistedState()`) fired `indexedDB.deleteDatabase()` while they were still open → `onblocked` → `deleteAllPersistedState` silently resolved without the delete actually running → the UI reloaded into stale data → user had to click Clear again. Two changes: (1) `clearAllPersistedState` now uses a single multi-store transaction, awaits `oncomplete` explicitly, and closes the db in `finally`; (2) `deleteAllPersistedState` now waits for `onsuccess` rather than bailing on `onblocked` (with a 3s timeout as a safety valve), and logs blocks/errors instead of swallowing them.
- **`validateParsedData` now flags the "0 lines parsed" case as a fatal error** regardless of total line count. The previous `total > 10` guard only caught large files, so small log files (e.g. a ~150-line SNMP trap log) where the AI-detected format matched nothing would return `{ ok: true }`, letting the caller run a zero-iteration ingest loop, create no DuckDB table, and still mark the file "done". The AI panel then sat stuck on "Couldn't find an ingested table" with a retry that couldn't help.

## 0.5.1 (2026-04-21)

### Fixes

- **IDB connection leak in `idb-storage`** — every `idbGet`/`idbSet`/`idbClear` opened a fresh `IDBDatabase` handle and never closed it. Over the course of a session those leaked handles accumulated and silently blocked `indexedDB.deleteDatabase()` during the host app's "Clear everything" flow, forcing users to click clear twice before the IDB actually went away. Each transaction now closes its connection on `oncomplete`/`onerror`/`onabort`.

## 0.5.0 (2026-04-19)

### Features

- **Lazy dataset loaders** — `DuckvizDataset.data` now accepts an async function `() => Promise<Record<string, unknown>[]>` in addition to a row array. Loader runs only when the dataset becomes active in `useAutoIngest`.
- **`useAutoIngest({ activeId })`** — optional selection-aware mode. Array-data datasets still ingest eagerly on mount; function-data datasets only ingest when their derived `tableName` matches `activeId`. Loaded rows are cached per-tableName, so re-selecting never re-fetches.
- Exports new `DatasetRowsLoader` type.

## 0.4.0 (2026-04-17)

### Features

- **`useAutoIngest` hook** — auto-ingest `DuckvizDataset[]` into DuckDB on mount. Derives table names from `name` via `pathToTableName()`. Drops tables on unmount by default (`dropOnUnmount`). Handles React Strict Mode.
- **`dropTable(tableName)`** — new method on `DuckDBContextValue` for dropping a single table
- **`DuckvizDataset` type** — shared `{ name, data }` interface for passing row data to components

## 0.3.1 (2026-04-17)

### Changes

- **Bundle size optimization** — dist reduced from 2.7 MB to 1.8 MB: added `cleanStaleChunks` plugin, externalized `papaparse`, `xlsx`, `fast-xml-parser`, and `zustand` (moved to peerDependencies), dropped CJS format (ESM-only)

## 0.3.0 (2026-04-15)

### Features

- **`deleteAllPersistedState()`** — new internal export that fully deletes the `duckviz-persist` IDB database (instead of just clearing its records). `idbClear()` empties rows but browsers still report the allocated page quota via `navigator.storage.estimate()`, so UIs that read that value (memory badges etc.) kept showing pre-clear footprints until the database itself was deleted. Resolves on `onblocked`/`onerror` so the surrounding flow never hangs. Pair with `clearAllPersistedState()`: clear first (so pending Zustand writes flush cleanly), then delete.

## 0.2.5 (2026-04-13)

### Fixes

- **Resolve `workspace:*` in published package** — devDependencies no longer contain unresolvable `workspace:*` specifiers that break consumers using bun

## 0.2.4 (2026-04-12)

### Fixes

- **Inline `@duckviz/types` in DTS** — type declarations no longer reference the unpublished `@duckviz/types` package

## 0.2.3 (2026-04-12)

### Fixes

- **Fix `@duckdb/duckdb-wasm` peer dep** — changed from `^1.33.0` (no stable release exists) to `*` so npm can resolve dev versions
- **Disable sourcemaps** — `.map` files no longer included in dist

## 0.2.2 (2026-04-12)

### Fixes

- **Remove condition gate from `./internal`** — `@duckviz/db/internal` subpath is now a regular export, resolvable by npm consumers without custom bundler conditions

## 0.2.1 (2026-04-12)

### Fixes

- **Bundle internal types** — `@duckviz/types` is now bundled into dist instead of externalized, so npm consumers don't need to install unpublished packages
- **Resolve `workspace:*` references** — peer/dependency version ranges now use semver instead of workspace protocol

## 0.2.0 (2026-04-12)

### Breaking Changes

- **Removed `configureDB()`** — all configuration is now via props on `<DuckvizDBProvider>` (`persistence`, `arrowIngest`, `batchSize`, `logLevel`)
- **Public API pruned to 10 exports** — internal helpers moved to `@duckviz/db/internal` (condition-gated, not resolvable by external consumers)

### Features

- **`useDuckDB().ingest()`** — unified file ingestion API with progress callback
- **`useDuckDBOrNull()`** — nullable variant for optional provider contexts
- **Subpath exports** — `@duckviz/db/internal` for sibling packages, gated behind `duckviz-internal` export condition

## 0.1.0 (2026-04-10)

Initial release. Extracted from `apps/app` into a standalone package.

### Features

- **DuckvizDBProvider** — React context provider that initializes DuckDB-WASM and exposes query/ingestion/schema APIs via `useDuckDB()` hook
- **File ingestion worker** — Self-contained Web Worker (blob URL) for CSV, XLSX, JSON, JSONL, XML parsing off the main thread with back-pressure via `batch-ack`
- **Arrow IPC ingestion** — Fast batch ingestion path with automatic JSON fallback
- **IndexedDB persistence** — Parquet table snapshots, file ArrayBuffer storage, Zustand adapter, and session detection
- **Log parser** — JSON, regex pattern, and delimited line parsing with data quality validation
- **Core utilities** — `pathToTableName()`, Arrow-clean `toCleanJSON()`, `toPlainValue()`
