---
summary: "Consolidate spillover onto canvas-only dataframes — remove brapi_manage_dataset and the brapi://dataset/{id} resource, mandate DuckDB, gate brapi_dataframe_drop opt-in, and return typed columns from brapi_dataframe_query."
breaking: true
---

# 0.5.0 — 2026-05-05

Collapses the dual-store spillover model. Pre-canvas, `find_*` tools wrote spilled rows twice — once as JSON in `ctx.state` (driving `brapi_manage_dataset`) and once as a DuckDB table on the canvas (driving `brapi_dataframe_*`). The two surfaces did the same job, so this release retires the JSON store and makes SQL the single way to work with spilled results. DuckDB is now a regular dependency rather than an optional peer; the server fails fast at startup when it isn't available. Resolves [#25](https://github.com/cyanheads/brapi-mcp-server/issues/25), subsumes [#24](https://github.com/cyanheads/brapi-mcp-server/issues/24).

## Migration

| Before (≤ 0.4.x) | After (0.5.0) |
|:------------------|:---------------|
| `brapi_manage_dataset { mode: 'list' }` | `brapi_dataframe_describe` (no args) |
| `brapi_manage_dataset { mode: 'summary', datasetId }` | `brapi_dataframe_describe { dataframe: 'df_<uuid>' }` |
| `brapi_manage_dataset { mode: 'load', datasetId, page, pageSize, columns }` | `brapi_dataframe_query { sql: 'SELECT col1, col2 FROM df_<uuid> LIMIT 100 OFFSET 0' }` |
| `brapi_manage_dataset { mode: 'delete', datasetId }` | `brapi_dataframe_drop { dataframe }` when `BRAPI_CANVAS_DROP_ENABLED=true`; otherwise wait for canvas TTL |
| `brapi://dataset/{id}` resource | `brapi_dataframe_describe { dataframe }` |
| `find_*` tool output `dataset.datasetId` | `find_*` tool output `dataframe.tableName` |

## Removed

- **Tool: `brapi_manage_dataset`.** The lifecycle surface for ctx.state-stored datasets is gone. Use `brapi_dataframe_describe` to discover dataframes, `brapi_dataframe_query` to read or aggregate, and `brapi_dataframe_drop` (opt-in, see Added) to release one explicitly.
- **Resource: `brapi://dataset/{datasetId}`.** Provenance now lives on the dataframe itself; describe surfaces source / baseUrl / query / createdAt / expiresAt under `provenance` for any auto-registered `df_*` table.
- **Service: `src/services/dataset-store/`** (entire module). Spilled rows go directly to the canvas — no parallel JSON copy.
- **Env var: `BRAPI_CANVAS_ENABLED`.** Canvas is mandatory now. Startup fails with a clear `ConfigurationError` when `core.canvas` is undefined (e.g., DuckDB unavailable).
- **Env var: `BRAPI_DATASET_STORE_DIR`** (filesystem path for the deprecated DatasetStore). No longer read.
- **Helper: `spillToDataset` in `src/mcp-server/tools/shared/find-helpers.ts`.** Replaced by `spillToCanvas`. The shared `DatasetHandleSchema` / `renderDatasetHandle` / `toDatasetHandle` exports are renamed to their `Dataframe*` equivalents.
- **`docs/compatibility.md`.** Stale live-server probe matrix dropped — to be regenerated as needed against the current dialect surface. README cross-link removed.

## Added

- **Env var: `BRAPI_CANVAS_DROP_ENABLED`** (default `false`). Opt-in for `brapi_dataframe_drop` — when unset, the tool is omitted from `tools/list` entirely and dataframes expire via TTL when left unmanaged. Resolves [#24](https://github.com/cyanheads/brapi-mcp-server/issues/24).
- **`brapi_dataframe_query` returns typed columns.** Output `columns` field changes from `string[]` (names only) to `{ name: string; type: string }[]` (name + SQL/DuckDB type). Types come from DuckDB's describe in both directions: when `registerAs` is set the user-named table is described directly, and when it isn't the handler runs the query under an internal `_brapi_probe_<uuid>` `registerAs` so DuckDB's authoritative types survive the JSON round-trip (which would otherwise flatten BIGINT and other non-JSON-native types to VARCHAR strings), then drops the probe before returning. The agent learns the schema implicitly through any query response — no need to round-trip through `dataframe_describe` between queries.
- **Server-side default `CANVAS_PROVIDER_TYPE=duckdb`.** The framework defaults this to `none`, which would fail every startup now that canvas is mandatory. The entry point sets the env var to `duckdb` before `createApp` reads config, so the server works out-of-the-box without operator setup. Operator-set values pass through untouched (including `none` for diagnostic runs that should fail closed).
- **`brapi_raw_get` and `brapi_raw_search` spill to a canvas dataframe.** When the upstream advertises more rows than `loadLimit` AND the result is a list shape (top-level array or BrAPI `result.data` envelope), the handler walks subsequent pages via `spillToCanvas` and attaches a `dataframe` handle on the response — same `df_<uuid>` shape as `find_*`. Inline `result` is returned **unchanged** so the raw escape hatch keeps its semantics; the dataframe is the "and there is more, here" pointer. Both tools now accept an optional `loadLimit` input (defaults to `serverConfig.loadLimit`, 1,000). Spillover is intentionally skipped when the caller drives paging via `params.page` / `params.pageSize` (raw_get) or `body.page` / `body.pageSize` (raw_search) — they are walking pages explicitly. Non-list results (single objects, scalars) pass through as before regardless of total count. New `extractListRows` helper in `find-helpers.ts` factors out the list-shape detection.
- **Tool: `brapi_dataframe_export`.** Write a dataframe to disk in CSV, Parquet, or JSON for human review — the agent calls it when the researcher signals they want to open the data in Excel, Tad, DuckDB CLI, etc. Returns the absolute path; the file lands inside the operator-configured `BRAPI_EXPORT_DIR`. Optional `columns` (thin projection) or `sql` (full SELECT, mutually exclusive) materializes a derived canvas table for the export and drops it after. Default filename is `<dataframe>-<unixSeconds>.<ext>` so re-exports never collide; explicit `filename` overwrites on collision. Best-effort sweep of stale files (mtime past `BRAPI_DATASET_TTL_SECONDS`) runs on each call; `brapi_dataframe_drop` unlinks paired export files alongside the dataframe. Resolves [#18](https://github.com/cyanheads/brapi-mcp-server/issues/18).
- **Env var: `BRAPI_EXPORT_DIR`.** Setting a writable path is the opt-in for `brapi_dataframe_export` (no separate enable flag). Bridged automatically to the framework's `CANVAS_EXPORT_PATH` so the existing canvas path-traversal sandbox (absolute paths and `..` segments rejected) governs filename inputs. Tool registration also requires `MCP_TRANSPORT_TYPE=stdio`; under HTTP transport the tool stays disabled regardless of this value because the returned path lives on the server, not the user. Operators see two distinct gate reasons in `/.well-known/mcp.json` — "stdio transport required" and "BRAPI_EXPORT_DIR unset" — depending on which check failed.
- **`CanvasBridge.export(ctx, tableName, target, options)`** wraps the framework's `instance.export()` with optional path tracking via `CanvasBridge.trackExport(ctx, sourceName, path)`. Tracked paths key off the *source* dataframe so `brapi_dataframe_drop(ctx, sourceName)` unlinks the file alongside the canvas table — the bridge's `drop()` reads `brapi/canvas/exports/<name>` and unlinks before deleting the state entry. Tracking is caller-driven (the export tool tracks; the bridge does not auto-track) so a transient projection table can be exported then dropped without unlinking the file.

## Changed

- **`find_*` tool output: `dataset` field renamed to `dataframe`.** Shape simplified: drops `datasetId` and `sizeBytes`; keeps `tableName`, `rowCount`, `columns`, `createdAt`, `expiresAt`, `truncated`, `maxRows`. Eight tools affected: `brapi_find_germplasm`, `brapi_find_studies`, `brapi_find_observations`, `brapi_find_locations`, `brapi_find_variables`, `brapi_find_variants`, `brapi_find_images`, `brapi_find_genotype_calls`.
- **`find_*` tool descriptions emit "dataframe handle" language.** Render output points at `brapi_dataframe_describe` and `brapi_dataframe_query` instead of `brapi_manage_dataset`.
- **`brapi_dataframe_query` description: explicit SQL-as-paging guidance.** `LIMIT/OFFSET`, projection (`SELECT col1, col2`), and aggregation (`COUNT`, `GROUP BY`, `AVG`) are the paging idiom — no need for a structured-paging shim on top.
- **`brapi_dataframe_describe` description: leads with "start here after a spillover."** The discovery surface is the natural first step after a `find_*` call returns a `dataframe` handle.
- **`brapi_dataframe_drop` description and error contract.** Drops the `dataframe_disabled` typed error (canvas is mandatory now); idempotent semantics unchanged — returns `dropped: false` rather than failing for unknown names.
- **`brapi_dataframe_describe` provenance schema: drop `datasetId`.** The dataframe name (`df_<uuid>`) is the only identity now. `provenance` exposes `source`, `baseUrl`, `query`, `createdAt`, `expiresAt`.
- **Internal canvas table prefix: `ds_` → `df_`** (in `canvas-bridge.ts`). Matches the new noun. The agent always sees the full table name from the handle, so this is a purely internal rename.
- **Prompts updated.** `brapi_eda_study` and `brapi_meta_analysis` now point at `brapi_dataframe_query` (with SQL paging guidance) instead of `brapi_manage_dataset` for accessing spilled rows.
- **Raw routing hints (`raw-routing-hints.ts`) updated** to reference dataframe spillover instead of dataset spillover.
- **`BRAPI_LOAD_LIMIT` default raised 200 → 1,000.** `loadLimit` doubles as the upstream `pageSize` during spillover walks, so the prior default silently capped dataframes at `200 × 50 pages = 10,000` rows even though `MAX_SPILLOVER_ROWS = 50,000` was the documented ceiling. The new default lines up the math: `1,000 × 50 = 50,000`, matching the cap. Operators on small/test BrAPI servers can lower it via env; raising it above 1,000 is rarely useful because most BrAPI servers cap server-side `pageSize` at 1,000.
- **`brapi_find_genotype_calls` honors `BRAPI_LOAD_LIMIT`.** The handler had a hardcoded `?? 200` fallback that ignored the configured default. Now reads `getServerConfig().loadLimit` like every other find_* tool. Tool description and `loadLimit` input description both updated to drop the stale "default 200" claim. Genotype calls remain a special case in one respect: upstream pageSize is fixed at 10,000 (decoupled from `loadLimit`), so the dual-role footgun other find_* tools have doesn't apply here — the input describe calls this out explicitly.
- **`LoadLimitInput.describe()` rewritten for honest agent guidance.** The shared input fragment now warns that lowering `loadLimit` to "see fewer rows inline" silently shrinks dataframe spillover capacity proportionally (because `loadLimit` doubles as upstream `pageSize`), and points the agent at SQL `LIMIT` on the dataframe for sampling instead. Drops env-var names and specific numeric values from the agent-facing copy.
- **`brapi_meta_analysis` prompt drops hardcoded `loadLimit: 200`.** The Step 2 directive let the deployment default win instead of pinning a number that's no longer the default. One-line fix.

## Dependencies

- **`@duckdb/node-api` promoted from optional peer to regular dependency.** Spillover is the primary value of this server; making DuckDB optional only protected users we don't target. Cloudflare Workers deployments are not supported until a non-DuckDB canvas provider lands upstream — documented as a known constraint.