# DS4 Context Engine 0.3.0-beta.1

Status: published prerelease on 2026-09-03; exact registry verification passed.

This coordinated beta contains SQLite growth caused by persisted Context Manifests and adds explicit, recoverable offline maintenance for already-large derived databases. It also hardens compaction provider retries while preserving strict grounding, privacy filtering, canonical Pi JSONL semantics, atomic tool exchanges, one final Pi compaction entry, and fail-open delegation to Pi.

## Added

- Retains at most 128 Context Manifests globally and at most 200 calibration samples per exact provider/model/estimator profile.
- Bounds each online manifest prune to 32 rows and 8 MiB, while allowing one individually oversized oldest row to be removed so convergence cannot stall.
- Persists complete manifests up to 256 KiB; larger manifests retain complete included provenance and a deterministic excluded-only rollup with complete counts, aggregates, and digests plus at most 256 sampled excluded details.
- Skips manifest persistence above the 1 MiB hard bound without altering the active in-memory manifest or provider request.
- Exposes persisted inventory completeness through `getStored()` while legacy complete rows remain readable through `get()`.
- Adds metadata-only `/context storage` diagnostics and storage high-water integration in `/context health`.
- Adds cooperative database client leases and a two-phase maintenance lock; active or ambiguous clients block physical maintenance.
- Adds the `ds4-context-storage inspect|compact|recover --database <exact-path>` CLI. Mutating commands require a local TTY and have no non-interactive bypass.
- Adds standalone backup, working-copy transformation, validated `VACUUM INTO` candidate, protected-table full-row digests, recoverable swap state, and deterministic crash recovery.

## Changed

- Provider usage is authoritative in scalar Context Manifest columns; `message_end` no longer rewrites `manifest_json`.
- Only compaction failures categorized as `transport` are retried, with at most three attempts, abort-aware 200 ms and 500 ms delays, fresh routing session IDs, and cumulative usage accounting.
- Runtime startup during maintenance degrades safely to Pi fallback with categorical diagnostics rather than exposing raw SQLite messages.
- SQLite lock timeout and rollback diagnostics expose bounded categories, operation names, attempt counts, and numeric SQLite codes rather than raw database errors.
- Package verification rejects database, WAL, SHM, backup, staging, retired-swap, maintenance-state, client-lease, JSONL, `.pi`, and `.serena` paths.

## Safety and compatibility

Pi JSONL remains canonical and append-only. Physical maintenance never edits session history or project files and is unavailable to model-callable tools. Every `context_persistence` write still requires a fresh positive local UI decision, exact destructive targeting remains revision-bound, and canonical Pin/Memory mutations still append through Pi before SQLite reconciliation.

An individually oversized compaction group, unsupported exact value, non-transport provider failure, abort, or bounded-operation failure still delegates to Pi default compaction. Rejected values and provider payloads are not emitted in diagnostics.

SQLite schema remains 15. Migrations 1–15, `ds4-context-config-v1`, `runtime-adapter-v1`, `ds4-context-persistence-tool-v1`, and `ds4-context-persistence-result-v1` are unchanged. Earlier schema-15 readers can parse additive rollup metadata but may present sampled excluded details as complete, so historical excluded-inventory rendering is unsupported after downgrade once rollups have been written.

## Package/version policy

The coordinated version is `0.3.0-beta.1` for:

```text
ds4-context-core
ds4-context-reference-adapter
ds4-context-engine
```

Both adapters depend exactly on `ds4-context-core@0.3.0-beta.1`. Publication uses the explicit npm `beta` dist-tag for all three packages while `latest` must remain stable `0.2.0`. GitHub Actions remains validation-only with OIDC and package-write permissions denied.

## Candidate validation evidence

Local candidate verification on Node.js `26.5.1`:

- `npm run check`: 69 files and 341 tests passed.
- Three consecutive focused multi-process SQLite concurrency runs passed.
- `npm run quality:compare`: candidate quality `0.9875` versus baseline `0.808156`.
- `npm run schema:context-persistence`: 1,266 bytes and 317 estimated tokens; below the 1,500 absolute and 320 relative limits.
- `npm run latency:check -- <exact ds4-context-core@0.1.2>`: passed after clean install with ratio `0.93982`, at or below `1.10`, using 200 samples and 50 calls per sample.
- `npm run pack:check`: verified core (223 files), reference adapter (7 files), and Pi adapter (67 files) in a clean consumer.
- `npm pack --dry-run --json` for all three packages passed with the same inventories and no forbidden local/session/storage files.
- `git diff --check`: passed.
- Version, exact core dependencies, package-lock entries, extension constant, and reference-adapter constant are synchronized to `0.3.0-beta.1`.
- Protected compatibility golden, Pi JSONL fixture, migration definitions, canonical persistence contracts, and CI publication permissions are unchanged.

Large-database validation used a temporary private copy created from a read-only source connection; the live database was not compacted, vacuumed, rebuilt, replaced, or maintained:

- before: schema 15, `quick_check=ok`, 3,691 manifests, 2,101,666,663 manifest bytes, 2,489,868,288 database bytes;
- after: 128 manifests, 15,626,529 manifest bytes, 414 calibration samples across three profiles, 389,746,688 database bytes;
- all 128 retained oversized manifests were rewritten as explicit `excluded-rollup` projections with no irreducible oversize;
- the final candidate passed quick check and foreign-key validation while the standalone backup remained present;
- temporary copy, backup, and stage files were removed afterward, with no maintenance artifact created next to the live database or in the repository.

Validation-only CI run [`33752225991`](https://github.com/Alucard24/ds4-context-engine/actions/runs/33752225991) passed on Node.js `22.19.0` and `24.x` for committed candidate `4ef936f5a7e5c6afb63fd42d0b745d9201a7b87f` before publication.

## Registry evidence

- `npm run registry:check -- 0.3.0-beta.1`: passed against all three exact published versions, including exact adapter/core dependencies, public core exports, compiled reference conformance, packaged quality corpus, packaged Pi extension startup, and the installed `ds4-context-storage` CLI shim/usage probe.
- npm `beta` resolves to `0.3.0-beta.1` for all three packages.
- npm `latest` remains `0.2.0`, `alpha` remains `0.3.0-alpha.5`, and `rc` remains `0.2.0-rc.1` for all three packages.
- Published SHA-1 values are `fc5d6aaf2304291dc147ecc639281fb947390a51` (core), `7bfe0cf61a0f1213db058d5f89c1b7feb25db54a` (reference adapter), and `6abad04e2a2c30566c042f1e0829930c6546ed54` (Pi adapter).

Exact registry verification has passed; the annotated tag and GitHub prerelease remain pending.

## Documentation

- [`../CONTEXT_MANIFEST.md`](../CONTEXT_MANIFEST.md)
- [`../STORAGE.md`](../STORAGE.md)
- [`../STORAGE_MAINTENANCE.md`](../STORAGE_MAINTENANCE.md)
- [`../COMPACTION.md`](../COMPACTION.md)
- [`../ARCHITECTURE.md`](../ARCHITECTURE.md)
- [`../DOGFOODING_0.3.0_BETA.md`](../DOGFOODING_0.3.0_BETA.md)
- [`../RELEASING.md`](../RELEASING.md)
- [`0.3.0-alpha.5.md`](0.3.0-alpha.5.md)