# Developer Guide Comprehensive guide to `@git-stunts/git-cas` -- content-addressed storage backed by Git's object database, with encryption, compression, chunking, and vault management. For security posture and threat model, see [SECURITY.md](./SECURITY.md). For architecture and port/adapter internals, see [ARCHITECTURE.md](./ARCHITECTURE.md). For API reference, see [docs/API.md](./docs/API.md). For extension points and custom adapter checklists, see [docs/EXTENDING.md](./docs/EXTENDING.md). For store/restore state machines, see [docs/STORE_RESTORE_PIPELINE.md](./docs/STORE_RESTORE_PIPELINE.md). For advanced topics (benchmarks, Merkle trees, large-asset strategies), see [ADVANCED_GUIDE.md](./ADVANCED_GUIDE.md). --- ## Choose Your Path ### 1. Library Integration Embed content-addressed storage into your JavaScript/TypeScript application. - Start with [Library Quick Start](#library-quick-start) below. - Understand the port/adapter model: [ARCHITECTURE.md](./ARCHITECTURE.md). - Custom adapters: [docs/EXTENDING.md](./docs/EXTENDING.md). ### 2. CLI / TUI Usage Store, restore, and manage assets from the terminal. - Jump to [CLI Reference](#cli-reference). - Interactive explorer: `git-cas vault dashboard`. ### 3. Agentic Automation Machine-facing agent CLI for structured CI/CD or agentic workflows. - [docs/API.md](./docs/API.md) for the agent command surface. - Run: `git-cas agent `. ### 4. Contributing - Read `WORKFLOW.md`, `docs/method/process.md`, and `BEARING.md` for project process. - See [Architecture](#architecture) for orientation. --- ## Feature Coverage Map This guide and [ADVANCED_GUIDE.md](./ADVANCED_GUIDE.md) are the complete feature coverage pair. The quick path lives here; implementation mechanics, limits, and port contracts live in the advanced guide. | Feature Area | Covered Here | Advanced Coverage | | ----------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Facade lifecycle, JSON/CBOR factories, full constructor | [Library Quick Start](#library-quick-start), [Configuration Reference](#configuration-reference) | [Direct CasService and Custom Port Contracts](./ADVANCED_GUIDE.md#direct-casservice-and-custom-port-contracts) | | Opaque assets, pages, bundles, retention evidence, publication | [Library Quick Start](#library-quick-start), [Application Storage Lifecycle](#application-storage-lifecycle), [Structured Materializations](#structured-materializations) | [Application Storage API](./docs/API.md#application-storage) | | Direct `CasService` construction | [Configuration Reference](#configuration-reference) notes the facade/direct split | [Direct CasService and Custom Port Contracts](./ADVANCED_GUIDE.md#direct-casservice-and-custom-port-contracts) | | File, stream, tree, vault-safe store workflows | [Store Operations](#store-operations), [Vault Management](#vault-management) | [Store/Restore Pipeline](./docs/STORE_RESTORE_PIPELINE.md), [Manifest Integrity Hash](./ADVANCED_GUIDE.md#manifest-integrity-hash), [Merkle Manifests](./ADVANCED_GUIDE.md#merkle-manifests) | | Restore modes and bounded memory behavior | [Restore Modes](#restore-modes) | [Store/Restore Pipeline](./docs/STORE_RESTORE_PIPELINE.md), [Parallel Chunk Restore](./ADVANCED_GUIDE.md#parallel-chunk-restore), [Streaming Decompression](./ADVANCED_GUIDE.md#streaming-decompression) | | Fixed chunking, CDC, FastCDC normalization | [Chunking](#chunking) | [Content-Defined Chunking (CDC)](./ADVANCED_GUIDE.md#content-defined-chunking-cdc) | | Encryption schemes and legacy migration | [Encryption](#encryption), [Migrating Legacy Encryption Schemes](#migrating-legacy-encryption-schemes) | [Encryption Schemes](./ADVANCED_GUIDE.md#encryption-schemes), [Convergent Encryption](./ADVANCED_GUIDE.md#convergent-encryption) | | Passphrases, PBKDF2, scrypt, KDF policy | [Passphrase-Based Encryption](#passphrase-based-encryption) | [KDF Policy](./ADVANCED_GUIDE.md#kdf-policy) | | Multi-recipient envelope encryption and key rotation | [Multi-Recipient (Envelope) Encryption](#multi-recipient-envelope-encryption), [Key and Recipient Management](#key-and-recipient-management) | [Envelope Encryption](./ADVANCED_GUIDE.md#envelope-encryption) | | Vault init/list/resolve/remove, metadata, rotation, privacy mode | [Vault Management](#vault-management), [Vault Privacy Mode](#vault-privacy-mode) | [Vault Privacy Mode](./ADVANCED_GUIDE.md#vault-privacy-mode) | | Compression and custom adapters | [Compression](#compression) | [CompressionPort Architecture](./ADVANCED_GUIDE.md#compressionport-architecture) | | Manifest versions, format version, integrity hash, diffing | [Manifest Features](#manifest-features), [Manifest Diffing](#manifest-diffing) | [Format Version](./ADVANCED_GUIDE.md#format-version), [Manifest Integrity Hash](./ADVANCED_GUIDE.md#manifest-integrity-hash), [Manifest Diffing](./ADVANCED_GUIDE.md#manifest-diffing) | | Inspection, verification, referenced chunk analysis, deprecated aliases | [Manifest Features](#manifest-features), [Inspection and Verification](#inspection-and-verification) | [Security Hardening Summary](./ADVANCED_GUIDE.md#security-hardening-summary) | | Human CLI, TUI dashboard, agent CLI | [CLI Reference](#cli-reference) | [Operational Tooling](./ADVANCED_GUIDE.md#operational-tooling) | | Runtime support, ports, custom adapters, resilience policy | [Architecture](#architecture) | [Direct CasService and Custom Port Contracts](./ADVANCED_GUIDE.md#direct-casservice-and-custom-port-contracts), [Configuration Reference](./ADVANCED_GUIDE.md#configuration-reference) | | Release verification, migration, build stamping | [Migrating Legacy Encryption Schemes](#migrating-legacy-encryption-schemes) | [Operational Tooling](./ADVANCED_GUIDE.md#operational-tooling) | --- ## Library Quick Start A complete stage-retain-open cycle: ```js import { createReadStream } from 'node:fs'; import ContentAddressableStore, { AssetHandle } from '@git-stunts/git-cas'; const cas = await ContentAddressableStore.open({ cwd: '/path/to/repo' }); // Stage bytes and receive an immutable content handle. This creates no root. const staged = await cas.assets.put({ source: createReadStream('./photo.jpg'), slug: 'photos/vacation', filename: 'photo.jpg', }); // Make the graph Git-reachable through a current-generation RootSet. const retained = await cas.retention.retain({ handle: staged.handle, root: { ref: 'refs/cas/rootsets/photos', name: 'vacation' }, policy: 'pinned', }); // Stream the payload without exposing its manifest or chunk OIDs. for await (const chunk of cas.assets.open({ handle: staged.handle })) { consume(chunk); } console.log(retained.witness.root.generation); await cas.close(); ``` Node's file stream is an `AsyncIterable`; Bun and Deno callers can supply their native async byte streams through the same contract. In production code, call `close()` from `finally` or use async disposal. Closure releases local resources only and does not change stored objects or retention. ## Application Storage Lifecycle The ordinary application boundary separates content identity from retention: 1. `cas.assets.put()` streams bytes through the existing chunking, compression, encryption, and deduplication pipeline. It returns a `StagedAsset` whose immutable `AssetHandle` locates the resulting manifest tree. 2. A staged result says this operation created no reachability root. It does not claim that globally deduplicated objects are unreachable through every other ref. 3. `cas.retention.retain()` installs the handle under a RootSet and returns a `RetentionWitness` naming the exact generation and tree path observed. 4. `cas.publications.commit()` validates the handle graph, creates an application commit with caller-controlled ordered parents, and updates an explicitly allowed application ref by compare-and-swap. Handles have a canonical, repository-location-independent token form: ```js const token = staged.handle.toString(); const sameHandle = AssetHandle.parse(token); ``` The token is portable only with its referenced Git object graph. Opening it in a clone or mirror succeeds after the graph is transferred; opening it in a repository without that graph fails with `HANDLE_TARGET_MISSING`. ### Structured Materializations Use pages for bounded immutable index/trie nodes and bundles for a named, independently addressable materialization. `putOrdered()` accepts an iterable that is already sorted by canonical member path, so construction does not need to collect the complete member set. ```js const [nodes, edges] = await cas.pages.putBatch({ pages: [{ source: encodedNodePage }, { source: encodedEdgePage }], }); const materialization = await cas.bundles.putOrdered({ members: (async function* members() { yield ['edges/root', edges.handle]; yield ['nodes/root', nodes.handle]; yield ['state/frontier', encodedFrontier]; })(), }); for await (const chunk of cas.bundles.openMember({ handle: materialization.handle, path: 'nodes/root', })) { consume(chunk); } ``` Bundle fanout trees contain direct Git edges to every member graph. Retaining the bundle handle therefore retains its transitive pages and assets. Targeted lookup reads only the descriptor path and selected payload; retention and publication perform full bounded graph validation. Identical page bytes and identically constructed bundles produce identical handles. `pages.putBatch()` collects and validates the complete explicit batch before persistence. It accepts at most 256 pages and 32 MiB by default, preserves input order, and uses one scoped Git write session when the plumbing adapter supports it. Use individual `pages.put()` calls when inputs should not share one bounded memory envelope. Independent assets and ordered bundles have equivalent bounded write-wave surfaces: ```js const stagedAssets = await cas.assets.putBatch({ assets: assetRequests, maxBatchAssets: 4, maxBatchObjects: 4096, maxBatchBytes: 256 * 1024 * 1024, }); const stagedBundles = await cas.bundles.putOrderedBatch({ bundles: orderedBundleRequests, maxBatchBundles: 64, maxBatchMembers: 8192, maxBatchObjects: 256, maxBatchBytes: 64 * 1024 * 1024, }); ``` Asset batching keeps at most `maxBatchAssets` source pipelines active, shares bounded blob phases, and emits complete manifest trees in one final dependency wave. Bundle batching collects only the explicitly bounded request array, preplans its deterministic descriptor graph, and writes each bottom-up tree depth as one bounded wave. Both methods preserve input-order output and return no partial result array on failure. Higher `maxBatchAssets` values can reduce protocol round trips when the caller can afford more simultaneously live source pipelines; the default remains four. When later bundle waves depend on handles created by earlier waves, keep the intermediate handles inside one compound staging-workspace operation: ```js const admitted = await workspace.batch({ maxOperations: 4, operation: async (scope) => { const assets = await scope.assets.putBatch({ assets: assetRequests }); const pages = await scope.pages.putBatch({ pages: pageRequests }); const leaves = await scope.bundles.putOrderedBatch({ bundles: leafRequests(assets, pages), }); return (await scope.bundles.putOrderedBatch({ bundles: [rootRequest(leaves)], }))[0]; }, retain: (terminal) => [terminal], }); ``` `admitted.value` becomes caller-visible only with `admitted.retention`, after one exact workspace generation anchors the selected terminal plus every root retained before the compound call. Omit `retain` to preserve the retain-all behavior. A selector must synchronously return at least one canonical handle staged by that exact callback; it cannot import another operation's handle or remove a prior workspace root. The default operation ceiling is 64 and the hard ceiling is 1,024; each scope call also preserves its ordinary asset, page, or bundle batch bounds. Use the existing independently retained workspace methods when intermediate handles must leave the private callback. Repeated `pages.get()` calls reuse immutable payload reads within the store's bounded page cache. The defaults retain at most 128 payloads and 8 MiB; use `pageCacheEntries` and `pageCacheBytes` to tune that ceiling. Every result is a caller-owned copy. `pages.open()` remains uncached and streaming, and neither form of access establishes retention. ### Generic Application Publication Publication is disabled until the facade receives an explicit application-ref allowlist. The `refs/cas/*` namespace is always reserved. ```js const cas = await ContentAddressableStore.open({ cwd: '/path/to/repo', applicationRefPrefixes: ['refs/my-app/'], }); const publication = await cas.publications.commit({ root: staged.handle, commit: { message: 'Publish vacation photo', parents: previousCommitId === null ? [] : [previousCommitId], }, ref: { name: 'refs/my-app/assets', expected: previousCommitId, }, }); ``` `expected` is mandatory, including `null` for create-only publication. A stale head fails with `PUBLICATION_CONFLICT` and includes `expected`, `observed`, and the attempted commit ID. The returned witness is generation-scoped: it proves what the ref reached when publication succeeded, not that a mutable ref still points there later. `pinned` is a retention-policy term. It means ordinary cache eviction must not release the entry; it does not create or imply a Git pack `.keep` file. ### Factory Methods | Factory | Codec | Use Case | | -------------------------------------------------- | ----- | ------------------------------------ | | `await ContentAddressableStore.open({ cwd })` | JSON | Shortest normal application setup | | `ContentAddressableStore.createJson({ plumbing })` | JSON | Human-readable manifests, debugging | | `ContentAddressableStore.createCbor({ plumbing })` | CBOR | Compact binary manifests, production | `open()` is async and accepts `cwd` plus the same facade options as the constructor except `plumbing`. Use `createJson()` or `createCbor()` when you already have a custom Git plumbing instance. Both explicit-plumbing factories accept the same facade options as the constructor except `codec`, which is fixed by the factory name. ### Full Constructor For complete control, use the constructor directly: ```js import ContentAddressableStore, { CborCodec, EventEmitterObserver, CdcChunker, NodeCompressionAdapter, } from '@git-stunts/git-cas'; const cas = new ContentAddressableStore({ plumbing, chunkSize: 128 * 1024, // 128 KiB chunks codec: new CborCodec(), observability: new EventEmitterObserver(), merkleThreshold: 500, // Merkle sub-manifests above 500 chunks concurrency: 4, // Parallel chunk I/O maxRestoreBufferSize: 256 * 1024 * 1024, // 256 MiB buffer limit compressionAdapter: new NodeCompressionAdapter(), chunking: { // CDC chunking strategy: 'cdc', targetChunkSize: 64 * 1024, minChunkSize: 16 * 1024, maxChunkSize: 256 * 1024, }, }); ``` --- ## Store Operations ### `store({ source, slug, filename })` Stores data from any async iterable: ```js async function* generateData() { yield new TextEncoder().encode('chunk one'); yield new TextEncoder().encode('chunk two'); } const manifest = await cas.store({ source: generateData(), slug: 'data/stream-example', filename: 'output.bin', }); ``` ### `storeFile({ filePath, slug })` Convenience method that reads a file from disk: ```js const manifest = await cas.storeFile({ filePath: '/absolute/path/to/archive.tar.gz', slug: 'backups/2026-04-23', }); ``` The filename defaults to the basename of `filePath`. Override it with the optional `filename` parameter. Use `chunking` on a single store call when one asset needs a different strategy from the facade default: ```js const manifest = await cas.storeFile({ filePath: './large-archive.tar', slug: 'archives/large', chunking: { strategy: 'cdc' }, }); ``` This override is scoped to that operation. The next `store()` or `storeFile()` call returns to the facade's configured chunker unless it also supplies `chunking`. --- ## Encryption All store and restore methods accept encryption options. Encryption is AES-256-GCM. ### Encryption Schemes | Scheme | Description | Default When | | ------------ | ----------------------------------------------------------------------------------------------- | ------------------------- | | `convergent` | Per-chunk encryption keyed by content hash; preserves CDC deduplication across encrypted stores | CDC chunking + encryption | | `framed` | Streaming framed encryption with per-frame AAD binding to slug | Non-CDC encrypted stores | | `whole` | Whole-object encryption with AAD binding to slug | Explicit only | AAD (Additional Authenticated Data) is always on for `whole` and `framed` schemes, binding ciphertext to the asset slug. **Convergent encryption** is the default when CDC chunking is active and an encryption key is provided. It encrypts each chunk independently using a key derived from the chunk content, so identical plaintext chunks produce identical ciphertext -- preserving deduplication. For non-CDC stores, the default is `framed`. #### Legacy Schemes The v1/v2 scheme identifiers (`whole-v1`, `whole-v2`, `framed-v1`, `framed-v2`, `convergent-v1`) are no longer supported. Attempting to restore a manifest with a legacy scheme throws a `LEGACY_SCHEME` error. Run `scripts/migrate-encryption.js` to upgrade old manifests to current scheme names. See [Migration](#migrating-legacy-encryption-schemes) below. ### Raw Key Encryption ```js import { randomBytes } from 'node:crypto'; const key = randomBytes(32); // 32-byte AES-256 key // Store encrypted const manifest = await cas.storeFile({ filePath: './secret.pdf', slug: 'docs/secret', encryptionKey: key, }); // Restore encrypted const treeOid = await cas.createTree({ manifest }); const readBack = await cas.readManifest({ treeOid }); const { buffer } = await cas.restore({ manifest: readBack, encryptionKey: key, }); ``` ### Passphrase-Based Encryption Derives a key using PBKDF2 (default) or scrypt: PBKDF2 works across Node, Bun, and Deno/Web Crypto. scrypt requires a Node/Bun crypto adapter; Web Crypto runtimes report a capability error for scrypt. ```js // Store with passphrase (PBKDF2) const pbkdf2Manifest = await cas.storeFile({ filePath: './secret.pdf', slug: 'docs/secret', passphrase: 'my-strong-passphrase', }); // Store with scrypt const scryptManifest = await cas.storeFile({ filePath: './secret.pdf', slug: 'docs/secret', passphrase: 'my-strong-passphrase', kdfOptions: { algorithm: 'scrypt' }, }); // Restore with passphrase const readBack = await cas.readManifest({ treeOid }); const { buffer } = await cas.restore({ manifest: readBack, passphrase: 'my-strong-passphrase', }); ``` ### Multi-Recipient (Envelope) Encryption Envelope encryption wraps a random DEK with each recipient's KEK. Adding or removing recipients never re-encrypts the data. ```js import { randomBytes } from 'node:crypto'; const aliceKey = randomBytes(32); const bobKey = randomBytes(32); // Store with recipients const manifest = await cas.storeFile({ filePath: './shared-doc.pdf', slug: 'team/shared-doc', recipients: [ { label: 'alice', key: aliceKey }, { label: 'bob', key: bobKey }, ], }); // Restore using any recipient's key const readBack = await cas.readManifest({ treeOid }); const { buffer } = await cas.restore({ manifest: readBack, encryptionKey: aliceKey, }); // Add a recipient const updated = await cas.addRecipient({ manifest: readBack, existingKey: aliceKey, // Any current recipient's key newRecipientKey: carolKey, label: 'carol', }); // Remove a recipient const shrunk = await cas.removeRecipient({ manifest: updated, label: 'bob', }); // List recipients const labels = await cas.listRecipients(readBack); // => ['alice', 'bob'] // Rotate a recipient's key (no data re-encryption) const rotated = await cas.rotateKey({ manifest: readBack, oldKey: aliceKey, newKey: aliceNewKey, label: 'alice', // Optional: target a specific recipient }); ``` ### Explicit Scheme Selection Override the default scheme: ```js // Force framed encryption (even with CDC chunking) const framedManifest = await cas.storeFile({ filePath: './large-video.mp4', slug: 'media/video', encryptionKey: key, encryption: { scheme: 'framed', // 'whole', 'framed', or 'convergent' frameBytes: 128 * 1024, // 128 KiB frames (framed scheme only) }, }); // Force whole-object encryption const wholeManifest = await cas.storeFile({ filePath: './small-config.json', slug: 'config/app', encryptionKey: key, encryption: { scheme: 'whole' }, }); ``` ### Migrating Legacy Encryption Schemes Manifests created with earlier versions may use v1/v2 scheme identifiers (`whole-v1`, `whole-v2`, `framed-v1`, `framed-v2`, `convergent-v1`). These are no longer supported in the main codebase. The migration script decrypts using the legacy logic and re-stores using current scheme names with AAD always on. ```sh node scripts/migrate-encryption.js printf '%s\n' 'my-secret' | node scripts/migrate-encryption.js --execute --passphrase-file - node scripts/migrate-encryption.js --execute --key-file ./asset.key ``` The migration script handles both fast (rename-only for v2 schemes) and full (re-encryption with AAD for v1 schemes) migration paths. Full migration accepts `--passphrase-file`, `--key-file`, or inline `--passphrase` for compatibility; inline passphrases print a warning because they can leak through shell history and process listings. For privacy-enabled vaults, add `--vault-passphrase-file` or `--vault-key-file` when the vault passphrase differs from the content passphrase. The inline `--vault-passphrase` fallback is still accepted with the same warning. The main `src/` codebase throws `LEGACY_SCHEME` if it encounters a v1/v2 identifier. --- ## Chunking Content is split into chunks before storage. Two strategies are available: ### Fixed Chunking (Default) Splits data into equal-sized chunks. Simple and predictable. ```js const cas = ContentAddressableStore.createJson({ plumbing, chunkSize: 512 * 1024, // 512 KiB chunks }); ``` ### Content-Defined Chunking (CDC) Uses a rolling hash to find chunk boundaries based on content. Provides deduplication across similar files. ```js const cas = new ContentAddressableStore({ plumbing, chunking: { strategy: 'cdc', targetChunkSize: 64 * 1024, // 64 KiB target minChunkSize: 16 * 1024, // 16 KiB minimum maxChunkSize: 256 * 1024, // 256 KiB maximum }, }); ``` CDC parameters: | Parameter | Description | | ----------------- | ------------------------------------------------- | | `targetChunkSize` | Average chunk size the rolling hash targets | | `minChunkSize` | Minimum chunk size (never split smaller) | | `maxChunkSize` | Maximum chunk size (force split at this boundary) | > **Note**: CDC deduplication is ineffective with standard encryption (`whole` or `framed`), since ciphertext is pseudorandom. Use the `convergent` scheme (the default for CDC + encryption) to preserve deduplication. --- ## Compression Enable gzip compression to reduce storage size. Compression runs before encryption (compress-then-encrypt). Plaintext and gzip-compressed content both stream during store and restore, so memory usage stays low regardless of file size. ### Library ```js const manifest = await cas.storeFile({ filePath: './large-log.txt', slug: 'logs/2026-04-23', compression: { algorithm: 'gzip' }, }); // Decompression is automatic on restore -- the manifest records the algorithm. const { buffer } = await cas.restore({ manifest: readBack }); ``` ### Custom Compression Adapter Implement the `CompressionPort` interface to use a different algorithm: ```js import { CompressionPort } from '@git-stunts/git-cas'; class BrotliAdapter extends CompressionPort { async compressBuffer(buffer) { /* ... */ } async decompressBuffer(buffer) { /* ... */ } async *compressStream(source) { /* ... */ } async *decompressStream(source) { /* ... */ } } const cas = new ContentAddressableStore({ plumbing, compressionAdapter: new BrotliAdapter(), }); ``` --- ## Manifest Diffing Compare two manifests to find added, removed, and unchanged chunks. This is a pure function with no I/O -- useful for incremental backup, sync, and deduplication analysis. ### Library ```js // Static method (requires class, not an instance) import CasService from '@git-stunts/git-cas/service'; const serviceDiff = CasService.diffManifests(oldManifest, newManifest); // Standalone function import { diffManifests } from '@git-stunts/git-cas'; const standaloneDiff = diffManifests(oldManifest, newManifest); console.log(standaloneDiff.summary); // => { // addedCount: 3, removedCount: 1, unchangedCount: 42, // addedBytes: 196608, removedBytes: 65536, unchangedBytes: 2752512, // } console.log(standaloneDiff.added); // Chunk[] -- new chunks in newManifest console.log(standaloneDiff.removed); // Chunk[] -- chunks only in oldManifest console.log(standaloneDiff.unchanged); // Chunk[] -- chunks in both (by digest) ``` --- ## Vault Management The vault is a GC-safe index backed by `refs/cas/vault`. It maps slugs to Git tree OIDs, ensuring stored assets are reachable by Git and not garbage-collected. ### Initialize the Vault ```js // Plain vault (no encryption) await cas.initVault(); // Encrypted vault with passphrase await cas.initVault({ passphrase: 'vault-secret', kdfOptions: { algorithm: 'pbkdf2' }, // or 'scrypt' }); // Encrypted vault with privacy mode await cas.initVault({ passphrase: 'vault-secret', kdfOptions: { algorithm: 'pbkdf2' }, privacy: true, }); ``` ### Add, List, Resolve, Remove ```js // Add an entry await cas.addToVault({ slug: 'photos/vacation', treeOid }); // Overwrite an existing entry await cas.addToVault({ slug: 'photos/vacation', treeOid: newTreeOid, force: true }); // List all entries const entries = await cas.listVault(); // => [{ slug: 'photos/vacation', treeOid: 'abc123...' }, ...] // Resolve a single slug to its tree OID const oid = await cas.resolveVaultEntry({ slug: 'photos/vacation' }); // Remove an entry const { commitOid, removedTreeOid } = await cas.removeFromVault({ slug: 'photos/vacation', }); ``` ### Rotate Vault Passphrase Re-wraps every envelope-encrypted entry's DEK with a new key derived from `newPassphrase`. Entries using direct-key encryption are skipped. ```js const { commitOid, rotatedSlugs, skippedSlugs } = await cas.rotateVaultPassphrase({ oldPassphrase: 'old-secret', newPassphrase: 'new-secret', kdfOptions: { algorithm: 'scrypt' }, // optional: change KDF algorithm }); ``` ### Get Vault Metadata ```js const metadata = await cas.getVaultMetadata(); // => { version: 1, encryption: { cipher: 'aes-256-gcm', kdf: { ... }, verifier: { ... } } } ``` Encrypted v6 vaults include a verifier in `.vault.json`, so tools can reject a wrong passphrase even before the vault contains entries. Normal CLI and facade flows verify derived vault keys automatically. Custom tools that derive vault keys themselves should call `verifyVaultKey({ encryptionKey })` before using the key. --- ## Vault Privacy Mode When privacy mode is enabled, vault entry names are stored as HMAC hashes instead of plaintext slugs. This prevents an attacker who has access to the Git object database from learning what assets are stored, even without the encryption key. ### Enabling Privacy Mode Privacy mode requires vault-level encryption. Enable it at vault initialization: ```js await cas.initVault({ passphrase: 'vault-secret', kdfOptions: { algorithm: 'pbkdf2' }, privacy: true, }); ``` ### Limitations - Privacy mode cannot be enabled on an existing vault -- it must be set at init time. - All vault read operations that need to resolve slugs require the encryption key, since the slug-to-HMAC mapping is itself encrypted. - Privacy mode adds overhead: an encrypted `.privacy-index` blob is maintained alongside the vault tree. --- ## Manifest Features ### Manifest Versions and Format Version Manifests are versioned value objects that describe a stored asset: - **v1**: Flat chunk list with SHA-256 integrity hashes. - **v2**: Merkle sub-manifest support for large files. Activated when chunk count exceeds `merkleThreshold` (default 1000). Each manifest also carries an optional `formatVersion` field -- a semver string (e.g. `"6.0.0"`) stamped by the library version that created it. This enables forward-compatible tooling to detect which features a manifest may use. ### Reading a Manifest ```js const manifest = await cas.readManifest({ treeOid }); console.log(manifest.slug); // 'photos/vacation' console.log(manifest.filename); // 'photo.jpg' console.log(manifest.size); // total bytes console.log(manifest.chunks.length); // number of chunks console.log(manifest.encryption); // encryption metadata or undefined console.log(manifest.compression); // compression metadata or undefined console.log(manifest.formatVersion); // '6.0.0' or undefined (older manifests) ``` ### Inspecting an Asset Returns metadata without performing a full restore: ```js const info = await cas.inspectAsset({ treeOid }); // => { slug: 'photos/vacation', chunksOrphaned: 4 } ``` ### Integrity Verification Verifies every chunk's SHA-256 hash against the manifest: ```js const ok = await cas.verifyIntegrity(manifest); if (!ok) { console.error('Integrity check failed -- data may be corrupted'); } ``` --- ## Restore Modes Three restore modes serve different use cases. When `concurrency` is greater than 1, chunk reads are parallelized for faster restore. ### `restore({ manifest })` -- Buffered to Memory Returns the entire file as a single buffer. Best for small-to-medium files. ```js const manifest = await cas.readManifest({ treeOid }); const { buffer, bytesWritten } = await cas.restore({ manifest }); ``` Encrypted content requires a key: ```js const { buffer } = await cas.restore({ manifest, encryptionKey: key, // or: passphrase: 'secret', }); ``` The `maxRestoreBufferSize` option (default 512 MiB) guards against out-of-memory errors. ### `restoreFile({ manifest, outputPath, baseDirectory })` -- Atomic File Write Writes directly to disk. Handles streaming internally for framed-encrypted and compressed content. ```js const manifest = await cas.readManifest({ treeOid }); const { bytesWritten } = await cas.restoreFile({ manifest, outputPath: '/tmp/restored-photo.jpg', baseDirectory: '/tmp', encryptionKey: key, // if encrypted }); ``` ### `restoreStream({ manifest })` -- Async Iterable Returns an async iterable of `Uint8Array` chunks. Best for large files, piping to other streams, or memory-constrained environments. ```js const manifest = await cas.readManifest({ treeOid }); for await (const chunk of cas.restoreStream({ manifest, encryptionKey: key })) { process.stdout.write(chunk); } ``` ### When to Use Each | Mode | Memory | Streaming | Use Case | | --------------- | ---------------- | -------------- | ---------------------------------- | | `restore` | Full file in RAM | No | Small files, in-memory processing | | `restoreFile` | Low | Yes (internal) | Disk targets, atomic writes | | `restoreStream` | Low | Yes (external) | Pipes, HTTP responses, large files | --- ## Streaming Surface | Surface | Streaming API? | Non-streaming API? | Notes | | ------------------------------- | ------------------------------------------ | --------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Write | `store({ source, ... })`, `storeFile(...)` | No dedicated non-streaming store facade | Write ingress is stream-based. CDC + encryption defaults to `convergent` (per-chunk deterministic encryption preserving dedup). Fixed + encryption defaults to `framed`. | | Read: plaintext | `restoreStream(...)`, `restoreFile(...)` | `restore(...)` | True chunk-by-chunk streaming restore. | | Read: encrypted `whole` | `restoreStream(...)`, `restoreFile(...)` | `restore(...)` | `restoreStream()` is the buffered compatibility path. Uncompressed `restoreFile()` verifies chunks, streams tentative plaintext through whole-object AES-GCM decryption into a temp-file path, and renames into place only after auth succeeds. AAD (slug) is always bound. On Web Crypto runtimes this decrypt step is still one-shot internally, bounded by `maxDecryptionBufferSize`. | | Read: encrypted `framed` | `restoreStream(...)`, `restoreFile(...)` | `restore(...)` | True authenticated streaming restore. Plaintext is yielded frame-by-frame after each frame is verified. Per-frame AAD is always bound. | | Read: compressed-only | `restoreStream(...)`, `restoreFile(...)` | `restore(...)` | Plaintext + gzip now streams end-to-end. `restoreFile()` streams gunzip output through a bounded temp-file path. | | Read: compressed + `whole` | `restoreStream(...)`, `restoreFile(...)` | `restore(...)` | Auth must complete before gunzip. `restoreFile()` therefore preserves the auth-before-decompress boundary and may buffer the encrypted compressed payload; use `framed` or `convergent` for large compressed encrypted assets. | | Read: compressed + `framed` | `restoreStream(...)`, `restoreFile(...)` | `restore(...)` | Streaming decrypt, then streaming gunzip. | | Read: encrypted `convergent` | `restoreStream(...)`, `restoreFile(...)` | `restore(...)` | True per-chunk streaming restore. Each chunk is decrypted individually using a key derived from its content hash. Parallel chunk restore via prefetch window when concurrency > 1. | | Read: compressed + `convergent` | `restoreStream(...)`, `restoreFile(...)` | `restore(...)` | Per-chunk convergent decrypt, then streaming gunzip. | | Verify | No streaming verify surface | `verifyIntegrity(manifest, options?)` | Verifies chunk digests for all content. `whole` auth-checks the full ciphertext; `framed` parses and auth-checks every frame; `convergent` decrypts each chunk and verifies plaintext digests. | Runtime note: `framed` is the honest cross-runtime streaming answer. On Node and Bun, `whole restoreFile()` has the stronger low-memory path; on Web Crypto runtimes such as Deno, `whole` remains bounded-buffer rather than true streaming. --- ## CLI Reference All commands support `--json` for machine-readable output and `--quiet` to suppress progress. ### Store and Restore | Command | Description | | ----------------------------------------------- | ------------------------------ | | `git-cas store --slug ` | Store a file into Git CAS | | `git-cas restore --out --slug ` | Restore a file from vault slug | | `git-cas restore --out --oid ` | Restore a file from tree OID | **Store flags:** | Flag | Description | | --------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------ | | `--slug ` | Asset slug identifier (required) | | `--tree` | Create Git tree and add to vault | | `--force` | Overwrite existing vault entry (requires `--tree`) | | `--key-file ` | Path to 32-byte raw encryption key file | | `--recipient ` | Envelope recipient (repeatable) | | `--vault-passphrase ` | Inline vault passphrase (accepted with warning; prefer `--vault-passphrase-file -`, `GIT_CAS_PASSPHRASE`, or `--os-keychain-target`) | | `--vault-passphrase-file ` | Read passphrase from file (`-` for stdin) | | `--os-keychain-target ` | Read passphrase from OS keychain via `@git-stunts/vault` | | `--os-keychain-account ` | Keychain account namespace (default: `git-cas`) | | `--gzip` | Enable gzip compression | | `--strategy ` | Chunking strategy | | `--chunk-size ` | Chunk size in bytes | | `--target-chunk-size ` | CDC target chunk size | | `--min-chunk-size ` | CDC minimum chunk size | | `--max-chunk-size ` | CDC maximum chunk size | | `--concurrency ` | Parallel chunk I/O operations | | `--codec ` | Manifest codec | | `--merkle-threshold ` | Chunk count for Merkle sub-manifests | | `--cwd ` | Git working directory (default: `.`) | **Restore flags:** | Flag | Description | | --------------------------------- | -------------------------------------------------------------------------------------------- | | `--out ` | Non-empty output file path (required) | | `--slug ` | Resolve tree OID from vault slug | | `--oid ` | Direct tree OID | | `--key-file ` | Encryption key file | | `--vault-passphrase ` | Inline vault passphrase (accepted with warning; prefer file/stdin, env, or keychain sources) | | `--vault-passphrase-file ` | Read passphrase from file (`-` for stdin) | | `--os-keychain-target ` | OS keychain passphrase source | | `--os-keychain-account ` | Keychain account namespace | | `--concurrency ` | Parallel chunk I/O | | `--max-restore-buffer ` | Max buffered restore size in bytes | | `--cwd ` | Git working directory | ### Vault Commands | Command | Description | | ----------------------------- | ------------------------------------- | | `git-cas vault init` | Initialize the vault | | `git-cas vault list` | List all vault entries | | `git-cas vault remove ` | Remove a vault entry | | `git-cas vault info ` | Show info for a vault entry | | `git-cas vault stats` | Vault size, dedup, encryption summary | | `git-cas vault history` | Show vault commit history | | `git-cas vault rotate` | Rotate vault passphrase | | `git-cas vault dashboard` | Interactive TUI explorer | **Vault init flags:** | Flag | Description | | -------------------------------- | ------------------------------------------------------------------------ | | `--vault-passphrase ` | Enable encrypted vault from an inline passphrase (accepted with warning) | | `--vault-passphrase-file ` | Read passphrase from file (`-` for stdin) | | `--os-keychain-target ` | OS keychain passphrase source | | `--algorithm ` | KDF algorithm | **Vault rotate flags:** | Flag | Description | | ------------------------------ | ------------------------------------------------------- | | `--old-passphrase ` | Current inline vault passphrase (accepted with warning) | | `--new-passphrase ` | New inline vault passphrase (accepted with warning) | | `--old-passphrase-file ` | Read old passphrase from file (`-` for stdin) | | `--new-passphrase-file ` | Read new passphrase from file (`-` for stdin) | | `--algorithm ` | KDF algorithm for new passphrase | ### Inspection and Verification | Command | Description | | -------------------------------------------- | ----------------------------------------- | | `git-cas inspect --slug ` | Inspect manifest (human-readable or JSON) | | `git-cas inspect --oid --heatmap` | Chunk size heatmap visualization | | `git-cas verify --slug ` | Verify chunk-level SHA-256 integrity | | `git-cas doctor` | Vault health report | | `git-cas tree --manifest ` | Create Git tree from manifest JSON file | ### Key and Recipient Management | Command | Description | | ------------------------------------------------------------------------------------------- | --------------------- | | `git-cas rotate --slug --old-key-file --new-key-file ` | Rotate encryption key | | `git-cas recipient add --label