# SPEC

Reference contract for public behavior guarantees and support coverage.

## Domain model (conceptual)
- Archive: a container of entries (files, directories, links) that can be opened, audited, normalized, and extracted.
- Compression: byte-level transforms for entry payloads or whole archives (e.g., deflate, zstd, xz).
- Audit: validation pass that reports issues without mutating data.
- Normalize: deterministic rewrite that fixes or flags issues according to a safety profile.

## Public API entrypoints

### npm (package.json exports)
- `@ismail-elkorchi/bytefold`
- `@ismail-elkorchi/bytefold/compress`
- `@ismail-elkorchi/bytefold/zip`
- `@ismail-elkorchi/bytefold/tar`
- `@ismail-elkorchi/bytefold/node/zip`

The package root exposes one archive/compression facade. Package export
conditions select the Node, Deno, Bun, or browser transport without changing
the exported names or TypeScript contract. ZIP/TAR classes, format errors,
codec registration, and format-specific types are exposed only by `/zip`,
`/node/zip`, and `/tar`.

### jsr (jsr.json exports)
- `@ismail-elkorchi/bytefold`

## Behavior guarantees

- The package is ESM-only, has no runtime dependencies, and selects its runtime transport from package export conditions.
- Default, compression, ZIP, and TAR entrypoints do not import `node:*` during module evaluation. The Node root loads its adapter only when `openArchive()` is called.
- Enqueued stream chunks remain immutable, and streaming decompression is independent of input chunk boundaries.
- Audit, normalization, decompression, and extraction enforce documented input, output, dictionary, entry-count, and compression-ratio limits without emitting beyond configured ceilings.
- Errors and reports use exported TypeScript shapes and JSON-safe values. Error context cannot replace top-level error fields.
- ZIP and TAR normalization is deterministic. Lossless ZIP normalization preserves compressed bytes where documented.
- Extraction rejects traversal, escaping symlinks, collisions, and pre-existing file-system targets without modifying existing host content.
- Runtime URL readers require HTTPS unless a Node, Deno, or Bun caller explicitly opts into HTTP. The browser implementation is always HTTPS-only.
- Seekable HTTP readers validate range lengths, validators, content encoding, and resource changes while keeping request and byte use bounded.
- ZIP64 parsing and writing preserve 64-bit offsets and fail with typed ZIP errors for malformed structures.
- XZ decoding validates stream checks, filter chains, indexes, concatenation, padding, and resource ceilings.
- Blob inputs remain bounded; ZIP blobs use random access, while other formats use bounded buffering.
- ZIP password support exists for compatibility, not as a confidentiality guarantee. Traditional ZipCrypto is weak.
- Every `ArchiveReader` exposes `close()` for a uniform lifecycle contract. `normalizeToWritable` is present only when the detected format can normalize.
- `createArchiveWriter` accepts only archive/container formats in `ArchiveWriterFormat`, requires an explicit entry source, and constrains façade-owned writer and entry options from the literal format. Options are flat; advanced format controls stay on `/zip` or `/tar`. Raw gzip, Brotli, and Zstandard output uses `createCompressor`.
- Every archive writer has two terminal lifecycle operations: `close()` finalizes output, while `abort(reason?)` releases it without writing archive trailers. After either succeeds, later terminal calls are no-ops and `add()` rejects.

## Safety profiles

- `SafetyProfile` is the shared public union `"compatible" | "strict" | "untrusted"` for archive, ZIP, TAR, and compression APIs.
- A reader's safety profile is selected at construction/open time with `safetyProfile`; audit, open-entry, and extraction calls cannot replace it with a second strictness setting.
- `compatible` favors interoperability, `strict` is the default, and `untrusted` applies tighter default resource limits and treats audit warnings as unsafe.
- Explicit `limits` override only the supplied fields; all other limits continue to come from the selected safety profile.

## Gzip support details
- Fixed header fields, reserved flags, optional-field bounds, ISO-8859-1 FNAME bytes, and header CRC (FHCRC) are handled per RFC 1952 (`https://www.rfc-editor.org/rfc/rfc1952`) before backend decompression.

## Deflate64 support details
- ZIP method 9 decoding uses a 64 KiB window and the Deflate64 distance and length-code semantics described by the checked-in APPNOTE references.
- Dynamic Huffman trees are validated for declared counts, repeat overruns, over-subscription, a required end-of-block symbol, and the literal-only single-zero-distance-code form. Parsing is independent of input chunk boundaries, and bytes after the final compressed block are rejected.

## Bzip2 support details
- Non-randomized bzip2 streams and concatenated members are decoded with block and combined CRC validation. Obsolete randomized blocks are rejected rather than carrying a legacy compatibility decoder.
- Empty, truncated, malformed-Huffman, oversized-run, trailing-data, and resource-limit failures are rejected with typed compression errors.

## XZ support details
- Checks supported: none (0x00), CRC32 (0x01), CRC64 (0x04), SHA-256 (0x0A).
- Filters supported: LZMA2 (0x21), Delta (0x03), BCJ x86 (0x04), PowerPC (0x05), IA64 (0x06), ARM (0x07), ARM-Thumb (0x08), SPARC (0x09), ARM64 (0x0A), RISC-V (0x0B).
- BCJ filter properties are size 0 or 4 bytes with alignment enforced; BCJ filters cannot be last.
- Mixed chains (Delta -> BCJ -> LZMA2) are supported; non-LZMA2 filters apply in reverse order during decode.
- BCJ start offsets are interpreted per stream (reset at concatenation) and applied as provided in filter properties.
- VLIs must use their canonical shortest encoding, reserved LZMA2 dictionary-property bits must be zero, and LZMA range-coded chunks must have valid initialization and termination state.

XZ filter identifiers follow the XZ file format and the upstream liblzma filter
definitions.

## Format and runtime support
Legend: ✅ supported · — capability absent from the typed façade · ⚠ explicit hint required · 🟦 capability-gated (throws `COMPRESSION_UNSUPPORTED_ALGORITHM` when missing).

### Node (>=24)
| Format | Detect | List | Audit | Extract | Normalize | Write |
| --- | --- | --- | --- | --- | --- | --- |
| zip | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
| tar | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
| tgz / tar.gz | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
| gz | ✅ | ✅ | ✅ | ✅ | — | — |
| tar.bz2 | ✅ | ✅ | ✅ | ✅ | ✅ | — |
| bz2 | ✅ | ✅ | ✅ | ✅ | — | — |
| tar.xz | ✅ | ✅ | ✅ | ✅ | ✅ | — |
| xz | ✅ | ✅ | ✅ | ✅ | — | — |
| tar.zst | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
| zst | ✅ | ✅ | ✅ | ✅ | — | — |
| tar.br | ⚠ (format: `tar.br` or filename) | ✅ | ✅ | ✅ | ✅ | ✅ |
| br | ⚠ (format: `br` or filename) | ✅ | ✅ | ✅ | — | — |

### Deno
| Format | Detect | List | Audit | Extract | Normalize | Write |
| --- | --- | --- | --- | --- | --- | --- |
| zip | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
| tar | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
| tgz / tar.gz | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
| gz | ✅ | ✅ | ✅ | ✅ | — | — |
| tar.bz2 | ✅ | ✅ | ✅ | ✅ | ✅ | — |
| bz2 | ✅ | ✅ | ✅ | ✅ | — | — |
| tar.xz | ✅ | ✅ | ✅ | ✅ | ✅ | — |
| xz | ✅ | ✅ | ✅ | ✅ | — | — |
| tar.zst | 🟦 | 🟦 | 🟦 | 🟦 | 🟦 | 🟦 |
| zst | 🟦 | 🟦 | 🟦 | 🟦 | — | — |
| tar.br | ⚠ (format: `tar.br` or filename; 🟦 when Brotli streams are unavailable) | 🟦 | 🟦 | 🟦 | 🟦 | 🟦 |
| br | ⚠ (format: `br` or filename; 🟦 when Brotli streams are unavailable) | 🟦 | 🟦 | 🟦 | — | — |

### Bun
| Format | Detect | List | Audit | Extract | Normalize | Write |
| --- | --- | --- | --- | --- | --- | --- |
| zip | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
| tar | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
| tgz / tar.gz | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
| gz | ✅ | ✅ | ✅ | ✅ | — | — |
| tar.bz2 | ✅ | ✅ | ✅ | ✅ | ✅ | — |
| bz2 | ✅ | ✅ | ✅ | ✅ | — | — |
| tar.xz | ✅ | ✅ | ✅ | ✅ | ✅ | — |
| xz | ✅ | ✅ | ✅ | ✅ | — | — |
| tar.zst | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
| zst | ✅ | ✅ | ✅ | ✅ | — | — |
| tar.br | ⚠ (format: `tar.br` or filename) | ✅ | ✅ | ✅ | ✅ | ✅ |
| br | ⚠ (format: `br` or filename) | ✅ | ✅ | ✅ | — | — |

### Web (Browser)
| Format | Detect | List | Audit | Extract | Normalize | Write |
| --- | --- | --- | --- | --- | --- | --- |
| zip | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
| tar | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
| tgz / tar.gz | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
| gz | ✅ | ✅ | ✅ | ✅ | — | — |
| tar.bz2 | ✅ | ✅ | ✅ | ✅ | ✅ | — |
| bz2 | ✅ | ✅ | ✅ | ✅ | — | — |
| tar.xz | ✅ | ✅ | ✅ | ✅ | ✅ | — |
| xz | ✅ | ✅ | ✅ | ✅ | — | — |
| tar.zst | 🟦 | 🟦 | 🟦 | 🟦 | 🟦 | 🟦 |
| zst | 🟦 | 🟦 | 🟦 | 🟦 | — | — |
| tar.br | ⚠ (format: `tar.br` or filename; 🟦 when Brotli streams are unavailable) | 🟦 | 🟦 | 🟦 | 🟦 | 🟦 |
| br | ⚠ (format: `br` or filename; 🟦 when Brotli streams are unavailable) | 🟦 | 🟦 | 🟦 | — | — |

## Web runtime notes
- Entry point: `@ismail-elkorchi/bytefold`; the `browser` export condition selects the implementation.
- Supported browser input kinds: `Uint8Array`, `ArrayBuffer`, `ReadableStream<Uint8Array>`, `Blob`/`File`, and HTTPS URL.
- URL behavior in browsers: only HTTPS URLs are accepted; responses are always full-fetched before archive detection/opening, and no seekable HTTP range session is attempted by design. `maxInputBytes` is enforced both from `Content-Length` and during streaming reads, with over-limit chunked/slow responses canceled before full transfer.
- ZIP on Blob uses seekable random access (`blob.slice(start, end).arrayBuffer()`), so listing/extracting ZIP from Blob stays bounded by seek budget and avoids full Blob buffering.
- Web write roundtrip contract: ZIP (store-only mode) and TAR can be created through the package root into pure Web `WritableStream` sinks, wrapped in Blob, and reopened with matching entry names/bytes plus safe audit/normalize behavior.
- XZ seekable preflight over Blob is not implemented in this iteration; Blob XZ paths use bounded full-buffer input handling and existing decode-time resource ceilings.
- Compression capability reporting for web runtime:
  - `getCompressionCapabilities()` probes `CompressionStream` and `DecompressionStream` constructor acceptance independently for algorithm strings `gzip`, `deflate`, `deflate-raw`, `brotli`, and `zstd`;
  - each algorithm reports `canCompress` and `canDecompress` truthfully per constructor acceptance; unsupported modes surface `COMPRESSION_UNSUPPORTED_ALGORITHM` when requested;
  - pure-JS decode support remains for `bzip2` and `xz`;
  - when either web compression constructor is missing, `limitations` includes an explicit missing-constructor message.
- Runtime detection for capabilities uses `runtime: "web"` when Bun/Deno/Node markers are absent and either web compression global exists (`CompressionStream` or `DecompressionStream`).

## Single-file compressed formats: entry naming
Naming is deterministic and sanitized to a single path segment.
- Inputs with a filename hint (file path or URL): use the final path segment, strip the compression extension, and return `name` or `name.tar` for `.tar.*` variants.
- Inputs without a filename hint (bytes/streams): default to `data`, except gzip may use a header FNAME if present.
- Gzip header options: FEXTRA and FCOMMENT fields are skipped per RFC1952 and do not block FNAME parsing.
- Gzip FNAME: used only after sanitization (basename only; NUL, empty, `.`/`..` rejected). If rejected or missing, fall back to filename or `data`.

## Normalization determinism
- Deterministic normalization (`isDeterministic: true`) emits stable bytes for zip and tar; normalizing an already normalized archive yields byte-identical output.
- For tar-wrapped compressed formats, normalization produces a deterministic tar stream and preserves entry names + contents.

## Ambiguity policy (normalize safe mode)
- Collision key pipeline: path normalization (slashes + dot segments) → `normalize('NFC')` → full Unicode case folding (from `specs/unicode/CaseFolding-17.0.0.txt`, statuses C+F, T excluded) → `normalize('NFC')`.
- Duplicate paths: error (tar → `ARCHIVE_NAME_COLLISION` + `TAR_DUPLICATE_ENTRY`, zip → `ZIP_NAME_COLLISION` + `ZIP_DUPLICATE_ENTRY`).
- Case-fold collisions: error (tar → `ARCHIVE_NAME_COLLISION` + `TAR_CASE_COLLISION`, zip → `ZIP_NAME_COLLISION` + `ZIP_CASE_COLLISION`).
- Unicode normalization collisions (NFC): error (tar → `ARCHIVE_NAME_COLLISION` + `TAR_UNICODE_COLLISION`, zip → `ZIP_NAME_COLLISION` + `ZIP_UNICODE_COLLISION`).
- Directory/file kind conflict (`dir` vs `dir/`): allowed; entries remain distinct after normalization.
- Symlinks: rejected in normalize safe mode (tar → `ARCHIVE_UNSUPPORTED_FEATURE` + `TAR_SYMLINK_PRESENT`, zip → `ZIP_SYMLINK_DISALLOWED`).
- Hardlinks (tar `link` entries): rejected in normalize safe mode (`ARCHIVE_UNSUPPORTED_FEATURE` + `TAR_UNSUPPORTED_ENTRY`).
- Path normalization: backslashes are normalized to `/`, `.` segments and repeated slashes are removed; absolute paths, drive-letter prefixes, and `..` segments are rejected with path traversal errors.

## Security model (name collisions)
- Unicode normalization collisions are rejected in audit/normalize/extract because filesystem normalization differs across platforms, and accepting them can cause nondeterministic overwrites.

## Concatenation semantics (gzip, bzip2, zstd)
- Gzip concatenated members decode sequentially across Node, Deno, and Bun.
- Bzip2 concatenated streams decode sequentially across runtimes.
- Zstandard standard frames decode sequentially and skippable frames are ignored, including when frame boundaries split across input chunks. Archive detection accepts streams beginning with skippable metadata.

## Resource ceilings
- Defaults: `maxXzDictionaryBytes = 64 MiB` (`untrusted`: 32 MiB), `maxXzBufferedBytes = 1 MiB`, `maxXzIndexRecords = 1,000,000` (`untrusted`: 200,000), `maxXzIndexBytes = 64 MiB` (`untrusted`: 16 MiB), `maxXzPreflightBlockHeaders = 1024` (`untrusted`: 256), `maxZipCentralDirectoryBytes = 64 MiB` (`untrusted`: 16 MiB), `maxZipCommentBytes = 65,535` (`untrusted`: 16,384), `maxZipEocdSearchBytes = 65,558`, `maxBzip2BlockSize = 9`.
- `maxXzPreflightBlockHeaders` bounds seekable dictionary preflight; `0` disables block-header scanning and yields `COMPRESSION_RESOURCE_PREFLIGHT_INCOMPLETE` (info).
- Rationale: XZ Index fields are encoded as VLIs up to 63 bits (`specs/xz-file-format.txt`), so record counts and index sizes can be arbitrarily large; the defaults cap scan time/space while allowing typical archives.
- XZ Index VLI decoding is streaming-safe even when VLI bytes split across chunks.
- Overrides: ceilings are configurable via `limits` in `openArchive(...)`, `ArchiveReader.audit(...)`, and `ArchiveReader.normalizeToWritable(...)`, and via `limits` in `createDecompressor(...)`.
- Precedence rules (safetyProfile vs limits):
  - `openArchive(...)`: `safetyProfile` chooses reader defaults for ZIP/TAR readers, and gzip/tgz layered opens resolve safetyProfile-derived decompression ceilings before inflate; `limits` overrides only the specified fields and unspecified ceilings stay on safetyProfile defaults.
  - `ZipReader` / `TarReader` construction: same rule as `openArchive` because the constructors resolve `safetyProfile` defaults first, then merge explicit `limits` field-by-field.
  - `createDecompressor(...)`: explicit scalar knobs (`maxOutputBytes`, `maxCompressionRatio`, `maxDictionaryBytes`, `maxBufferedInputBytes`) take precedence over their `limits` counterparts; remaining values come from `limits`; `safetyProfile` still controls behavior independently (for example, unsupported XZ checks under `strict` versus `compatible`).
  - `createArchiveWriter(...)`: no `safetyProfile`/`limits` API exists for writer creation; precedence is not applicable.
- Audit preflight: bzip2 block size and xz dictionary size are checked from headers and reported as `COMPRESSION_RESOURCE_LIMIT` without full decompression.
- `maxTotalDecompressedBytes` enforces output ceilings for gzip/deflate/brotli/zstd with `COMPRESSION_RESOURCE_LIMIT` and no output beyond the limit.
- All decompression ceilings, including XZ buffered-input/index/output/ratio limits and bzip2 output/ratio limits, fail with `COMPRESSION_RESOURCE_LIMIT`. XZ index record and byte limits are cumulative across concatenated streams and enforced during direct decompressor use, independently of archive preflight. Limit values must be non-negative safe integers (or `bigint` for byte limits), and ratios must be positive and finite.

## Concatenation and resource ceilings
- XZ: preflight scans concatenated streams using stream headers + Index records (no payload decompression), applies Index ceilings across the concatenation, and scans Block Headers to enforce dictionary limits up to `maxXzPreflightBlockHeaders` per stream. If block count exceeds the limit, audit emits `COMPRESSION_RESOURCE_PREFLIGHT_INCOMPLETE` (info) with `requiredBlockHeaders` + `limitBlockHeaders`, and dictionary limits remain enforced during decode.
- XZ preflight scanning is O(1) memory with respect to Index record count (no per-record arrays).
- Bzip2: preflight only inspects the first member; audit emits `COMPRESSION_RESOURCE_PREFLIGHT_INCOMPLETE` when bzip2 limits are in effect, and concatenated members are enforced during decode.
- Seekable XZ preflight: for file paths or HTTP Range URLs, index + dictionary limits run before full buffering; HTTP failures map to `ARCHIVE_HTTP_*` codes with preserved `context.httpCode`.
- Seekable ZIP preflight: for file paths or HTTP Range URLs, EOCD/central-directory limits run before full buffering; Range is required for HTTP preflight; multi-disk archives are rejected.
- HTTP Range random access: validators (ETag/Last-Modified) are pinned, `If-Range` is used only with strong ETags, header-only failures abort before body consumption, content codings are rejected, and 206 body length must exactly match the requested range.

## Error model
- Stable error classes: `ZipError`, `CompressionError`, `ArchiveError`.
- Error JSON includes one canonical human-readable `message` and a `context` object; it serializes as a plain JSON-safe object.
- Error JSON context policy: `context` MUST NOT duplicate any top-level key name. Keys that are top-level in an error payload (`name`, `code`, `message`, `context`, plus optional top-level fields like `entryName`, `method`, `offset`, `algorithm`) are stripped from `context` during serialization so machine consumers have one canonical location per fact.

## Report model
- Exported TypeScript report types are the sole structural contract; reports do not carry a separately versioned schema marker.
- Reports are plain JSON-safe objects. `ArchiveDetectionReport`, `ArchiveAuditReport`, `ArchiveNormalizeReport`, and `CompressionCapabilities` contain no `bigint`.
- Audit reports expose `isSafe`; normalization reports expose `isSuccessful`; summary counts use explicit singular names ending in `Count`.
- Compression capability entries expose `canCompress` and `canDecompress`, and runtime caveats are collected in `limitations`.
- Issue codes are exported literal unions, while `details` is recursively JSON-safe.