# Project Reference: sequelize-paper-trail

## Overview
Sequelize Paper Trail tracks model changes for auditing and versioning. It records revisions on create/update/destroy and can optionally track users via CLS.

## Current Runtime and Tooling
- Runtime: Node.js (engine in package.json is `>=20`), CommonJS output in `dist/`.
- Build: Babel (`npm run build` transpiles `lib/` to `dist/`).
- Tests: Jest (`npm test`) using sqlite in-memory.
- Linting: ESLint (Airbnb + Prettier + Flowtype rules).
- Package manager: npm (canonical lockfile is `package-lock.json`).

## Repository Structure
- `lib/`: source code (`index.js`, `helpers.js`).
- `dist/`: build output (generated by Babel).
- `test/`: Jest tests (`*.spec.js`) with fixtures in `test/models/` and `test/migrations/`.
- `config/config.json`: test DB config (sqlite in-memory).
- Root configs: `.eslintrc`, `.prettierrc`, `.editorconfig`, `.node-version`.

## Key Scripts
- `npm run build`: transpile `lib/` to `dist/`.
- `npm test`: run Jest tests.
- `npm run lint`: lint `lib/` and `test/`.
- `npm run release`: publish via `release-it` (maintainers).

## Public API (High-Level)
- `init(sequelize, options)`: configure and returns object with `defineModels()`.
- `defineModels()`: creates revision models/tables.
- `Model.hasPaperTrail()`: attach hooks and revision association.
- Options include: `revisionAttribute`, `revisionModel`, `revisionChangeModel`, `enableRevisionChangeModel`,
  `enableCompression`, `enableMigration`, `enableStrictDiff`, `underscored`, `underscoredAttributes`,
  `userModel`, `continuationNamespace`, `continuationKey`, `metaDataFields`.

## Behavior Inventory (v5 Baseline)
- Hook flow: `beforeCreate/Update/Destroy` computes diffs and increments revision only when there is a delta or destroy.
- Revision increments: revision attribute is set to previous value first (manual edits are disallowed), then incremented when a revision is created.
- No-op updates (no diff) do not increment revision and do not create a revision record.
- Exclude list: excluded fields are omitted from snapshots and diff calculations.
- Nested objects: non-Date object fields are omitted from snapshots and diffs.
- Compression: when enabled, only `opt.defaultFields` are tracked for diffs/snapshots.
- Revision payload: stored as JSONB by default; MySQL mode uses MEDIUMTEXT with JSON string.
- RevisionChange: if enabled, per-field diffs are stored and diffed with `diffChars` on stringified values.
- Diff paths: only the top-level `difference.path[0]` is stored for RevisionChange.
- Meta data fields: required metaDataFields are enforced; missing required values throw errors.
- User attribution: uses CLS namespace and continuationKey when configured, otherwise uses `opt.userId`.
- Fail-hard mode: throws if revisionAttribute missing on update or if CLS userId missing when CLS is used.
- enableMigration: best-effort `addColumn` if revisionAttribute is missing; errors are logged and ignored.
- noPaperTrail: passing `{ noPaperTrail: true }` in update options does not prevent revision creation in current behavior.
- Exclude list: if exclusions remove all deltas, no revisions are created (including create).
- CLS propagation: CLS user attribution requires explicit namespace.run/bind; without it userId may be null.
- Meta data fields: missing metaData object can throw a TypeError before the intended error message.
- Object-only updates: updating only object/JSON fields still creates a revision, but the revision attribute does not increment.
- Bulk operations: revisions are created only when `individualHooks: true` is passed.
- Paranoid restore: restore operations do not create revisions (no restore hooks wired).
- Strict diff: disabling strict diff still records revisions for string/number changes in practice.
- Meta data fields are not persisted unless the Revision model defines those attributes explicitly.

## Known Deviations / Quirks (Baseline Behaviors to Preserve Unless Intentional)
- `noPaperTrail: true` on update options does not suppress revision creation.
- CLS user attribution depends on proper namespace.run/bind; missing binding yields null userId.
- `metaDataFields` enforcement can throw a TypeError when the metaData object is missing (before the intended error).
- `metaDataFields` values are not persisted unless the Revision schema defines those columns.
- Exclude lists can remove all deltas and suppress revisions even on create.
- Object/JSON or array-only updates create a revision without incrementing the revision attribute.
- `enableStrictDiff: false` still produces revisions for string/number changes.
- `enableCompression: true` limits document payloads to default fields rather than full snapshots.
- Paranoid restore operations do not create revisions.
- RevisionChange foreign key casing can create duplicate columns on sqlite without explicit defaultAttributes.

## Schema Inventory (v5 Baseline)
- Revision model fields: `model`, `document`, `operation`, `documentId`, `revision`, timestamps.
- RevisionChange fields (optional): `path`, `document`, `diff`, `revisionId`, timestamps.
- UUID option changes `id` and `documentId` types to UUID.
- Underscored/underscoredAttributes affect timestamp and foreign key naming.
- User foreign key is added via `Revision.belongsTo(userModel)` when configured.

## Associations (v5 Baseline)
- `Model.hasPaperTrail()` adds a `hasMany` association to revisions scoped by `model` name.
- Revision belongsTo user model if `userModel` is configured.
- Revision hasMany RevisionChange; RevisionChange belongsTo Revision.

## Data Model Summary
Creates tables for:
- Revisions: stores model snapshot or diff, operation type, documentId, revision number.
- RevisionChange (optional): stores per-attribute diff data.

## Known Limitations
- No support for composite primary keys (documented in README).

## Current Dependencies (Highlights)
- Runtime: `continuation-local-storage`, `deep-diff`, `diff`, `lodash`.
- Dev: `sequelize@^5`, `sequelize-v6` (alias pinned for tests), `sqlite3@^5`, `jest`, `eslint`, `prettier`, Babel toolchain.
- Optional: `cls-hooked` for Sequelize v6 CLS support (lazy required).
- Peer: `sequelize@^5 || ^6` (declared compatibility range).

## Environment Notes
- `.node-version` pins Node 20.20.0 (recommended baseline). Align your local environment for reproducible builds.
- For sqlite3 native builds on newer systems, setting `PYTHON` to a 3.9.x interpreter with `distutils` may be needed.
- Tooling major upgrades are deferred until after the v3/v4 releases; see `docs/archive/prd_phase6.md` for the historical notes.

## Testing Notes
- Tests run against sqlite in-memory tables and use migrations under `test/migrations/`.
- README notes limited coverage; regression tests should be expanded before refactors.
- Sequelize v6 tests run via `npm run test:v6` using the pinned `sequelize-v6` alias.

## Test Strategy (Target)
- Focus on user-facing behaviors rather than internal code paths.
- Cover all features and configuration combinations:
  - create/update/destroy lifecycle, bulk operations, paranoid behavior.
  - revision snapshots vs diffs, strict diff behavior.
  - user tracking with CLS/ALS and missing user IDs.
  - revisionChange model on/off and diff accuracy.
  - underscored/UUID/tableName/revisionAttribute options.
  - migration option behavior.
  - edge cases: JSON/JSONB, nulls, arrays, default values, soft deletes.
- Use fixtures and golden outputs to lock behavior.
- Run tests across a Sequelize version matrix before declaring support.
- Snapshot stability rules are defined in `TESTS.md` (volatile fields stripped, UUIDs normalized, keys sorted).

## Test Coverage Notes (Current)
- Coverage is scoped to sqlite in-memory; no multi-dialect matrix yet.
- No performance/stress tests for large audit tables or high write rates.
- No CLS/ALS adapter tests beyond the v5 baseline behavior.

## v6 Adapter Verification Notes
- Hooks: v6 still supports before/after create/update/destroy with `(instance, options)`.
- Metadata: `Model.getAttributes()` exists; `rawAttributes` still present on defined models.
- Table name: `getTableName()` returns a string for sqlite; schema-aware dialects may return an object.
- `refreshAttributes` remains available on models in v6.

## Current Issues/Risks
- Tooling and dependencies are outdated (Jest 24, ESLint 6, sqlite3 4).
- CLS dependency is unmaintained; may need AsyncLocalStorage adapter.
- Lint currently fails due to a typo in an eslint directive in `test/migrations/20190206193643-create-user.js`.
- RevisionChange default foreign key naming (`revisionId` vs `RevisionId`) can cause duplicate column issues on sqlite.
- User attribution requires aligning `userModelAttribute` with the actual foreign key name.

## Backward Compatibility Considerations
- Preserve hook semantics and revision schema.
- Preserve option names and defaults.
- Preserve behavior of user tracking (CLS) and compression/strict diff logic.

## Compatibility Contract (Target)
- Hook order and timing relative to model writes.
- Revision schema fields, names, and default values.
- Deterministic revision ordering and increment behavior.
- Diff format and compression semantics (round-trip correctness).
- Behavior when `noPaperTrail` or exclusions are used.

## Transactional Integrity (Target)
- Revisions should be created atomically with the original write, or the behavior must be clearly documented.
- Rollbacks should not leave orphan revisions.
- Concurrent writes should not corrupt revision ordering.

## Data Retention and Indexing (Guidance)
- Recommend indexes on revision table fields used in queries (documentId, model, revision).
- Provide retention strategies (TTL, archival, pruning) for large datasets.
- Document performance considerations for large revision payloads.

## Security and PII (Guidance)
- Document risks of storing sensitive fields in audit logs.
- Provide patterns for allowlists/redaction/exclusion lists.
- Clarify implications for compliance (GDPR/CCPA) and data deletion requests.

## Support and Release Governance (Target)
- Define a Node LTS and Sequelize support policy.
- Define deprecation windows and semantic versioning rules for behavior changes.
- Require a compatibility review and migration notes for breaking changes.
- Active PRDs live in `docs/prd/` and are committed; historical phase PRDs are archived under `docs/archive/`.

## Sequelize v5 → v6+ Diff Review (Required Before Support Claim)
Track and explicitly verify:
- Hook lifecycle changes (before/after hooks, bulk hooks, individual hooks).
- Model metadata APIs used (tableName, attributes, primaryKey handling).
- Transaction and CLS/ALS integration differences.
- Migration and model sync behaviors.
- Data type changes or default behavior shifts.

## Sequelize v5 → v6+ Diff Scan (Phase 0 Findings)
- CLS: v6 requires `cls-hooked` + `Sequelize.useCLS`; continuation-local-storage is no longer recommended.
- Bluebird removal: Sequelize now uses native Promises; `Sequelize.Promise` is removed.
- `sequelize.import` is removed (CLI users must update).
- QueryInterface property names on models/dialects are now lowerCamelCase.
- `options.returning: true` returns only model-defined attributes (use `['*']` for previous behavior).
- `Model.changed()` does not detect deep mutations; JSON changes require reassignment or manual change flag.
- `bulkCreate` throws `Sequelize.AggregateError` (not Bluebird AggregateError).
- `upsert` return signature changed (instance + created flag).

## v6 Impact Notes for This Library (Initial)
- User attribution: CLS behavior must be reworked via adapter (cls-hooked vs ALS).
- Diff accuracy for JSON fields depends on Sequelize change tracking; deep mutations may not be detected.
- No direct dependency on `sequelize.import`, `Sequelize.Promise`, or `upsert`, but tests should ensure no implicit reliance.