# Differential testing in separated layers

Load when: before building the first test, and whenever a residual appears. The bar any reconstructed behavior meets before it is `reconstruction-verified`. It binds any lane that reconstructs behavior (scoring formula, ranking function, pricing calculation, matcher, state machine), whether or not a full reconstruction run is entered. A word list is checked by reading it; an algorithm is `partial` until all four hold.

## The bar

1. It is **executable** and committed to the corpus, not described in prose.
2. It is differentially tested against the live product on a real corpus plus deliberately adversarial inputs, not only the examples it was derived from.
3. The test separates the estimator from the feature extraction, and per-layer pass rates are recorded separately.
4. Inputs and the product's observed outputs are frozen as re-runnable fixtures (in `raw/` or beside the implementation).

## Layers

A reconstruction has two independent places to be wrong, the computation and the inputs fed to it; tested together they are indistinguishable and the aggregate points at whichever you already suspect.

| Layer | Feed | Compare | Tests |
| --- | --- | --- | --- |
| A, estimator | the product's own reported feature values | reconstruction output against the product's own output | the mathematics alone |
| B, feature extraction | raw input | your computed features against the product's reported features | tokenisation, parsing, counting alone |
| C, end to end | raw input | final output against the product's output | both together |

C failing while A passes localises the fault to B in one step, with no re-derivation of a formula that was already right.

## Residuals

- A residual that survives a correct layer B is a finding about the target, often the most valuable output of the lane. Chase it.
- Adversarial inputs expose such defects; static lanes cannot.
- Bug-parity (reproducing a target defect to match numbers) sits behind an explicit, documented flag; the default is the correct behavior, so the next reader is not taught the bug.

## Fixtures

`scripts/layers.mjs <fixtures.json> <impl.mjs>` runs the fixtures. Fixture shape:

| Field | Meaning |
| --- | --- |
| `input` | raw input sent to the product |
| `product_features` | features the product reported for it |
| `product_output` | the product's own output |

`impl.mjs` exports `extract(input)` (layer B) and `estimate(features)` (layer A). Layer C is `estimate(extract(input))`. Comparison is exact, or within `tolerance` (default 0) for numbers.