# Checklist Stats — hit-rate sidecar schema and write protocol

`.claude/cabinet/checklist-stats.json` records how the change-impact
checklist (`qa-dimensions.yaml`) performs over time: which dimensions
fire, which checks actually catch problems, and what pruning verdicts
the operator has already given. It is the evidence base for the audit
skill's `checklist-pruning` phase — without it, the checklist only ever
grows (debrief's `checklist-feedback` is add-only by design) and decays
into noise.

**This file is RUNTIME STATE, generated on first write — never shipped
as a template.** Shipping it would clobber accumulated stats on every
reinstall (same rule as `advisories-state-schema.md`). And it never
lives inside `qa-dimensions.yaml`: config files do not contain runtime
state.

## Who writes what

| Writer | When | What |
|--------|------|------|
| `/execute` `post-impl-checklist` phase | every run past its no-op guard | increments `runs`; per triggered dimension increments `fires`, sets `last_fired` |
| Ring 3 close `checklist-catch` lens (Phase 2p) | session close, when the transcript shows a surfaced check that caught a real bug | appends to that dimension's `catches` |
| `/debrief` `checklist-feedback` phase (pre-watchtower; retiring) | when a session bug WAS caught via a surfaced check | appends to that dimension's `catches` |
| `/audit` `checklist-pruning` phase | every pruning verdict (including "keep") | appends to `pruning_reviews` |

The **catch side is recorded automatically at session close** by the
Ring 3 `checklist-catch` lens (`watchtower-ring3-close.mjs`, Phase 2p)
for watchtower-installed projects — it reads the transcript, attributes
catches to a named dimension, and writes them via the protocol below.
`/debrief`'s `checklist-feedback` phase did this interactively and is
retired by the watchtower burn-in (its sharpening side — proposing new
checks for bugs that *slipped through* — needs operator approval and
cannot move to a background ring; only the catch tally moved). The two
writers are mutually exclusive in practice: a watchtower project closes
through Ring 3, a non-watchtower project through debrief, and both
follow the same atomic, fail-open write protocol — so the catches a
later audit prunes against are recorded the same way regardless.

## Schema (`schema_version: 1`)

```json
{
  "schema_version": 1,
  "runs": 14,
  "dimensions": {
    "data-coherence": {
      "fires": 12,
      "last_fired": "2026-06-11",
      "catches": [
        {
          "date": "2026-06-10",
          "check": "[run] Run schema validation if any schema or migration file changed.",
          "note": "caught missing FK backfill before commit"
        }
      ]
    }
  },
  "pruning_reviews": [
    {
      "date": "2026-06-11",
      "target": "test-staleness",
      "verdict": "keep",
      "note": "fires often, zero catches, but cheap insurance at moderate severity"
    }
  ]
}
```

Field semantics:

- **`runs`** — total executions of the post-impl-checklist phase that
  passed its no-op guard, INCLUDING runs where zero dimensions
  triggered. This is the denominator for "never fired in N runs."
- **`dimensions.<name>.fires`** — number of runs in which the dimension
  triggered (matched at least one changed path). Dimension-level, not
  check-level: checks have no stable IDs, so firing is counted where it
  happens (path match) and catching is attributed by quoting the check.
- **`dimensions.<name>.catches`** — append-only evidence that a
  surfaced check caught a real issue. `check` quotes the check text as
  written in the yaml at the time.
- **`pruning_reviews`** — append-only verdict log. `verdict` is one of
  `removed | trimmed | paths-fixed | severity-changed | keep`. The
  pruning phase skips candidates with any verdict in the last 90 days,
  so a "keep" decision is not re-litigated at every audit.

## Write protocol

1. Read the file. If absent, bootstrap the skeleton
   (`{"schema_version": 1, "runs": 0, "dimensions": {}, "pruning_reviews": []}`).
   If present but unparseable, move it aside to
   `checklist-stats.json.corrupt-<YYYY-MM-DD>` (never delete) and
   bootstrap fresh.
2. Modify in memory.
3. Write to `checklist-stats.json.tmp`, then rename over the original
   (atomic — safe under concurrent sessions).

**Fail-open, always:** a stats read or write failure must never block
the phase doing the recording. Emit one warning line and continue —
losing a data point is fine; blocking an execute/debrief/audit run over
bookkeeping is not.

## Anti-trap rules

- **Stats inform; the human decides.** Nothing auto-prunes from this
  data, ever. Low hit-rate is evidence presented at audit, not a
  trigger.
- **Per-dimension judgment, not universal thresholds.** A high-severity
  security check that fires often and never catches may still be cheap
  insurance; an info-severity check with the same profile is noise.
  The pruning phase presents severity alongside the numbers.
- **Renames orphan stats.** If a dimension is renamed in
  `qa-dimensions.yaml`, its stats entry goes stale. The pruning phase
  reports entries with no matching dimension as orphans (offer to fold
  or drop them); writers simply start a fresh entry under the new name.
- **Counts are honest, not precise.** Concurrent sessions can lose an
  increment to a race; the rename-based write keeps the file valid and
  the trend signal is what matters. Do not build exact-count logic on
  top of this file.