# Layer 7 — Phase 1 calibration protocol

## Purpose and pre-registration

Objective Guard is built first against synthetic deviations, not against usage impressions. This document fixes the corpus, metrics, gates, and decisions before running the calibration. A later result that changes these rules is recorded as exploratory.

The scope is Phase 1: local universal detectors. It does not yet prove that the product reduces an agent's semantic drift; it proves that its detectors are reproducible instruments, sufficiently low-noise to observe real usage.

## Frozen corpus

The acceptance corpus contains 100 independent scenarios:

| Group | Count | Rule | Expected outcome |
| --- | ---: | --- | --- |
| Violations | 13 per rule; 52 total | The four rules | `block` for error rules; `warn` for `new-dep`; finding for the rule |
| Controls | 12 per rule; 48 total | The four rules | `pass`, zero findings |

The controls include text in comments, strings, fixtures, and tests; pre-existing changes; reorderings; already-declared dependencies; files with extensions out of scope; and content that resembles a violation but is not one.

The scenarios are `before`/`after` pairs, not mutations applied on top of the development repository. This makes them deterministic, portable, and free of contamination to `.oguard/` state.

## Metrics and synthetic gates

For each scenario, the expected rule, verdict, findings, duration, and detector version are recorded. The metrics are computed as:

\[
recall=\frac{TP}{TP+FN},\qquad FPR=\frac{FP}{FP+TN}
\]

The calibration gate requires, over the frozen corpus:

1. Zero FN and zero FP per rule, not only in aggregate.
2. Three clean runs, in random order, with the same result and no delta-only state leakage.
3. p95 warm of each detector under 200 ms and p95 cold of the CLI under 400 ms across three independent runs of the defined benchmark; zero timeouts or internal errors.
4. Each finding retains rule, file, location, and reproduction.

This corpus of 100 is a build-time smoke benchmark, not a strong statistical estimate: the 13 positives and 12 controls per rule only allow detecting obvious regressions. A holdout of 60 violations and 60 controls per rule (480 evaluations) will only be necessary if the intent is to claim a bound of approximately 5% for FN or FP of an individual rule. It does not block the start of the statistical layer: continuing to accumulate patterns does not prove the product thesis.

If a positive mutation or a control fails, the rule is fixed or degraded and the entire corpus is re-run. The scenario that failed is not removed.

`npm run calibration:gate` combines those conditions across three independent passes and preserves `synthetic/results/f1-calibration-gate-v1.json` even when the verdict is `fail`. The artifact does not change thresholds nor turn a slow run into a warm one. Episode E-023 records the first versioned run: corpus 100/100, but the `syntax-js` budget and CLI cold-start were not approved; Phase 1 remains unpromoted until a reproducible improvement under the same gate.

During the pilot, a failure of this gate does not disable or block the local hooks: they continue to fail-open and only report their deterministic findings. The failure does block the **promotion** of Phase 1 as an approved performance budget. This separation avoids confusing fitness to keep observing with compliance with a latency promise.

## Transition to statistical analysis

Phase 1 detectors are deterministic facts and have a finite set of rules. They are not turned into a statistical text classifier. Their function is to instrument sessions and produce reliable labels.

The next layer works with tasks, not patterns. Each synthetic task declares an objective contract, a set of expected paths, and a scope-drift mutation. The runtime measures one vector per step:

\[
\mathbf{s}_t=[F_t,L_t,M_t,P_t,D_t,H_t,E_t]
\]

where (F\) is file count, (L\) churn, (M\) modules, (P\) public boundaries, (D\) distance to the objective, (H\) dispersion, and (E\) evidence. The features are computed deterministically; the statistical layer appears when comparing trajectories of comparable tasks, for example via median/MAD or conformal prediction.

The first experiment of this layer is versioned in `synthetic/scenarios/task-trajectories-v1.mjs`: 30 synthetic tasks, 15 in-contract changes and 15 scope deviations, with distractors, contracts, two steps per task, and a known outcome label. `npm run synthetic:trajectories` preserves the reproducible artifact at `synthetic/results/task-trajectories-v1.json` and runs only the passive extractor. It demonstrates that the pipeline can **measure and preserve coverage** of known trajectories; it does not demonstrate that it can separate them correctly, because there is no classifier or threshold.

No calendar is set for calibrating a passive alert: at least 100 eligible real trajectories and 30 adjudicated deviations are required, ideally complemented with historical replays. Before that volume, the product can log features but cannot emit a statistical signal to the agent.

## Dogfooding the passive round

On 2026-08-02, Objective Guard opened `R-F4-PASSIVE` from a clean tree via `adopt-baseline` followed by `next-round`. The prior contract `R-SESSION-BETA` and its baseline were archived under local hashes, and the `round-transition` event preserves both IDs. The new round's KR bounds and declares the extractors, datasets, task corpus, and Phase 4 documentation. This transition does not reinterpret the diff that produced E-005: it leaves it as historical evidence and starts the subsequent phase with versioned intent before new work.

## Organic observation

An organic finding is a signal produced during a real work round, not inserted for the experiment. It is a true positive only if an independent review confirms that:

1. there was a real deviation from a supported rule;
2. Objective Guard surfaced it before the round's close or the conventional check that would have revealed the problem;
3. the agent or developer explicitly fixed, justified, or rejected the finding;
4. diff, reproduction, and label are preserved without including secrets in the versioned dataset.

The unit of exposure is an eligible real round: a round with at least one file covered by one of the active detectors. The first observation window is 100 eligible rounds. It is not promised that an error will appear within 100 rounds: without knowing the base rate of drift, zero findings is evidence of low incidence or low coverage, not a validation nor a refutation.

## Organic gates and decisions

| Gate | Criterion | Decision |
| --- | --- | --- |
| O0 | 100 eligible rounds logged and labeled | Publish summary, even if there are no signals. |
| O1 | First confirmed organic TP, detected before the round's close/conventional check | Declare proof of case, not general efficacy. |
| O2 | 3 independent organic TPs, across at least two task types or rules, and at most 1 confirmed FP across the 100 rounds | Declare provisional evidence of usefulness of deterministic detectors. |
| O3 | Pre-registered control/placebo/guard trial | Evaluate H5: causal effect on time, tokens, and surface. |

A case does not count twice for re-runs, nor for multiple findings derived from the same patch. The pause caused by the guard is not causally attributed to the detector until O3: a placebo pause may produce part of the same benefit.

## Pre-registered design of O3

`npm run synthetic:trial-design` generates `synthetic/results/intervention-trial-design-v1.json`. It starts from the corpus of 30 tasks, assigns each task once to `control`, `placebo`, and `guard`, and permutes the order deterministically per task. The design therefore contains 90 units, 30 per arm, and verifies that each task is present in all three arms.

The artifact declares H5, the primary outcomes (`correct_resolution`, `tokens`, `elapsed_ms`, `unnecessary_surface`), the treatment of each arm, and its seed. Units are born with `outcome: null` and `observation_status: not-observed`; `causal_effect` is `null` and `intervention.enabled` is `false`. It is an executable, audited pre-registration of assignment, not a run trial or an efficacy estimate. Real observations must preserve the original assignment and be logged separately, without overwriting this design.

## Stopping criteria

- A synthetic FN blocks promotion of the corresponding rule.
- Two confirmed organic FPs before O2 suspend its blocking severity and require the rule to be reviewed.
- Zero TPs after O0 does not allow claiming usefulness; it is published as a null result, and a decision is made on whether to expand coverage or change the task family.
- Reaching O2 does not enable statistical alerts or objective contracts: it enables planning Phase 2 with an honest observation base.

## Artifacts

- `synthetic/scenarios/v1.mjs`: versioned definition of the frozen corpus.
- `synthetic/results/`: reproducible results of the calibration runs.
- `evidence/runs/`: sanitized export of dogfood and organic rounds.
- `.oguard/`: disposable local state and unversioned operational log.

Phase 1 evidence is interpreted together with the [scientific validation](05-scientific-validation.md) and the [roadmap](06-roadmap-and-criteria.md). Objective Guard must not turn an approved gate into a claim of the thesis's truth.