# v0.3 — Measurement and Provenance

v0.3 turns the harness from an execution gate into a measurable engineering platform.

## Engineering evals

An eval case lives at `evals/corpus/<case>/eval.yaml` and names a frozen `baseRef`, a task, optional fixture/setup, variants and deterministic expectations. `aeh eval run` creates a detached worktree, applies the fixture, runs the chosen variant, reads the harness run/report artifacts, scores the outcome, and destroys the worktree.

The score rewards deterministic success, first-pass success, low repair count, zero human intervention and bounded cost. `aeh eval compare` ranks historical results for the same case.

## Runtime metrics

Every `aeh run` now records first-pass success, repair count, human interventions, wall-clock duration and any token/cost fields discoverable from structured worker output. Human interventions are explicit events recorded with `aeh intervention` so they cannot be hidden inside model prose.

## OpenTelemetry

Local NDJSON remains the audit trail. Set `telemetry.exporter: otlp-http-json` and an endpoint (or standard `OTEL_EXPORTER_OTLP_ENDPOINT` / `OTEL_EXPORTER_OTLP_TRACES_ENDPOINT`) to send trace-shaped events to an OTLP collector. `aeh init` installs `.harness/otel-collector.yaml` as a minimal OSS collector configuration.

## Memory benchmark

`aeh memory-benchmark` runs the same YAML retrieval cases against every configured command provider. This intentionally does not make Engram, Cognee, Graphiti, or any memory store authoritative. Providers compete on recall, stale-answer contamination and latency.

## Test quality

`mutation` and `property` are first-class validator adapters. Their commands are explicit because the correct implementation depends on the consumer stack (for example Stryker.NET vs StrykerJS or FsCheck vs fast-check).

## Provenance

`aeh provenance generate --artifact <file>` creates:

- a SLSA v1 provenance predicate;
- an in-toto Statement v1 with the artifact SHA-256 as subject;
- an optional CycloneDX SBOM via Trivy;
- an optional Sigstore bundle via `cosign sign-blob` when `--sign` is requested.

The predicate links the artifact to the Git commit and, when a task ID is supplied, hashes of the run and validation reports.

## Release hardening

The release workflow reruns the complete harness check and `npm pack --dry-run`. Publishing requires either a published GitHub Release or explicit manual dispatch plus `NPM_TOKEN`, and uses npm provenance.
