# Verification Harness Specification

This is the contract for testing code that changes agent behavior, context
assembly, tools, orchestration, or user-visible development workflow.

## What the harness must observe

Do not assert only implementation labels. Observe the boundary the user or
parent model experiences.

For an agentic coding cycle, a useful harness contains:

```text
user objective
  -> model response/tool calls
  -> real or faithful tool result
  -> context assembly for next request
  -> model decision after evidence
  -> focused verifier result
  -> completion truth
```

The harness may use a scripted backend, temporary filesystem, fake clock, and
registered test tools. It must not bypass the changed dispatch path.

## Required scenarios for context/tool changes

### Full-source delivery

1. Create a bounded source file with a unique sentinel.
2. Have the model request an explicit full read.
3. Assert the next parent request contains the sentinel.
4. Assert it does not contain a partial branch wrapper in place of the body.

### Partial extraction

1. Create an oversized source with several explicit requirements.
2. Run the isolated extractor through the real extraction boundary.
3. Assert every requirement is either anchored or has a source-based unresolved
   reason and a deterministic recovery query.
4. Assert a partial extract never blocks a parent read or edit.
5. Assert a partial extract is not reused as a complete cache result.

### Rehydration and compaction

1. Read source, then compact the transcript.
2. Request the same file again.
3. If exact full source is visible and unchanged, assert it is returned as
   actual source.
4. If only a pointer/extract is visible, assert the read executes or restores
   actual source; never return a rejection-only wrapper.
5. Inspect compaction audit output and durable ledger separately from model
   context.

### Edit freedom and natural safety

1. Give the model exact current old text without a hash.
2. Assert `file_edit` reaches the actual tool and succeeds when unique.
3. Give stale text/hash and assert the actual tool reports its conflict.
4. Assert no branch/todo/controller admission layer turns either case into a
   synthetic tool failure.

### Verification truth

1. Change a source file.
2. Run a relevant verifier and record its pass/fail output.
3. Change that source again.
4. Assert the previous proof becomes stale.
5. Assert completion reports the current evidence state rather than the old
   pass.

## Test quality rules

- Name tests after the user-visible guarantee, not an internal mechanism.
- Include a sentinel or exact assertion that would fail under the previous bug.
- Assert both positive and negative behavior.
- Keep fixtures small except when file/context size is itself the behavior.
- Do not weaken a test merely because an implementation changed; update the
  contract only when the intended user-visible behavior changed.
- Avoid source-string assertions unless testing a literal compatibility/wiring
  requirement that cannot be observed another way.

## Live inference harness rule

Before a live model/extraction harness:

1. Confirm endpoint and exact model.
2. Inspect actual runner GPU placement.
3. Confirm the requested capable accelerator is used and an unintended device
   is clear.
4. State the result before generating tokens.

If any of these cannot be verified, do not run the live inference test.