# Reviewer Coverage Recovery Design

## Summary

Make reviewer coverage reliable across models that return prose before JSON or reconstruct incomplete coverage receipts during JSON repair. Every review derives one canonical changed-file list from the current pull request snapshot, supplies that exact list to reviewers and repair agents, and retries only a reviewer whose parsed receipt still differs. A second mismatch fails closed with actionable missing and unexpected path diagnostics.

The design was informed by a controlled replay of packageauth PR 262 with `volc-ark/glm-5.2`. All five reviewers stated that they reviewed all 11 changed files, but all five returned natural-language analysis rather than a pure JSON object. The existing repair contract knew only that `reviewedFiles` contained paths; it did not receive the expected paths, so a repaired receipt could omit files and trigger an opaque mismatch.

## Goals

- Preserve the requirement that every reviewer covers every changed file.
- Generate coverage requirements dynamically for each pull request.
- Give initial and repair executions the same canonical file list.
- Recover once from a parsed but mismatched coverage receipt without rerunning successful reviewers.
- Identify the reviewer instance and exact missing or unexpected paths when recovery fails.
- Keep incomplete coverage fail-closed and prevent partial publication.

## Non-goals

- Hard-coding packageauth, PR 262, or any repository path.
- Removing or silently overriding model-provided coverage receipts.
- Treating natural-language claims alone as proof of coverage.
- Retrying indefinitely.
- Splitting the review into one model call per changed file.
- Inferring coverage from incomplete tool-call telemetry.

## Canonical Changed-File List

At the beginning of the reviewer stage, the orchestrator derives:

```ts
const expectedFiles = [
  ...new Set(request.snapshot.files.map((file) => file.path)),
].sort();
```

This array is the single source of truth for prompting, repair, verification, retry diagnostics, and progress reporting. It contains repository-relative current paths. For renamed files, `file.path` is required; `previousPath` remains diff context and is not a second coverage entry.

The generated list is serialized as JSON and included verbatim in the reviewer task. No repository-specific path appears in package source.

## Initial Reviewer Contract

The reviewer task includes a dedicated coverage section before the output contract:

```text
Expected changed files (copy these paths exactly into coverage.reviewedFiles):
<expected-files>
[...dynamic JSON array...]
</expected-files>
```

The instructions state:

- Inspect every listed file, even when it contains only documentation, tests, changelog text, or scoped instructions.
- When every file was inspected, return `complete: true` and copy the exact array into `reviewedFiles`.
- When any file could not be inspected, return `complete: false` and list only files actually inspected.
- Do not add `./`, absolute paths, previous rename paths, aliases, or omitted “irrelevant” files.

The runtime continues to validate the receipt rather than trusting the echoed array.

## Repair Contract

Reviewer repair receives:

- The complete reviewer JSON contract.
- The canonical expected-file array.
- The invalid prior response.
- The parser error.

The isolated repair prompt remains tool-free and receives no snapshot path. It may return `complete: true` with the canonical array only when the prior response explicitly states that every expected changed file was reviewed. If the prior response does not establish full coverage, repair must return `complete: false` and preserve the paths it can support.

Summary, triage, and validator repairs do not receive a changed-file list because their schemas contain no coverage receipt.

## Coverage Verification

Coverage verification returns a structured comparison rather than only throwing a generic error:

- `complete`: the reviewer decision.
- `missingFiles`: canonical paths absent from the receipt.
- `unexpectedFiles`: receipt paths not present in the canonical list.

Comparison remains exact and case-sensitive after duplicate removal. Paths are not normalized from `./`, absolute forms, or alternate separators because the contract requires exact repository-relative values.

## Targeted Reviewer Retry

When a parsed reviewer output has `complete: false`, missing files, or unexpected files, the orchestrator reruns only that reviewer instance once. The retry uses the same role prompt and repository read-only tools. Its task contains:

- The original snapshot task and PR summary.
- The canonical expected-file list.
- The previous structured reviewer output.
- Exact missing and unexpected paths.
- An instruction to inspect any missing files, re-check complete coverage, preserve still-valid findings, and return one complete replacement reviewer result.

The retry is a reviewer execution, not a JSON repair. It can access the repository because it may need to inspect omitted files. The replacement output passes through the normal parser and isolated repair path.

Each of the five reviewer instances has a stable diagnostic label:

- `instructions-review #1`
- `instructions-review #2`
- `bug-review #1`
- `bug-review #2`
- `objective-review #1`

Successful reviewers are never rerun. Parallel reviewer execution remains intact; each worker performs its own optional retry before it increments the completed counter.

## Failure Diagnostics

If the retry still has incomplete or mismatched coverage, the stage fails with a message such as:

```text
bug-review #2 coverage mismatch after retry.
Missing files: CHANGELOG.md, docs/verification/example.md
Unexpected files: ./AGENTS.md
```

Empty groups are omitted. A `complete: false` receipt with no path mismatch is reported explicitly as incomplete coverage.

The safe error is retained in the final incomplete report. No findings are published.

## Progress Reporting

Before a targeted retry, the orchestrator emits a `reviewer-coverage-retry` event containing:

- Reviewer label.
- Missing-file count.
- Unexpected-file count.
- Whether the reviewer returned `complete: false`.

The presenter retains one concise milestone and changes the live status to the reviewer retry. It does not print the complete file list in the conversation; the list appears only if the retry ultimately fails.

## Error and Cancellation Behavior

- A reviewer execution or repair process failure remains incomplete without a coverage retry.
- Cancellation remains aborted and does not start a retry.
- Invalid output after isolated repair remains incomplete; coverage retry requires a successfully parsed reviewer result.
- A retry receives the same abort signal and model selection as its original reviewer.
- UI progress failures remain observational and cannot affect review semantics.

## Testing

- Verify canonical lists are deduplicated and sorted from arbitrary snapshot files.
- Verify the initial reviewer task includes the exact dynamic list.
- Verify reviewer repair receives the exact list but no snapshot path or tools.
- Verify a complete exact receipt does not retry.
- Verify missing, unexpected, and `complete: false` receipts each retry only the affected reviewer.
- Verify a successful retry replaces the original output and preserves the five-reviewer result.
- Verify a second mismatch reports reviewer label plus exact missing and unexpected paths.
- Verify successful sibling reviewers are not rerun.
- Verify retry progress events and milestones.
- Verify execution failure, invalid JSON after repair, and cancellation do not enter coverage retry.
- Run formatting, TypeScript, complete tests, package verification, and a controlled PR 262 replay.

## Acceptance Criteria

- No changed-file path is hard-coded.
- Models receive an exact dynamically generated coverage list before reviewing.
- Repair cannot invent a full receipt without an explicit full-coverage claim in the prior response.
- A single malformed receipt does not discard successful sibling reviewer work.
- At most one coverage retry occurs per reviewer instance.
- Persistent mismatches expose actionable path differences and remain unpublished.
- PR 262 can complete the reviewer stage without weakening the all-files coverage requirement.