# Essential external field testing

Real-provider checks consume tokens and normally run external CLIs in dangerous/no-approval modes. Claude uses `auto` permission mode instead when its effective UID is 0. Use them only in a trusted checkout. Receipts can contain prompts and source excerpts despite best-effort redaction.

## Preparation

```bash
cd /path/to/pi-flow-external
export PI_CODING_AGENT_DIR="${PI_CODING_AGENT_DIR:-$HOME/.pi/agent}"
export ROOT_MODEL="openai-codex/gpt-5.6-sol"
pi auth check --model "$ROOT_MODEL" --json
claude --version
codex --version
agy --version
```

Root Pi authentication is separate from each delegated CLI.

## Direct backend receipt

Run only the backend affected during development:

```bash
npm run e2e -- --backend claude
npm run e2e -- --backend codex
npm run e2e -- --backend agy
```

Each run creates a collision-safe temporary profile and fixture, delegates once, requires the expected token, checks for one complete `done` receipt, verifies that the fixture stayed clean, and removes its temporary files.

Defaults:

| Lane | Model | Thinking |
|---|---|---|
| Root Pi | `openai-codex/gpt-5.6-sol` | `high` |
| Claude | `claude-sonnet-5` | `high` |
| Codex | `gpt-5.6-sol` | `high` |
| Agy | `gemini-3.7-flash-high` | `high` |

Override with `--model`, `--thinking`, `--root-model`, or `--root-thinking`. Use `--keep` only when evidence inspection is necessary; it preserves sensitive output and the temporary profile path printed by the runner.

### Named Pi harness receipt

Run only when the pi runtime contract (§7 of the design), harness registry, or canonical role synthesis changes, and only if you have already registered a real `pi-*` harness in your own `harnesses.json` with real credentials configured:

```bash
npm run e2e -- --backend pi --harness pi-deepseek
```

This lane never registers a harness or writes a temporary profile — canonical synthesis already provides the `worker` role for any registered harness — it only delegates to the harness you name and requires the same one complete `done` receipt.

## Supervised workflow receipt

When workflow runtime, scheduling, supervision, or tool integration changes, run the affected backend; run all three before release:

```bash
npm run e2e -- --backend claude --workflow
npm run e2e -- --backend codex --workflow
npm run e2e -- --backend agy --workflow
npm run e2e -- --backend pi --harness pi-deepseek --workflow
```

Each command declares `meta.apiVersion: 1`, starts a two-child workflow with `background: true`, waits for the selected workflow through `external_runs`, inspects its summary/output, requires two complete child receipts, and verifies the read-only fixture stayed clean. Forced multi-page cursor behavior remains covered by deterministic offline tests; this check exercises real workflow child handling without duplicating those semantics.

## Change-triggered interruption and output check

When adapter cancellation, process-tree termination, partial output, or supervision changes, run the affected backend:

```bash
npm run e2e -- --backend claude --interrupt
npm run e2e -- --backend codex --interrupt
npm run e2e -- --backend agy --interrupt
```

The runner starts a background `Agent`, cancels its stable ID with an explicit reason, waits for its `cancelled` outcome, and asks `external_runs` for every available output and diagnostic page. A very early cancellation may legitimately have diagnostics but no assistant text; the durable receipt must still be `aborted`/`cancelled` with the exact reason. Do not retry a failure automatically. Use `--keep` for one deliberate evidence inspection, then remove the printed run root and temporary profile.

## Natural-language routing smoke

When role discovery, tool descriptions, or coordinator guidance changes, run three fresh Pi sessions against the current checkout and a read-only fixture. Ask for two named harnesses to review, two named harnesses to research, and a task split between Pi plus two named harnesses. For each session, verify that each requested external harness produced one complete `done` receipt through `role` plus `harness`, no help/discovery call was needed for the built-in roles, and the fixture stayed unchanged.

This is a qualitative trigger smoke, not a statistical regression comparison or a model-independent guarantee. It does not cover workflow routing or the omitted-`harness` default; use the workflow receipt above and a separate direct role request without `harness` for those paths.

## Change-triggered nested timeout check

Run a real nested-agent timeout scenario only when nested-event detection or timeout-extension code changes. Use an Agy profile that invokes one native subagent, set a short bounded timeout, and verify the latest `summary.json`:

```bash
jq -e '.summary.status == "done" and
  .summary.nestedActivitySeen == true and
  .summary.nestedTimeoutExtended == true and
  .summary.effectiveTimeoutMs > .summary.configuredTimeoutMs' \
  "$PI_FLOW_EXTERNAL_RUNS_DIR"/run_*/summary.json
```

## Change-triggered context transfer check

Run only when parent-context selection or serialization changes. In one fresh Pi session against a read-only fixture, delegate once with `context: { mode: "recent", turns: 1 }` and verify the child still returns the expected marker, the run has one complete `done` receipt, and that `summary.json` reports the matching `context` block:

```bash
jq -e '.summary.context.mode == "recent" and .summary.context.requestedTurns == 1 and .summary.context.sharedTurns >= 1' \
  "$PI_FLOW_EXTERNAL_RUNS_DIR"/run_*/summary.json
```

Local fixture tests own selection, boundary, and failure behavior; this check only proves a real backend tolerates the transferred block.

Do not run damaged-record and expected-failure provider scenarios manually. Their behavior is deterministic and belongs in the local fixture tests.

## Change-triggered project default-harness check

Run only when project default-harness resolution changes. In a fresh Pi session against a trusted read-only fixture containing `.pi/pi-flow-external/settings.json` with `"defaultHarness": "claude"`, verify `/external settings` reports `defaultHarness: claude (project: ...)`, delegate one read-only task with `role` only (no `harness`) and verify the receipt names a `claude-*` profile, then confirm an explicit `harness: "codex"` call still routes to codex. Repeat once in the same fixture with trust removed and confirm the override is ignored with a warning.

## Release minimum

Before a runtime release:

1. Run `npm run check`.
2. Run all three direct backend receipts.
3. Run all three supervised workflow receipts.
4. Run interruption checks for adapters whose cancellation/output path changed.
5. Run the natural-language routing smoke only if role discovery, tool descriptions, or coordinator guidance changed.
6. Run the nested timeout check only if nested detection or timeout behavior changed.
7. Remove temporary evidence and profiles.