# Testing Herdr Subagents

Run the core suite:

```sh
npm test --workspace @maxiaochao/pi-herdr-subagents
```

It verifies:

- global/project `defaultExtensions` replacement, trimming, source attribution, Persona JSON/Markdown precedence, explicit empty-list clearing, invalid-value diagnostics, and per-call reloads;
- concise Persona names/descriptions in model-facing listings and unknown-Persona errors, including empty lists and visible configuration diagnostics;
- resolved defaults in control result metadata, settings output, and child CLI arguments while preserving mandatory extensions;
- strict serial start order at concurrency one;
- bounded read-only parallelism;
- ordered results despite concurrent completion;
- sibling continuation after failure and all-failed aggregation;
- required explicit Personas in the tool schema and execution, including legacy single-task argument normalization;
- independently optional task `model` and `thinking`, override precedence above Persona/Markdown/default/parent values, all seven Pi thinking levels and trimmed values, explicit `off`, unchanged omitted-field inheritance, CLI/record/history propagation, and invalid types/blank/unknown thinking rejected before side effects;
- model-facing role selection guidance for broad independent review, codebase investigation, file-changing work, and charter-driven agent-team execution;
- effective Persona descriptions in parent listings, corresponding bodies in child system prompts, and unchanged read/write tool boundaries;
- child system prompts require a terminal `subagent_report` when work finishes or cannot be completed, not a normal text reply or a process-exit condition;
- rejection of missing, blank, invalid, and unknown Personas before creating runs, while custom Personas remain supported;
- explicit rejection of parallel write-capable tasks with the conflicting task, Persona, and corrective action;
- waiting through a delayed authoritative report after a prompt transport error while the Pi process is alive;
- explicit failure when the process exits without a report;
- parent cancellation of running and queued tasks;
- startup isolation: a new Pi session neither mutates historical records nor closes their recorded Herdr tabs;
- omitted exposure delegates both tools to Pi's default, while project configuration can consistently select model-only exposure;
- compact history pages (10 by default, maximum 50), stable cursors during newer inserts, invalid-page rejection, no task/report expansion, and on-demand status summaries;
- immutable terminal task records;
- idempotent duplicate reports and rejection of conflicting replacements;
- keeping a settled child alive for focus-aware cleanup instead of force-exiting its TUI;
- clearing the legacy parent shortcut footer hint while preserving other extension status, with `Alt+P` and `/parent` still focusing the exact parent pane;
- adding `apply_patch` to the child CLI allowlist only when the Persona permits `edit`;
- preserving the configured smoke model, thinking level, and authentication extensions while isolating unrelated settings;
- project-over-global smoke configuration, explicit extension clearing, model overrides, and local-source paths after moving to a temporary directory.

For the current test entry point and the disposition of the former lifecycle suite, see [Maintained suite and legacy migration](#maintained-suite-and-legacy-migration).

Run extension source loading. The direct-Pi fixture resolves global `defaultExtensions` and validates real file/package loading; Herdr's `builtin:codemode` alias is validated by the real-Herdr smoke because the alias is translated by `herdr agent start`:

```sh
npm run load-test --workspace @maxiaochao/pi-herdr-subagents
```

The native CLI fixture also exercises an extension that switches `edit` to `apply_patch`. It checks the final active tools, not just status text: writable children retain the selected patch tool, read-only/no-editor children do not gain it, and children without the patch extension retain native `edit`. This deterministic probe does not call a model or test Codex Edit's parser.

Run the real Herdr smoke test from a Herdr session:

```sh
npm run test:real-herdr --workspace @maxiaochao/pi-herdr-subagents
```

The smoke reads the effective `analyst` model, thinking level, and explicit `extensions` from the invoking project's normal Herdr configuration (npm's `INIT_CWD`, or the current directory when run directly). It copies only those fields into its temporary configuration. It keeps the bundled read-only Persona, tool allowlist, and test concurrency; user skills, prompts, and unrelated settings are not copied. Local extension paths remain relative to the invoking project. The concurrent package-loading probe appends its test package rather than replacing the configured extensions. The isolated smoke supplies these sources through global `defaultExtensions`. The single child inherits configuration; concurrent tasks explicitly select the configured model (when present) and `thinking: "low"`. Both cases check task records and the real child's persisted thinking-level selection, actual child loading, and tab cleanup.

This preserves authentication extensions such as `npm:@maxiaochao/pi-openai-account-pool` in both the single and concurrent cases. Children still use `--no-extensions`: only the mandatory child extensions, configured explicit sources, and test probe load. The smoke does not copy or edit credentials. If no Herdr model is configured, Pi's normal startup model selection still applies.

`HERDR_SUBAGENTS_E2E_MODEL=provider/model` overrides only the smoke model; configured extensions remain loaded. Set `HERDR_SUBAGENTS_E2E_FOCUS_CLEANUP=1` to also verify deferred cleanup while focused and restoration of the original workspace/tab focus.

For real smoke testing, verify that each child appears in a visible pane and can be focused and inspected. An early-finishing child must remain visible while sibling tasks continue. After the whole run settles, a focused child must remain at its restored Pi UI until focus leaves the run tab; the tab must then close automatically. Verify that the parent returns only after all requested tasks settle. Also verify cancellation closes the run tab and does not overwrite a report already atomically published.

## Maintained suite and legacy migration

`npm test` runs `tests/run-tests.mjs`. Each file runs with the installed Pi peer resolver in an isolated process. A 120-second per-file watchdog terminates a stuck test and names the file; it does not delay or decide task completion. `npm run test:legacy` is a compatibility alias for the same suite.

The former monolithic legacy suite encoded removed contracts: background queues, `respond`/`resume`, and interpreting an idle child as final. Its `silent` fake returned `isTaskRunning() === true` forever without publishing an attempt report. The parent correctly waited for that still-live child. Separate stale validation assertions omitted the required Persona and allowed removed actions; exact CLI-call assertions omitted normal process-liveness observations. These were stale expectations, not grounds for changing production completion semantics.

Current coverage is in focused suites:

- `core-state-machine.mjs`: required Personas, serial/read-only parallel work, transport recovery, report/exit completion, cancellation, immutable terminal tasks, tool exposure and startup isolation.
- `live-agent.mjs`: a live silent child that accepted the prompt keeps waiting after prompt acknowledgement failure; a delivery failure without an accepted attempt fails with the Herdr cause instead of waiting forever; an explicit report completes it, explicit process exit without a report fails it, and explicit cancellation ends the wait. Event gates advance these cases without fixed sleeps.
- `fleet.mjs`: retained Fleet keyboard selection, usage accounting, elapsed-time compatibility, widths/colours, watcher cleanup and stale-update regressions from the old monolith.
- `fleet-click.mjs`: select task rows, not the header or blank rows; ignore non-primary clicks; focus exact panes; handle queued/missing panes; preserve the last painted row's identity across an unpainted update; ignore disposed widgets.
- `cleanup-timing.mjs`: focused deferred cleanup under a held state lock, then successful/failed durable cleanup commits. It does not test retired restart/recovery workflows.
- `config.mjs`, `report-boundary.mjs`, `report-lifecycle.mjs`, `standalone-package.mjs` and `real-herdr-config.test.mjs`: configuration, native child settlement, standalone lifecycle and smoke configuration. `load-test` also runs `extension-sources-load.mjs` against native Pi package loading.

The `extension-contract.ts` load fixture now checks registration gates, the current schemas and single-task normalization, foreground waiting through a failed sibling, Fleet keyboard focus, historical blocked rendering, and no duplicate background notification. `load-extension-contract.mjs` requires its explicit completion marker: a zero Pi exit without finishing the fixture is a failure. Both the default suite and `load-test` execute this check.

Removed fixture scenarios asserted retired background queues, early handoff, public respond/resume and automatic startup recovery. They are not restored. Current owner cancellation and shutdown coverage lives in `ownership-cleanup.mjs`; keyboard/rendering coverage remains in `fleet.mjs` and `fleet-click.mjs`. No retained current contract is dropped to make the fixture complete.

## Restored regression coverage after migration review

The default runner also executes these retained current-contract checks:

- `config-resolution.mjs`: package/global/project whole-Persona replacement; model/thinking/skills/extensions merge, diagnostics and sources; Markdown precedence; effective values, per-task model overrides, CLI tool boundaries, isolated child environment, no focus theft, system prompts, durable records and history through public batch execution.
- `task-overrides.mjs`: neither/either/both optional task overrides; Persona/Markdown/default precedence; parent-model thinking inheritance and unchanged behavior when switching model; CLI, task records and history; invalid thinking fails without creating a run.
- `live-agent.mjs`: normal idle/done acknowledgements and transient process observation errors remain nonterminal; a final report stops an outstanding prompt acknowledgement. These checks use real temporary files and explicit events, without patching Node file APIs or imposing additional cancellation/join guarantees.
- `batch-contracts.mjs`: current control allow/deny table, Herdr-required zero-side-effect rejection, exact cancel dispatch, uncertain create/split/start failures without duplicate execution, invalid JSON/status/documents/attempt reports and archived failure records, stale canonical report rejection, cross-attempt result isolation, cleanup warnings preserving business results, and a fake-clock stalled warning that does not terminate live work.
- `history-storage.mjs`: selected status/history without full results or stale running-task reports; damaged record isolation; positive-integer task order; active/blocked/pending-cleanup deletion refusal including an internal active-run invariant with a corrupt task; public selected-only removal and safe failure on damaged records; atomic snapshots; explicit lock publication/release gates, cancelled lock waits and concurrent field preservation; idempotent runtime ignore rules that retain Persona sources.
- `ownership-cleanup.mjs`: durable owner/run isolation; registered control cancel and session shutdown abort/join only their owned in-flight work; immediate/deferred cancellation, missing tabs, focus/close errors, preserved business outcomes and concurrent run fields. It exercises current shutdown, not automatic startup restoration of historical work.
- `herdr-cli.mjs`: actual local executable/JSON adapter for shell versus native/Node-hosted Pi, temporary query failures versus confirmed missing panes, malformed replies, command argv, and abort-listener disposal. Focus-poll timers are advanced by a fake clock; no real Herdr or paid model is needed.

Background queues, public blocked/respond/resume, early detach, automatic startup reconciliation and historical restart remain retired per the README. They are not counted as missing current execution coverage. Historical read/status/cleanup compatibility and shared locking remain supported and are covered above. No fixed grace sleep was added to decide business completion.

The corrupt-task deletion case checks an internal invariant of `cleanSubagentRun`: deletion eligibility comes from the durable run, not the error-tolerant history display. This is not evidence of a public-tool deletion bug. The public cleanup action still rejects damaged task/report records during status loading; it does not gain a damaged-record recovery mode.

## Real Fleet click check

Use a fullscreen parent Pi loaded from `extensions/herdr-subagents.ts`. Run three read-only Explorer tasks concurrently in real child Pi panes. Hold each child's real Bash tool behind an explicit release event, then read the parent's visible Fleet rows. Send primary SGR press/release events to each row through `herdr pane send-text`. After each click, check `herdr pane list`: the focused pane must equal that task's persisted `paneId`. Release the tasks, verify all three reports complete, and verify the child tab is removed. An isolated offline provider makes this repeatable; override the Persona's model and extensions explicitly, since Persona settings outrank defaults. Restore focus and remove only test-owned tabs.

## Missing-report regressions

`tests/report-boundary.mjs` checks completed/failed report idempotency, one durable reminder across rounds and extension reload, conflicting report rejection, unconfirmed diagnostics with a session path, Esc grace period (timer cancelled by steering or shutdown, `child_aborted` after expiry), and error without continuation.

`tests/report-lifecycle.mjs` uses the real installed Pi SDK and a fully offline provider. It covers ten cases: normal report, failed report, successful reminder, repeated omission, transient retry, exhausted retry, queued follow-up, another before-settle continuation, Esc abort, and a fresh prompt from a settled listener. A normal report makes one request; a reminder adds at most one. Native abort does not emit before-settle; its terminal diagnostic appears only after `abortGraceSeconds` (set to 1 in the test). These are native lifecycle checks, not a claim of real-Herdr pane/concurrency coverage.

Use `PI_TEST_BINARY` to select Pi 1.4.0 for these checks rather than an older workspace-local Pi. No paid model or real credential is used by the new regression fixtures.

`tests/standalone-package.mjs` packs the actual child package into a temporary directory, unpacks it outside the workspace, and runs all ten native/offline lifecycle scenarios there. `tests/runtime.mjs` resolves peers from the selected installed Pi; no sibling package or copied Pi runtime is needed.

The media recorder also uses the effective analyst model, thinking level and explicit extension sources for its parent Pi. `PI_HERDR_MEDIA_MODEL` overrides only the model; required authentication extensions remain loaded. It does not copy or expose credentials.