# Verification results

## Release 1.0.0

Version 1.0.0 closes the run that started at the unreleased 0.2.0. Since then, each change shipped as its own verified release: pi 1.3.0 support (0.3.0), throwaway-sshd verification in CI (0.4.0), target visibility (0.5.0), bounded remote payloads (0.6.0), dead-peer detection (0.7.1), the Diataxis docs rework (0.8.0), and catalog metadata (0.8.1). This version changes no tool behavior over 0.8.1. It marks the line where all of that is the stable baseline.

Final local evidence on macOS with Node.js 26.1.0 and pi 1.3.0. `npm run check` passed 57 tests with strict TypeScript, run unpiped after the 0.7.0 lesson. `npm run test:docs` checked 41 relative links, zero broken. `npm run test:package` passed the isolated pi 1.3.0 install with zero production dependencies. `npm run test:e2e` passed both real-SSH scenarios through the loopback sshd. Both npm audits reported zero vulnerabilities. Hosted CI on the release commit runs the six-job matrix plus the `ssh-e2e` job. Their results ride with the tag and the GitHub release.

npm state at release time. Staged versions awaiting maintainer approval with 2FA, oldest first: 0.2.0, 0.3.0, 0.4.0, 0.5.0, 0.6.0, 0.7.1, 0.8.0, 0.8.1, 1.0.0. Only versions the maintainer approves become public. Approving 1.0.0 alone is sufficient for users. Version 0.7.0 was never staged and must never be published. Until an approval lands, `npm view @michaelasper/pi-ssh` keeps showing 0.1.2 and the pi.dev catalog keeps its current listing state.

## pi.dev catalog investigation: 0.8.1

Question. Why does `@michaelasper/pi-ssh` not show up in the pi.dev package catalog despite public npm publication with the `pi-package` keyword?

Answer, in two layers. First, the catalog could only ever show version 0.1.2, because every later version sat staged on npm without maintainer approval. Second, and worse, catalog search omits the package entirely while its detail page renders. Measured 2026-10-19: npm search reports 12,300 packages with the `pi-package` keyword. Paging that query in blocks of 250 reaches 5,249 unique names and then stops returning new ones. The catalog lists about 5,360 packages. `@michaelasper/pi-ssh` is outside the reachable window, so a crawler that enumerates this way cannot index it. Exact-name npm search finds the package, its detail page returns 200, and its metadata carries the keyword plus the `pi.extensions` manifest. Eligibility is fine. Ranking inside a capped enumeration is the failure. Upstream issue [earendil-works/pi 10145](https://github.com/earendil-works/pi/issues/10145) documents the identical symptom and mechanism for another package.

Fixes applied here. Package metadata gained the standard `author` field and the `pi-extension` keyword. The release guide now records the mechanism, the staged-version list, and a three-step post-approval confirmation. The expected unlock is the maintainer approving a fresh version, which gives the package a new publish event in npm search. Until then, `pi install npm:@michaelasper/pi-ssh` and the detail page keep working. Sources: the live catalog and search endpoints above, npm registry search at https://registry.npmjs.org/-/v1/search, npm downloads at https://api.npmjs.org/downloads/point/last-month/@michaelasper/pi-ssh, and the staged publishing docs at https://docs.npmjs.com/staged-publishing/ (no expiry stated for staged packages).

## Diataxis docs cleanup: 0.8.0

User docs are now exactly four files, one mode each. The tutorial teaches by doing, the how-to gives task recipes, the reference states facts, and the explanation covers boundaries and reasons. Development, releasing, and this file moved under `docs/maintainers/`. The dated 0.2.0 release page and the root `PLAN.md` were retired. Their living content already existed in the how-to migration guide and the explanation. The README shrank to a landing page that links into the four modes.

Evidence. `npm run test:docs` runs the new link checker in `scripts/check-links.py`. It checked 40 relative links and anchors across the README, the docs, and the verification skill with zero broken. CI runs the same check. `npm run check` and `npm run test:package` passed. The package still ships `docs/`, so the maintainer pages travel with it.

## Dead-peer detection: 0.7.1

Version 0.7.0 was tagged with a typecheck error in the new `ssh -G` test, which a piped local check hid. The CI verify job failed and nothing was staged or released, so the fix shipped as 0.7.1 and the v0.7.0 tag stays unreleased by design.

Every SSH invocation now carries `ServerAliveInterval=15` and `ServerAliveCountMax=3`. A peer that stops answering at the protocol level ends bash and file calls after about 45 seconds of silence instead of hanging until TCP gives up.

Evidence. `npm run check` passed 57 tests. The exact-argv test pins both options, and a new test runs the real `ssh -G` against the built arguments to confirm the effective client configuration resolves to 15, 3, the configured connect timeout, and batch mode. `npm run test:e2e` passed both real-SSH scenarios through the loopback sshd, which confirms live peers are unaffected. A true dead-peer run, with a server that halts mid-session, is not in the suite. The option values are OpenSSH's documented mechanism for exactly that case.

## Bounded remote payloads: 0.6.0

Remote reads now refuse files over 33,554,432 bytes with an error that names the size and the limit. The guard covers read, the read inside edit, and the context load inside grep. Grep without context still searches such files. The local transport also stops any helper response over 67,108,864 bytes.

Evidence. `npm run check` passed 56 tests. New tests build a sparse 33,554,433-byte file and assert the exact size and limit in the read, edit, and grep-context errors, plus a flooding fake SSH peer that the client cuts off at a test-sized limit. `npm run test:e2e` passed both real-SSH scenarios, including a remotely truncated sparse file that the read tool rejects. `npm run test:package` passed.

## Target visibility: 0.5.0

The session now shows its SSH target before any tool runs. A `/ssh` command reports the default host, remote cwd, and connection timeout. The footer shows `ssh: <host>` under a remote default and `ssh: config error` for invalid configuration. Local defaults show nothing.

Evidence. `npm run check` passed 54 tests, including a new extension test that drives remote, local, and broken configurations and asserts the exact status values and notice text. A real pi 1.3.0 launch with a probe extension listed `ssh` among the registered commands. `npm run test:e2e` passed both real-SSH scenarios. `npm run test:package` passed under pi 1.3.0.

## Throwaway sshd verification: 0.4.0

The real-SSH suite no longer needs an external authorized host. `scripts/ssh-e2e.sh` starts a user-owned sshd on loopback with generated keys, runs `test/ssh.e2e.ts` against it through an isolated client config, and removes every artifact it created. The same script backs a new CI job and the project-local verification skill at `.pi/skills/verify-pi-ssh/`.

Recorded run on macOS with Node.js 26.1.0. `npm run test:e2e` passed both scenarios. Bash routing took about 2.8 seconds. The file tools scenario took about 14.9 seconds and asserted its own remote cleanup. After the run, the evidence log remained at `.artifacts/ssh-e2e.log`, and no temp dir or harness sshd process remained. Limits stay honest. Loopback runs prove protocol, quoting, and routing behavior. They do not prove behavior against a different remote operating system or account.

## Pi 1.3.0 compatibility: 0.3.0

Development dependencies moved from pi 0.85.1 to pi 1.3.0. Three upstream changes required source or test updates.

- Tool `execute` now receives `ExtensionToolContext`, which adds `tools` and `executeTool`. The extension and its test fixtures use the new type. Spreading the live context preserves both fields for the in-memory native algorithms.
- Bash failures now resolve instead of throwing. A command that exits non-zero returns `isError: true` with `structuredContent.exit_code`. Timeouts, aborts, and invalid arguments still throw. Tests assert the resolved shape for local, remote, bad-cwd, and transport failures, including literal exit codes 7 and 23.
- The image detector now requires a full `GIF87a` or `GIF89a` signature. Text that starts with `GIF` no longer counts as an image. The mirrored detector in `src/image-mime.ts` matches, and the differential tests cover both signatures and the rejected prefix.

Recorded commands on macOS with Node.js 26.1.0 and pi 1.3.0 on `PATH`:

| Command | Outcome |
| --- | --- |
| `npm run check` | All 53 tests passed, including the differential find, grep, and image suites against the 1.3.0 natives. Strict TypeScript passed. |
| `npm run test:package` | The 21-file production tarball passed isolated pi 1.3.0 installation. All seven host-aware schemas registered. Zero installed production dependencies and zero audit findings. |
| `npm run bench` | Local median 1.733 ms built-in against 1.739 ms through the wrapper over 100 samples. SSH command construction averaged 0.78 microseconds. |
| `npm audit --audit-level=low` | Zero vulnerabilities. |

Schema tests also assert that every wrapped tool keeps the native `outputSchema` and `constrainedSampling`. Codemode scripts reach these tools through the same definitions, so they keep the structured results pi 1.3.0 provides. No real-SSH run is recorded for 0.3.0. The opt-in suite assertions were updated for the new bash failure shape, and the run itself moves to throwaway sshd coverage in the next release.

## Release preparation: 0.2.0

After updating version metadata and release documentation:

| Command | Outcome |
| --- | --- |
| `npm run check` | All 53 tests passed, with no failures or skips; strict TypeScript checks passed. |
| `npm run test:package` | The 21-file production tarball passed isolated pi installation and startup; all seven host-aware schemas registered; zero installed production dependencies and zero audit findings. |
| `npm audit --audit-level=low` and `npm audit --omit=dev --audit-level=low` | Both reported zero vulnerabilities. |
| `PI_SSH_TEST_HOST=<authorised-target> npm run test:ssh` | Both scenarios failed at their initial connection because the previously authorised host was unreachable (SSH connection timeout). No test fixtures were created. This release-time rerun does not supersede the earlier successful SSH evidence below. |

Before release preparation, live calls through the installed extension also passed for all six file tools on the authorised remote and locally, including special-character paths, pagination, multi-block edits, a remote PNG read, rejected edits without modification and host isolation. One local read overlapped an edit and failed; a sequential retry passed. All live-test fixtures were removed. No feature implementation changed between those successful checks and release preparation.

Local release logs are retained in ignored `.artifacts/release-0.2.0-{check,package,audit,ssh}.log`. These checks do not establish npm staging or public publication. Consult the [release workflow](https://github.com/michaelasper/pi-ssh/actions/workflows/publish.yml) for hosted verification and staging outcomes; npm still requires maintainer approval.

## Host-aware file tools: implementation verification

The current source adds `host` to read, write, edit, find, grep and ls. These observations are from macOS with Node.js 26.1.0 and pi 0.85.1; they are not cross-platform or model-reliability guarantees. Private target names, fixture paths and raw transcripts are excluded.

Recorded commands:

| Command | Outcome |
| --- | --- |
| `npm run check` | 53 tests passed; zero failed, cancelled or skipped; strict TypeScript checks passed. |
| `npm run test:package` | Production tarball passed isolated pi installation and startup; all seven host-aware schemas registered; both Python helpers included; zero installed production dependencies and zero production audit findings. |
| `PI_SSH_TEST_HOST=<authorised-target> npm run test:ssh` | Both opt-in scenarios passed: existing bash behaviour and all six file tools. Temporary local and remote fixtures were removed and remote removal was asserted. |

The real target was explicitly authorised. Python 3.9.6 and ripgrep 15.2.0 were already installed there. With separate user authorisation, Homebrew installed fd 10.5.0. The extension itself never installs prerequisites. No SSH security, credentials, shell profile or global `PATH` settings were changed; the search helpers found the Homebrew executables through their documented fallback locations. No remote pi installation, persistent helper or file mirror was created.

Coverage includes:

- Six schemas add only optional `host`; native local results and live session cwd; configuration precedence and errors blocking all seven tools.
- Default and explicit remote execution, explicit local override for every tool, configured remote cwd, remote login cwd, absolute paths, selected-machine home expansion, `@`, file URLs, Unicode-space and screenshot conveniences, symlink canonicalisation, special-character paths and literal contents over stdin.
- Native read pagination, line and byte truncation, image results and signature parity for JPEG, PNG, GIF, WebP and BMP (including malformed headers and animated-PNG exclusions). Real SSH image execution used PNG; other formats have local differential signature coverage.
- Native edit multi-replacement matching against the original, overlap/non-unique/missing-match rejection without writing, BOM/CRLF preservation, diffs and unified patches. Concurrent edit/edit and write/edit calls sharing remote symlink or hard-link aliases retain both changes. Queue tests distinguish hosts and unrelated files and recover after acknowledged failures and cancellation. An unacknowledged mutation transport failure blocks queued and future mutations of the affected path/inode until pi restarts; reads remain available for inspection.
- Find and grep differential coverage for ignore rules, nested repositories, glob/regex handling, case/literal flags, context, result limits and byte limits; ls sorting, hidden files, symlinks, skipped stat failures and entry limits. Search output has no artificial 2,000-line cap. Intentional safety differences: remote find uses NUL framing instead of trimming/splitting filenames, and remote grep preserves Unicode line separators in JSON records and rejects malformed utility output rather than silently discarding it.
- Mocked SSH authentication failure, malformed protocol output, missing Python/search utilities, pre-aborted and active cancellation, bounded stderr and subprocess cleanup. Active writes settle before cancellation releases the mutation queue. Real SSH includes existing bash timeout/active-cancellation checks and pre-aborted file mutation.
- Target headers at narrow and wide widths, native result rendering and prevention of local edit previews for remote targets, including a streamed host override arriving after an initial local-default header. A local overflow artifact from remote bash is read with `host="local"`.

Current README, tutorial, how-to, reference, explanation, development and plan guidance were re-read. A source/docs scan found no obsolete guidance claiming that only bash can run remotely; historical bash-only records remain explicitly labelled. The CI workflows now explicitly provision local Python/fd/rg test prerequisites; both YAML files parsed successfully and `git diff --check` passed. No hosted workflow was triggered as part of that implementation verification.

Coverage limits: Windows, the hosted CI matrix and Linux-to-remote execution were not run in this record. There is no multiprocess/external-editor locking, transactional rollback or guarantee of terminating arbitrary remote descendants after disconnect. Complete file payloads and directory listings are buffered before native output limits; pagination is not a bounded-network-transfer promise. The existing model-comprehension and benchmark results below apply to the earlier bash-only implementation, not the new file tools. No release or publication was performed as part of that implementation verification.

Raw local logs are retained in ignored `.artifacts/file-tools-check.log`, `.artifacts/file-tools-focused.log`, `.artifacts/file-tools-ssh-e2e.log` and `.artifacts/file-tools-package.log` and `.artifacts/file-tools-guidance.log`.

## Historical bash-only verification

The following sections describe development and distribution of versions 0.1.0–0.1.2 before the file-tool extension. They are retained as historical evidence, not current file-tool coverage.

## Test-first development

| Increment | Observed red state | Observed green state |
| --- | --- | --- |
| Routing and configuration | New tests failed because the config module did not exist. | 5 tests passed. |
| SSH quoting and directory handling | New tests failed because the command builder did not exist. | 8 tests passed. |
| Tool integration | New tests failed because the tool module did not exist. | 11 tests and strict TypeScript checks passed after correcting test-fixture path canonicalisation and escaping. |
| Extension lifecycle | New tests failed because the entry point did not exist. | 13 tests and strict TypeScript checks passed. |
| Review: local settings | Regression test returned `unset:unset` instead of the configured prefix and shell marker. | 14 tests passed, including trusted/untrusted settings and no remote leakage. |
| Review: target visibility | Regression test rendered only `$ pwd`, without a target or cwd. | 15 tests passed, including non-mutating headers and wrapping at 40, 80 and 120 columns. |

The final 15-test suite and strict TypeScript checks pass. The suite also runs on the declared minimum Node.js version, **22.19.0**, using a temporary isolated Node installation.

Raw red/green logs are retained locally in ignored `.artifacts/01-*` through `.artifacts/06-*`. They are not package contents.

## Real SSH

The opt-in real-host test passed against an authorised SSH server using strict host-key verification. It checked explicit and default remote targeting, explicit local execution, literal quotes and Unicode, remote working directories, non-zero exits, prevention of commands after a failed `cd`, 60,000-byte output with a local full-output file, timeouts, active and pre-aborted cancellation, and remote temporary-directory removal.

The combined scenario took approximately 2.1 seconds, excluding test-runner startup. That is not a fresh-connection latency benchmark; existing SSH configuration can reuse connections. No persistent remote installation was made. Remote descendant termination is not guaranteed or claimed.

## Model comprehension

Two fresh isolated CLI sessions used **`openai-codex/gpt-5.6-luna`**, thinking level **low**, with pi **0.85.1**. Both passed initially and again after the review fixes, with three calls per scenario in each run:

| Scenario | Remote success | Intentional remote failure | Local success |
| --- | --- | --- | --- |
| Explicit SSH target in prompt | Named `host`, output `REMOTE_OK` | Same host, error containing exit 7 and `EXPECTED_FAILURE` | `host="local"`, output `LOCAL_OK` |
| Configured remote default | Omitted `host`, output `REMOTE_OK` | Omitted `host`, error containing exit 7 and `EXPECTED_FAILURE` | `host="local"`, output `LOCAL_OK` |

The prompt pattern was:

> Use the available bash tool to run exactly `printf 'REMOTE_OK\n'` on [SSH target / the configured default machine]. Then run exactly `printf 'EXPECTED_FAILURE\n' >&2; exit 7` on that same machine. That failure is intentional: do not retry or change it. Finally, on the LOCAL machine running pi, run exactly `printf 'LOCAL_OK\n'`. Do not run any other commands or inspect any files. Briefly report the three outcomes and where each ran.

Both final reports correctly distinguished remote success, intentional remote failure and local success. The default-host report was:

```text
- Remote: `REMOTE_OK` — succeeded.
- Remote: `EXPECTED_FAILURE` — exited with code 7 (intentional).
- Local: `LOCAL_OK` — succeeded.
```

A test-only gate restricted execution to those exact probe commands. These are constrained usability checks of the requested model, not evidence that arbitrary weaker models always route correctly or that the model can diagnose an unexpected transport failure.

## Performance

One run on macOS arm64, Node.js 26.1.0, pi 0.85.1:

| Local execution | Samples | Median | p95 |
| --- | ---: | ---: | ---: |
| Built-in bash | 100 | 1.494 ms | 1.624 ms |
| pi-ssh, local route | 100 | 1.490 ms | 1.595 ms |

SSH command construction averaged approximately **0.63 µs** over 100,000 iterations. The local medians differed by approximately **0.004 ms** in this run, within normal run-to-run noise; this is not evidence that the wrapper is faster. Scheduling noise and environment differences preclude a general overhead guarantee. No network acceleration is claimed.

## Packaging

A production tarball containing 12 allowlisted files passed an isolated `pi install` and live startup check. A normal `npm install --omit=dev` installed zero production dependencies and emitted no warnings, with zero production audit findings. The extended bash schema was present without development dependencies, legacy-peer flags or personal pi settings. Strict TypeScript checks also cover the real-host test and benchmark. The initial publication’s [six GitHub Actions jobs](https://github.com/michaelasper/pi-ssh/actions/runs/33984923551) passed on macOS and Linux with Node.js 22, 24 and 26.

The packaging review followed the upstream [pi package guidance](https://github.com/earendil-works/pi/blob/main/packages/coding-agent/docs/packages.md): explicit `pi.extensions`, the `pi-package` keyword, narrow published files, optional `*` peers supplied by pi, and no bundled core packages or build/install hooks.

## Dependency review before release

Registry `latest` versions checked: pi **0.85.1**, TypeBox **1.3.27**, TypeScript **7.0.2** and Node typings **26.4.1**. All are pinned for development; `npm outdated` reports no outdated direct dependencies. Both the complete dependency audit and production-only audit report **zero vulnerabilities**. GitHub Actions uses the latest checked checkout **v7.0.1** and setup-node **v7.0.0**, pinned to immutable commits, with audit gates for all severities.

A clean development install still reports the upstream `node-domexception@1.0.0` deprecation notice described in [development guidance](development.md#dependency-security). It is not a security advisory and is absent from production installs. No unsafe transitive override or disabled audit is used to conceal it. The minimum Node version was corrected to **22.19.0** to match pi’s declared engine requirement, and the 15-test suite passed on that exact version.

## npm distribution: 0.1.1

Version 0.1.1 introduces the scoped npm identity `@michaelasper/pi-ssh`; the bash implementation is unchanged from 0.1.0. The README and tutorial now start with npm installation, and the package includes public publish settings plus repository, homepage and issue-tracker metadata. The unscoped npm package is unrelated.

For this distribution, all 15 tests, strict TypeScript checks, both npm audits and the production tarball smoke test passed again. The smoke test handles scoped package names and verifies zero installed production dependencies and no installation warnings. Authentication material is not part of the repository or package.

## Next distribution: 0.1.2

The next version adds the MIT licence, a manifest/discovery regression test and a stage-only OIDC release workflow. All 16 tests, strict TypeScript checks and both npm audits pass. The production smoke test verifies the MIT licence in the 14-file tarball, zero installed production dependencies and no installation warnings.

The workflow separates verification from the OIDC staging job and never approves a release automatically. A verify-only workflow run is not evidence that npm accepted the trusted-publisher configuration; that requires a real staging run followed by maintainer approval. The [release guide](releasing.md) documents this boundary and the catalog requirements.

## Coverage limits

- Real execution was verified from macOS, not Windows. IPv6 syntax is unit-tested; an IPv6 server was not used.
- Interactive authentication, persistent shell state, file synchronisation and remote job supervision are deliberately absent.
- The output tail is bounded by pi; full-output files remain disk-backed and can grow with total command output.
- A public repository or registry release is not created by these checks.