--- name: verify description: Verify that a code change actually does what it's supposed to by exercising it end-to-end and observing behavior — drive the affected flow, not just tests or typecheck. Run before committing nontrivial changes. Don't invoke it on a diff that only touches tests, docs, or other code with no runtime surface to drive (a change to product source always has one) — there's nothing to observe. --- **Verification is runtime observation.** You build the app, run it, drive it to where the changed code executes, and capture what you see. That capture is your evidence. Nothing else is. **Don't run tests. Don't typecheck.** Running them here proves you can run CI — not that the change works. Not as a warm-up, not "just to be sure," not as a regression sweep after. The time goes to running the app instead. **Don't import-and-call.** `import { foo } from './src/...'` then `console.log(foo(x))` is a unit test you wrote. The function did what the function does — you knew that from reading it. The app never ran. Whatever calls `foo` in the real codebase ends at a CLI, a socket, or a window. Go there. ## Find the change The scope is what you're verifying — usually a diff, sometimes just "does X work." In a git repo, establish the full range (a branch may be many commits, or the change may still be uncommitted): ```bash git log --oneline @{u}.. # count commits (if upstream set) git diff @{u}.. --stat # full range, not HEAD~1 git diff origin/HEAD... --stat # no upstream: committed vs base git diff HEAD --stat # uncommitted: working tree vs HEAD ``` State the commit count. Large diff truncating? Redirect to a file then read it. Repo but no diff from any of these → say so, stop. **No repo → the scope is whatever the user named; ask if they didn't.** **The diff is ground truth. Any description is a claim about it.** Read both. If they disagree, that's a finding. ## Surface The surface is where a user — human or programmatic — meets the change. That's where you observe. | Change reaches | Surface | You | |---|---|---| | CLI / TUI | terminal | type the command, capture the pane — see the CLI example below | | Server / API | socket | send the request, capture the response — see the server example below | | GUI | pixels | drive it headless (xvfb / a browser driver), screenshot | | Library | package boundary | sample code through the public export — `import pkg`, not `import ./src/...` | | Prompt / agent config | the agent | run the agent, capture its behavior | | CI workflow | the CI system | dispatch it, read the run | **Internal function? Not a surface.** Something in the repo calls it and that caller ends at one of the rows above. Follow it there. A shell security gate's surface isn't the function's return value — it's the CLI prompting or auto-allowing when you type the command. **No runtime surface at all** — docs-only, type declarations with no emit, build config that produces no behavioral diff — report **SKIP — no runtime surface: (reason).** Don't run tests to fill the space. **Tests in the diff are the author's evidence, not a surface.** CI runs them. You'd be re-running CI. Tests-only change → SKIP, one line. Mixed src+tests → verify the src, ignore the test files. Reading a test to learn what to check is fine — it's a spec. But then go run the app. Checking that assertions match source is code review. ## Get a handle **Check your available skills first — even if you already know how to build and run.** A skill that matches your surface (a verifier or run recipe for this kind of app) is a verified path: follow its setup. A stale recipe (fails on mechanics unrelated to the change) → note it; don't FAIL the change for recipe rot. Also check the repo's own entry points: README, package.json scripts, Makefile, docs. - **A matching launch/verify recipe exists** → use its build/launch primitives as your handle. - **Nothing** → cold start from README/package.json/Makefile. Timebox ~15min. Stuck → BLOCKED with exactly where it stopped. Got through → **persist what you learned** in your report: the exact build/launch/drive commands that worked, the flows worth driving, any gotchas — so the recipe can be captured as a project skill and the next session skips this cold start. ## Drive it Smallest path that makes the changed code execute: - Changed a flag? Run with it. - Changed a handler? Hit that route. - Changed error handling? Trigger the error. - Changed an internal function? Find the CLI command / request / render that reaches it. Run that. **Read your plan back before running.** If every step is build / typecheck / run test file — you've planned a CI rerun, not a verification. Find a step that reaches the surface or report BLOCKED. **The verdict is table stakes. Your observations are the signal.** A PASS with three sharp "hey, I noticed…" lines is worth more than a bare PASS. You're the only reviewer who actually *ran* the thing — anything that made you pause, work around, or go "huh" is information the author doesn't have. Don't filter for "is this a bug." Filter for "would I mention this if they were sitting next to me." **End-to-end, through the real interface.** Pieces passing in isolation doesn't mean the flow works — seams are where bugs hide. If users click buttons, test by clicking buttons, not by curling the API underneath. **Destructive path?** If the change touches code that deletes, publishes, sends, or writes outside the workspace and there's no dry-run or safe target, don't drive it live. Verify what you can around it and say which path you didn't exercise and why. ## Push on it The claim checked out — that's the first half. Confirming is step one, not the job. The description is what the author intended; your value is what they didn't. You know exactly what changed. Probe *around* it, at the same surface you just drove: - **New flag / option** → empty value, passed twice, combined with a conflicting flag, typo'd (does the error name it?) - **New handler / route** → wrong method, malformed body, missing required field, oversized payload - **Changed error path** → the adjacent errors it didn't touch — did the refactor catch them too, or only the one in the diff? - **Interactive / TUI** → Ctrl-C mid-op, resize the pane, paste garbage, rapid-fire the key, Esc at the wrong moment - **State / persistence** → do it twice, do it with stale state underneath, do it in two sessions at once - **Wander** → what's adjacent? What looked off while you were confirming? Go back to it. These aren't a checklist — pick the ones the change points at. Stop when you've covered the obvious adjacents or hit something worth a ⚠️. A probe that finds nothing is still a step: "🔍 passed `--from ''` → clean `error: --from requires a value`, exit 2." That the author didn't test it is exactly why it's worth knowing it holds. Still not a test run. You're at the surface, typing what a user would type wrong. ## Capture Stdout, response bodies, screenshots, pane dumps. Captured output is evidence; your memory isn't. Something unexpected? Don't route around it — capture, note, decide if it's the change or the environment. Unrelated breakage is a finding, not noise. Shared process state (tmux, ports, lockfiles) — isolate. `tmux -L name`, bind `:0`, `mktemp -d`. You share a namespace with your host. ## Report Inline, final message: ``` ## Verification: **Verdict:** PASS | FAIL | BLOCKED | SKIP **Claim:** **Method:** ### Steps Each step is one thing you did to the **running app** and what it showed. Build/install/checkout are setup, not steps. Test runs and typecheck don't belong here — they're CI's output. 1. ✅/❌/⚠️/🔍 → 🔍 marks a probe — a step off the claim's happy path, trying to break it. At least one. A Steps list that's all ✅ and no 🔍 is a happy-path replay: still PASS, but you stopped at the first half. **Screenshot / sample:** ### Findings ``` **Evidence has to reach the reader.** A file path is only evidence if the person reading the report can open it. If the `SendUserFile` tool is in your toolset, you're on a remote surface where they can't — send the screenshots and recordings with it and let the report name what you sent. Without it, reference the path and keep the evidence that matters inline — pane captures and response bodies travel in the report; a bare path only works when the reader shares your filesystem. **Verdicts:** - **PASS** — you ran the app, the change did what it should at its surface. Not: tests pass, builds clean, code looks right. - **FAIL** — you ran it and it doesn't. Or it breaks something else. Or claim and diff disagree materially. - **BLOCKED** — couldn't reach a state where the change is observable. Build broke, env missing a dep, handle wouldn't come up. Not a verdict on the change. Never report an approach blocked or impossible until you've checked the skills and recipes available to you — environment-specific unlocks (headless runners, login helpers, VM harnesses) usually live there. Say exactly where it stopped. - **SKIP** — no runtime surface exists. Docs-only, types-only, tests-only. Nothing went wrong; there's just nothing here to run. One line why. No partial pass. "3 of 4 passed" is FAIL until 4 passes or is explained away. **When in doubt, FAIL.** False PASS ships broken code; false FAIL costs one more human look. Ambiguous output is FAIL with the raw capture attached — don't interpret. ## Appendix: verifying a CLI change The handle is direct invocation. The evidence is stdout/stderr/exit code. 1. Build (if the CLI needs building) 2. Run with arguments that exercise the changed code 3. Capture output and exit code 4. Compare to expected Worked example — **diff:** adds a `--json` flag to the `status` subcommand. **Claim:** "machine-readable status output." **Inference:** `tool status --json` now exists, emits valid JSON with the same fields the human output shows; `tool status` without the flag is unchanged. ```bash go build -o /tmp/tool ./cmd/tool /tmp/tool status # → Status: healthy / Uptime: 3h12m / Connections: 47 /tmp/tool status --json # → {"status":"healthy","uptime_seconds":11520,"connections":47} /tmp/tool status --json | jq -e .status # → "healthy" (jq -e exits nonzero if the path is null/false) echo $? # → 0 ``` **Verdict:** PASS — flag works, JSON is valid, fields line up. What FAIL looks like: `unknown flag: --json` → not wired up, or a stale build; output isn't valid JSON → serialization bug; `tool status` (no flag) changed → regression; JSON field names differ from expected → claim/code mismatch, note it. If the CLI reads stdin → pipe in test data. If it writes files / hits a network / deletes things → point it at a tmp dir / a mock / a dry-run flag. No safe mode and the diff touches the destructive path → say so and verify what you can around it. ## Appendix: verifying a server/API change The handle is `curl` (or equivalent). The evidence is the response. 1. Start the server (background, with a readiness poll) 2. `curl` the route the diff touches, with inputs that hit the changed branch 3. Capture the full response (status + headers + body) 4. Compare to expected Lifecycle, when no launch recipe exists: ```bash &> /tmp/server.log & SERVER_PID=$! for i in {1..30}; do curl -sf localhost:PORT/health >/dev/null && break; sleep 1; done # ... your curls ... kill $SERVER_PID ``` No readiness endpoint? Poll the route you're about to test until it stops returning connection-refused, then add a beat. Worked example — **diff:** adds a `Retry-After` header to 429 responses in `rateLimit.ts`. **Claim:** "clients can now back off correctly." **Inference:** hitting the rate limit should now return `Retry-After: ` in the response headers; it didn't before. ```bash # trigger the limit — 10 fast requests, limit is 5/sec per the diff for i in {1..10}; do curl -s -o /dev/null -w "%{http_code}\n" localhost:3000/api/thing; done # → 200 200 200 200 200 429 429 429 429 429 # capture the 429 headers curl -si localhost:3000/api/thing | head -20 # → HTTP/1.1 429 Too Many Requests # → Retry-After: 12 ``` **Verdict:** PASS — `Retry-After: 12` present, positive integer. What FAIL looks like: header absent → the diff didn't take effect, or you're not actually hitting the 429 path (check the status code first); header present but `NaN` / `undefined` / negative → the logic is wrong; 200s all the way through → you never triggered the changed path — tighten the burst or check the limit config.