--- name: investigate description: | Systematic debugging with root cause investigation. Iron Laws: no fix without root cause, no diagnosis without evidence, investigation produces a task — not a fix. Use when asked to "debug this", "fix this bug", "why is this broken", "investigate this error", or "root cause". allowed-tools: - Bash - Read - Edit - Grep - Glob - AskUserQuestion - WebSearch --- # Systematic Debugging Invoked by the admin agent directly. ## Iron Laws **NO FIXES WITHOUT ROOT CAUSE INVESTIGATION FIRST.** Fixing symptoms creates whack-a-mole debugging. Every fix that doesn't address root cause makes the next bug harder to find. **NO DIAGNOSIS WITHOUT EVIDENCE. NO GUESSING.** 100% of the diagnosis must be grounded in observable evidence — log output, error messages, stack traces, reproduction results, or code-path tracing. Inconclusive evidence is not a mystery; it is an **observability failure**. Fix the observability gap first, reproduce with proper instrumentation, then resume. **INVESTIGATION PRODUCES A TASK, NOT A FIX.** The default output is a task entry describing the root cause, the fix, its verification strategy, and its observability requirements. Do not implement fixes directly unless the user explicitly asks. `Edit` is in `allowed-tools` only for the temporary instrumentation in Phase 3 step 1. Investigation does not write source files; the deliverable is a task entry, written through the resolved task surface. --- ## Step 0: Evidence First — prerequisites before any reasoning **Do this before resolving the task surface, searching prior work, reading code, or forming any hypothesis.** The failure mode this guards against is forming a hypothesis from the conversation context — the error text the user pasted, what you assume the code does — and then treating evidence retrieval as redundant. By the time you have read the rest of this skill you have already started doing it. Stop. First action in this skill: create a TodoWrite list of the prerequisites below. Work through them in order. Do not read code, do not search prior tasks, do not propose a cause while any prerequisite is still open. 1. **Retrieve the requested evidence.** When the user provides a specific artifact — a session key, log reference, error excerpt, or pointer to a particular event — that artifact is the primary evidence. Retrieve it with the canonical log-retrieval tool for the surface (e.g. `logs-read` for Pi sessions on deployed maxy). If a retrieval hook injects an exact command, execute it as given; do not improvise. This todo may not be marked complete until the **actual retrieved output appears in the transcript** — the real log lines, not a summary, not a paraphrase, not "the logs confirm X." If you cannot produce the output, the todo stays open and that itself is the finding: an observability or retrieval gap (see Phase 1 step 6). 2. **If no specific artifact was provided**, mark item 1 N/A and proceed to Phase 1 — but state explicitly that no evidence artifact was supplied, so the investigation starts from code-path tracing. --- ## Task surface (where tasks live in this environment) Before searching prior work or writing the deliverable, resolve where tasks live in the current install. Check in this order; use the first one that exists: 1. **Graph tasks (deployed maxy).** If the `work-*` MCP tools are available (`work-list`, `work-get` for prior-work search; `work-create` to write the deliverable — they persist `:Task` nodes in Neo4j), tasks live in the graph. Use those tools. (The plugin is `work`, not `tasks` — renamed to avoid colliding with Claude Code's built-in `Task*` family.) 2. **Local `.tasks/` directory.** If the repo has a `.tasks/` directory (optionally with `archive/`, `in-progress/`, `parked/`, `backlog/` subdirs and a `LANES.md`), tasks live on disk as `.tasks/NNN-.md`. 3. **Neither.** State this explicitly: "No task surface detected — this install has no `work-*` graph tools and no `.tasks/` directory. The deliverable will be a written report only." Throughout this skill, "search prior tasks", "cite prior work", and "write a task" mean: use the resolved surface. Do not invent a new location. ## Phase 1: Root Cause Investigation 1. **Check prior work.** Search the task surface (graph or `.tasks/`) for tasks related to the affected area — by component, filename, or symptom keywords. Read any matches. Prior tasks carry root cause analyses, evidence, scope boundaries, and observability findings. What to look for: - Same root cause already identified — fix may be in progress or parked. Do not re-investigate. - Related symptoms in the same area — recurring bugs are an architectural smell, not coincidence. - Prior observability findings — earlier gaps may still be open, which explains sparse evidence now. - Scope boundaries from related tasks — what was deferred may be exactly what's failing. Cite prior work: ``` PRIOR WORK: — RELEVANCE: [same root cause | same area | related symptom] STATUS: [open | archived | parked — and whether findings still apply] ``` If no prior work is found, note it explicitly — that itself is context. 2. **Collect symptoms.** Read error messages, stack traces, reproduction steps. If context is missing, ask ONE question at a time via AskUserQuestion. 3. **Read the code.** Trace the code path from symptom back to potential causes. Grep for references, Read to understand logic. 4. **Check recent changes.** ```bash git log --oneline -20 -- <affected-files> ``` A regression means the root cause is in the diff. 5. **Reproduce.** Can you trigger the bug deterministically? If not, gather more evidence before proceeding. 6. **Assess observability.** Can the logs, error messages, and metrics answer "what happened and why?" If not, this is the first problem to solve. Flag the gap: ``` OBSERVABILITY GAP: [what evidence is missing and why it matters] INSTRUMENTATION NEEDED: [what would make the cause visible] ``` If the gap is significant, **stop the investigation** and write a task for the observability fix. Resume after instrumentation is deployed and the issue is reproduced with proper evidence. **Test for the no-event blind spot explicitly.** Transition-only logging (a line per action) is structurally blind to any failure that emits *no event and does not reproduce on demand* — a leak, an orphan, a stuck or idle resource, a silently-failed cleanup, slow drift. There is no action to hang a log on, and re-running the code will not surface it. For **every failure mode you can name**, ask the forcing question: *"what signal reveals it, without reproducing the failure and without waiting for unrelated future activity?"* If the answer is "none until something else happens," the missing instrumentation is a **standing check — an independent periodic audit that reconciles expected state against actual** — not another action log, and that gap is itself the finding. Verify post-conditions, never log intentions: "called the cleanup" is not evidence the resource was freed. Output: **"Root cause hypothesis: ..."** — a specific, testable claim supported by cited evidence, plus the narrowest module the fix would touch (informational; investigation does not edit source). --- ## Phase 2: Pattern Analysis | Pattern | Signature | Where to look | |---------|-----------|---------------| | Race condition | Intermittent, timing-dependent | Concurrent access to shared state | | Nil/null propagation | NoMethodError, TypeError | Missing guards on optional values | | State corruption | Inconsistent data, partial updates | Transactions, callbacks, hooks | | Integration failure | Timeout, unexpected response | External API calls, service boundaries | | Configuration drift | Works locally, fails in staging/prod | Env vars, feature flags, DB state | | Stale cache | Shows old data, fixes on cache clear | Redis, CDN, browser cache | Also check: - Prior tasks (graph or `.tasks/archive/`) in the same area — recurring bugs in the same files are an architectural smell. - `git log` for prior fixes in the same area. **External pattern search.** If no known pattern matches, WebSearch for "{framework} {generic error type}" or "{library} {component} known issues". **Sanitize first** — strip hostnames, IPs, file paths, SQL, customer data. Search the error category, not the raw message. If WebSearch is unavailable, skip and proceed. --- ## Phase 3: Hypothesis Testing 1. **Confirm the hypothesis.** Add a temporary log/assertion/debug output at the suspected root cause (this is the one Edit investigation makes; remove it before delivering findings). Run the reproduction. Does the evidence match? 2. **If the hypothesis is wrong.** Sanitize and search the generic error (or skip if WebSearch unavailable), then return to Phase 1. Gather more evidence. Do not guess. 3. **3-strike rule.** If 3 hypotheses fail, **STOP**. Three failed hypotheses almost always means insufficient observability — guessing, not investigating. Use AskUserQuestion: ``` 3 hypotheses tested, none confirmed by evidence. This is most likely an observability failure. OBSERVABILITY GAP: [what evidence is missing] A) Fix observability first — add instrumentation, reproduce, then resume investigation (RECOMMENDED) B) Continue investigating — new hypothesis with supporting evidence: [describe the evidence, not just the guess] C) Escalate for human review — needs domain knowledge I don't have ``` Option A is almost always correct. B requires citing specific evidence, not speculation. **Red flags** — slow down if you see: - "Quick fix for now" — there is no "for now." - Proposing a fix before tracing data flow — guessing. - Each fix reveals a new problem elsewhere — wrong layer. - "I think it's probably X" — you have an observability gap. - Jumping to implementation without asking — investigation produces findings, not fixes. - "The logs don't show anything useful" — that IS the finding. --- ## Phase 4: Deliver Findings Once root cause is confirmed with evidence, the **default output is a task entry on the resolved surface** — not a code fix. ``` ROOT CAUSE CONFIRMED: [one-line summary with evidence citation] Evidence: [log output, reproduction result, code trace] A) Create a task on the resolved surface (graph: work-create; .tasks/: write .tasks/<N>-<slug>.md) with root cause, fix scope, verification strategy, and observability requirements (RECOMMENDED) B) Fix it now — implement the fix in this session (only if the user explicitly asked for a fix) C) Just the report — document findings, no task, no fix ``` A is the default. B requires explicit user approval. If no task surface exists, A degenerates into C. ### If the user chooses B Investigation is over; this is surgery. Scope discipline is the sprint skill's problem from here. 1. Fix the root cause, not the symptom — smallest change that eliminates the actual problem. 2. Minimal diff — fewest files, fewest lines. 3. Regression test that **fails without** the fix and **passes with** it. 4. Run the full test suite. Paste the output. No regressions. 5. If the fix touches >5 files, AskUserQuestion on blast radius: ``` Fix touches N files — large blast radius for a bug fix. A) Proceed — root cause genuinely spans these files B) Split — fix critical path now, defer the rest C) Rethink — maybe a more targeted approach exists ``` --- ## Phase 5: Verification & Report Reproduce the original bug scenario and confirm it's fixed. Run the test suite. Output: ``` DEBUG REPORT ════════════════════════════════════════ Symptom: [what the user observed] Root cause: [what was actually wrong — cite the evidence] Evidence: [specific log lines, reproduction output, traces] Observability: [sufficient? if not, what gap was found / added] Deliverable: [task created on <surface> | fix applied | report only] Fix (if applied): [what changed, with file:line refs] Regression test: [file:line of new test, or N/A] Related: [prior tasks, prior bugs, architectural notes] Status: DONE | DONE_WITH_CONCERNS | BLOCKED ════════════════════════════════════════ ``` --- ## Important Rules - **Inconclusive evidence = observability failure.** Fix the gap first. - **Default deliverable is a task, not a fix.** Only implement if the user explicitly said "fix it now." - **3+ failed hypotheses → STOP.** Almost certainly observability. - **Never apply a fix you cannot verify.** - **Never say "this should fix it."** Verify and prove it. - **Fix touches >5 files → AskUserQuestion** on blast radius. - **Completion status:** - DONE — root cause confirmed with evidence; task created (or fix applied if requested) - DONE_WITH_CONCERNS — root cause identified, evidence partial; observability gap flagged for follow-up - BLOCKED — root cause unclear due to insufficient observability; instrumentation task created