# /zibby-test-debug — diagnose a failing Zibby test You are helping the user figure out why a test failed. Canonical docs: **https://docs.zibby.app/tests/debugging** ## Diagnostic recipe Apply in order. Stop at the first thing that explains the symptom. ### 1. Was the failure in the AGENT phase or the PLAYWRIGHT phase? Two places things go wrong: - **Agent phase** (`generate_script` / `execute_live`) — the AI couldn't figure out what to do or what selector to use. Look for `MCP error`, `tool timeout`, or generic LLM gibberish in the output. - **Playwright phase** (after script generation) — script ran but an assertion failed or a selector didn't match. Look for `expect(...)` failures, `Timeout` waiting for selector, or "element not visible". ### 2. Open the artifacts Each run produces: - `test-results//video.webm` — watch what happened - `test-results//trace.zip` — Playwright trace; open with `npx playwright show-trace ` - `.zibby/output/sessions//` — agent's internal state, prompts, responses - `tests/.spec.js` — the generated Playwright code The video usually tells you in 30 seconds what's wrong. ### 3. Check the spec Spec ambiguity is the most common cause. If the spec says "click the button" and there are five buttons, the agent picks one — possibly the wrong one. Re-read the spec asking: would a stranger know exactly which element this means? ### 4. Re-run with more verbosity ``` Bash(zibby test test-specs/.txt --verbose) # info-level logs Bash(zibby test test-specs/.txt --debug) # all logs, lots Bash(zibby test test-specs/.txt) # default is headed — drop --headless to watch the browser ``` ### 5. Re-execute one node from a prior session If `execute_live` failed but the script generation was OK, re-execute just that node against the existing session: ``` Bash(zibby test --node execute_live --session last) ``` ### 6. Common errors and fixes | Symptom | Likely cause | Fix | |---------|-------------|-----| | "ZIBBY_API_KEY required" | not authenticated | `Bash(zibby login)` | | "MCP server not responding" | playwright-mcp config drifted | `Bash(zibby setup-playwright)` | | "Selector not found" | UI changed since last run | re-generate from the spec, or update selector in `.spec.js` | | Agent loops forever in `execute_live` | spec is too vague | tighten the spec; add the explicit selector text | | "Module not found" | missing dep in repo | `Bash(npm install)` in the repo root | ### 7. When to escalate If the same spec passed yesterday and fails today, and the codebase didn't change → check if Zibby pushed an agent update (`zibby --version`). Otherwise it's almost always spec ambiguity or a real product regression.