# Vibe Test — Complete Reference for LLMs

## Overview

Vibe Test is a code-aware browser testing agent distributed as an npm package (`vibe-testing`, CLI command: `vibe-test`). It provides an MCP server with 13 tools that let AI code editors (Claude Code, Cursor, Windsurf, VS Code Copilot, Roo Code) autonomously test web applications using Playwright.

The core philosophy: **the editor LLM is the brain, Vibe Test is the hands.** Each tool returns structured JSON data plus screenshots as base64 images, allowing the LLM to see the page and reason about it visually. No internal LLM — pass/fail is determined by heuristics (URL changes, toasts, API errors).

## Installation

```bash
# One-command setup for all editors (recommended)
npx vibe-testing@latest init

# As MCP server (add to editor's mcp.json)
{ "command": "npx", "args": ["-y", "vibe-testing@latest", "--mcp"] }

# As CLI
npx vibe-testing@latest run https://your-app.com --codebase /path/to/project
npx vibe-testing@latest converge https://your-app.com
```

## Complete Tool Reference (13 Tools)

### 1. scan_codebase
**Must be called first.** Analyzes the project source code:
- Detects framework (Next.js App/Pages, React SPA, SvelteKit, Nuxt, Vue, Express, Fastify)
- Detects monorepos (Turborepo, pnpm, yarn, Lerna) and finds frontend app
- Parses all routes (file-based and code-defined, including src/ variants)
- Extracts forms, buttons, dialogs, CRUD operations, search/filter features per route
- Reads existing Playwright/Jest/Cypress/Vitest test files
- Maps test coverage and identifies gaps
- Generates test scenarios for uncovered features
- Writes `.vibe/route-manifest.json` and diffs against the previous scan

Parameters:
- `codebase_path` (string, optional): Absolute path to project root. Defaults to cwd.
- `url` (string, optional): App URL to test against.
- `mode` (string, optional): "fast" or "deep". Default "deep".

Returns: JSON with routes, behaviours, coverage map, gaps, generated scenarios, and `route_changes: { new_routes, removed_routes } | null` — `null` on first scan, populated when routes differ from the previous scan. Lets the editor surface "new routes added since last test — want to cover them?"

### 2. get_context
**Call before writing test steps.** Returns the most relevant source files for a feature or route with relevance scoring — so the LLM knows real field names, selectors, and API endpoints instead of guessing.

Parameters:
- `feature` (string): Feature name, route path, or description (e.g., "login", "/checkout", "user profile form")

Returns: JSON with relevant source file contents (budget-capped), matched routes, and extracted selectors.

### 3. login
Authenticates in a real Playwright browser:
- Navigates to login page
- Fills email/password using cascading selectors (name, type, placeholder, aria-label, fallback to first text input)
- Clicks submit
- Waits for redirect
- Captures localStorage tokens and cookies
- Returns screenshot of post-login state
- Supports HTTP Basic Auth via `basic` strategy

Parameters:
- `email` (string): Login email/username
- `password` (string): Login password
- `login_url` (string, optional): Login page path. Default "/login".

Returns: JSON with success status, final URL, tokens found, API calls observed + screenshot.

### 4. scan_page_elements
Discovers all interactive elements on a page without clicking them:
- Buttons, links, inputs, selects, checkboxes, textareas, tabs
- For each: tag, type, text, selector, disabled state, href

Parameters:
- `route` (string): Page route to scan
- `authenticated` (boolean, optional): Use authenticated context. Default false.

Returns: JSON element list + page screenshot.

### 5. explore_page
Full interactive exploration — acts like a senior QA tester:
- Clicks every button and observes state changes
- Tests every link and verifies navigation
- Fills inputs with appropriate test data
- Monitors all API calls (excluding static assets)
- Records success/failure for each interaction
- Respects action blocklist (never_interact patterns)

Parameters:
- `route` (string): Page route to explore
- `authenticated` (boolean, optional): Use authenticated context. Default false.

Returns: JSON with interaction results, API calls, timings + screenshot.

### 6. execute_scenario
Runs a specific multi-step test scenario:
- Actions: navigate, fill, click, select, wait, assert, upload
- Step-by-step execution with screenshots after each state change
- Heuristic verification of outcomes (URL changes, toasts, API errors)
- Reuses existing browser context (fast)

Parameters:
- `scenario` (object): Scenario with id, name, route, steps[], expected_outcome, requires_auth
- `authenticated` (boolean, optional): Default false.

Step format:
```json
{ "action": "fill", "selector": "[name='email']", "value": "test@example.com", "description": "Fill email" }
```

Returns: JSON with status (pass/fail/error), step logs, screenshots, API errors.

### 7. get_coverage
Returns test coverage analysis from the last scan_codebase call:
- Routes with test counts and test frameworks used
- Coverage gaps with priority levels (high/medium/low)
- Available scenarios to run

Returns: JSON with coverage map, gaps, suggested scenarios.

### 8. suggest_tests
Intelligent gap analysis that returns executable scenarios:
- Untested CRUD operations (critical priority)
- Missing form validation tests (high priority)
- Broken elements from exploration (high priority)
- Previously failing routes for retest (high priority)
- API errors found during exploration (critical/high)
- Untested navigation flows (low priority)

Parameters:
- `route` (string, optional): Focus on specific route. Omit for all routes.

Returns: JSON with prioritized suggestions, each containing an executable scenario object.

### 9. take_screenshot
Quick visual verification:
- Navigates to URL, waits for page load, captures screenshot

Parameters:
- `url` (string): Full URL or route path
- `authenticated` (boolean, optional): Default false.
- `full_page` (boolean, optional): Full page or viewport. Default false.

Returns: Screenshot as base64 data URI.

### 10. generate_report
Generates interactive HTML report:
- All test results with pass/fail status
- Step-level screenshots with click-to-enlarge
- Page exploration findings
- API monitoring results
- Coverage gap suggestions
- Auto-opens in browser

Parameters:
- `title` (string, optional): Report title.

Returns: Report path + text summary.

### 11. run_full_test
All-in-one orchestration: scan_codebase → generate scenarios → execute all → explore pages → generate report. Writes `.vibe/run-snapshot.json` and diffs against the previous run.

Parameters:
- `url` (string): App URL
- `codebase_path` (string, optional): Project root path
- `mode` (string, optional): "fast" or "deep"
- `headed` (boolean, optional): Show browser. Default `false` when invoked via the MCP server (so editor sessions aren't disrupted), `true` from the CLI.

Returns: Full results + report path + `snapshot_diff` (see below).

### 12. run_converge
Iterative testing until coverage thresholds are met:
- Runs baseline suite first
- Automatically runs follow-up rounds from coverage gaps and failed scenarios
- Stops when pass rate and gap thresholds are met, or max rounds reached
- Writes `.vibe/run-snapshot.json` and diffs against the previous run

Parameters:
- `url` (string): App URL
- `codebase_path` (string, optional): Project root path
- `max_followup_rounds` (number, optional): Max follow-up rounds. Default 4.
- `target_pass_rate` (number, optional): Stop when pass rate >= this (0-1). Default 0.92.
- `max_high_severity_gaps` (number, optional): Stop when critical+important gaps <= this. Default 2.

Returns: Full results across all rounds + final report + `snapshot_diff` (see below).

### 13. cleanup
Closes all open browsers and resets session state. Call when done testing.

## Recommended Workflow

```
1. scan_codebase   → understand routes, forms, coverage gaps
2. get_context     → read actual source code for the feature (real selectors)
3. login           → authenticate if app requires it
4. explore_page    → discover elements and interactions visually
5. execute_scenario → run targeted test steps with real selectors
6. generate_report → produce HTML report with screenshots
7. cleanup         → close browsers
```

## Auth Strategies

vibe.config.json supports three auth strategies:
- `"credentials"` — form-based login (fills email/password fields, clicks submit)
- `"basic"` — HTTP Basic Auth (sets credentials on browser context for preview-gated sites)
- `"skip"` — no authentication

## VIBE.md Project Guidance

Projects can include a `VIBE.md` file to guide testing:

```markdown
## Login URL
/login

## Test Credentials
- Email: test@example.com
- Password: TestPass123!

## Never Automate
- delete account
- cancel subscription

## Known Flaky
- /notifications

## Notes
- Profile page requires clicking "Edit Profile" first
```

## Self-Improvement

Vibe Test persists intelligence in `.vibe/`:
- Working selectors per route
- Route load timings and timeout adjustments
- Auth state and credentials (reused across runs)
- Failure patterns and flaky route tracking
- Run history with pass rate trends
- `route-manifest.json` — snapshot of all routes from the last scan, diffed on the next scan
- `run-snapshot.json` — per-route pass/fail status from the last run, diffed on the next run

Each subsequent run uses this intelligence to be faster and more accurate.

## Snapshot Diff (Regression Detection)

Every call to `run`, `converge`, `run_full_test`, and `run_converge` includes a `snapshot_diff` field (or `null` on the first run):

```json
{
  "snapshot_diff": {
    "previous_run_id": "run-1716000000000",
    "current_run_id": "run-1716086400000",
    "newly_passing": ["/login"],
    "newly_failing": ["/checkout"],
    "still_failing": ["/admin/billing"],
    "new_routes": ["/admin/users"],
    "removed_routes": []
  }
}
```

- `newly_passing` — routes that were failing in the previous run and now pass (regressions fixed)
- `newly_failing` — routes that were passing in the previous run and now fail (new regressions)
- `still_failing` — routes failing in both runs (persistent issues)
- `new_routes` — routes present this run that weren't tested previously
- `removed_routes` — routes tested previously that weren't tested this run

Editors should highlight `newly_failing` as the highest-priority finding — those are regressions the user just introduced.

## Route Manifest Diff (Scaffold Detection)

`scan_codebase` returns `route_changes` (or `null` on first scan):

```json
{
  "route_changes": {
    "new_routes": ["/admin/users", "/billing/invoices"],
    "removed_routes": ["/old-dashboard"]
  }
}
```

Use this to proactively suggest tests for routes the user just added.

## Supported Frameworks

| Framework | Routes | API | Forms |
|-----------|--------|-----|-------|
| Next.js App Router | Yes | Yes | Yes |
| Next.js Pages Router | Yes | Yes | Yes |
| Next.js src/ variant | Yes | Yes | Yes |
| React SPA (react-router) | Yes | — | Yes |
| SvelteKit | Yes | Yes | Yes |
| Nuxt | Yes | Yes | Yes |
| Vue + Vite | Yes | — | Yes |
| Express / Fastify | — | Yes | Yes |
| Monorepos | Yes | Yes | Yes |

## Configuration Reference

`vibe.config.json` keys:

| Key | Type | Default | Notes |
|-----|------|---------|-------|
| `url` | string | required | App URL |
| `mode` | `"fast"` \| `"deep"` | `"deep"` | Scan depth |
| `routes` | `"auto"` \| `"config"` | `"auto"` | `auto` discovers routes from the codebase; `config` uses only routes listed in config |
| `auth.strategy` | `"credentials"` \| `"basic"` \| `"skip"` | — | Auth mode |
| `auth.login_url` | string | `/login` | Login page path |
| `auth.credentials` | `{ email, password }` | — | Persisted across runs |
| `scope.include` | string[] | `["/**"]` | Route glob include list |
| `scope.exclude` | string[] | `[]` | Route glob exclude list |
| `scope.max_routes` | number | `30` | Cap on routes tested per run |
| `never_interact` | string[] | `[]` | Text or selectors to skip during exploration |
| `browser.headed` | boolean | CLI `true`, MCP `false` | Visible vs headless browser |
| `browser.slowMo` | number | `40` | ms between actions |
| `browser.timeout` | number | `30000` | Default action timeout (ms) |

## Runtime

- **Node.js** ≥ 20 (vitest 4.x requires Node 20+)
- **Playwright Chromium** (auto-installed on first run if missing)
- **Dockerfile** — Node 20 + Chromium image included; ENTRYPOINT runs MCP server over stdio

## Links

- GitHub: https://github.com/AishwaryShrivastav/vibe-testing
- npm: https://www.npmjs.com/package/vibe-testing
- MCP: https://modelcontextprotocol.io