# Dino

An Automated Verification agent based on Pi Harness

## What's Inside

| Type | Name | What it does |
|------|------|-------------|
| Extension | `dino` | Replaces the pi TUI header with a Dino-themed ASCII header and provides a `/dino-header` command to restore the built-in header |
| Extension | `playwright-mcp` | Bridges Microsoft Playwright MCP tools into pi.dev so the agent can drive a browser for live page inspection, POM generation, and web verification |
| Extension | `mabl-mcp` | Bridges the mabl MCP server into pi.dev so the agent can query tests, runs, plans, and workspaces via mabl's official CLI using OAuth authentication |
| Skill | `acli` | Interact with Atlassian Cloud (Jira and Confluence) via the `acli` CLI — search work items, manage sprints/boards, create Confluence pages, and more |
| Skill | `bdd-gherkin` | Generate domain-driven Gherkin/Cucumber BDD scenarios focused on business requirements with minimal technical details |
| Skill | `playwright-cli` | Automates browser interactions via `playwright-cli` for terminal-first browser control, navigation, screenshots, tracing, and test generation |
| Skill | `playwright-core` | Battle-tested Playwright patterns for writing and debugging reliable E2E, API, component, visual, accessibility, and security tests |
| Skill | `playwright-pom` | Page Object Model patterns for Playwright — when to use POM, how to structure page objects, and when fixtures or helpers are a better fit |
| Skill | `mabl-mcp` | Guidance for using the mabl MCP bridge — workspace context, running tests, retrieving results, and handling OAuth auth |
| Prompt | `create-pr` | Create a GitHub pull request for changes in the current workspace against a repository URL, installing `gh`, authenticating from an optional env file, and setting up git if needed |
| Prompt | `generate-bdd-from-url` | Generate exhaustive Gherkin BDD scenarios for a single page by inspecting it with `playwright-cli` or Playwright MCP |
| Prompt | `generate-pom-from-url` | Generate Playwright Page Object Models from a live URL using the Playwright MCP bridge (`browser_*` tools) |
| Prompt | `generate-pom-from-url-cli` | Generate Playwright Page Object Models from a live URL using `playwright-cli` commands |
| Prompt | `generate-spec-from-feature` | Generate a Playwright test spec from a Gherkin feature file, reusing existing POMs |
| Prompt | `heal-playwright-failures` | Detect, diagnose, and auto-heal failed Playwright tests by updating selectors, waits, and assertions, then re-run to verify |
| Prompt | `setup-playwright` | Set up a vanilla Playwright project in the current workspace with TypeScript, config, and a starter test |

## AI Evals Skills

These skills help design, run, and validate evaluations for LLM-powered systems.

| Skill | Description |
|-------|-------------|
| `build-review-interface` | Build a custom browser-based annotation interface for reviewing LLM traces and collecting structured human labels. |
| `error-analysis` | Systematically identify and categorize failure modes in an LLM pipeline by reading traces. |
| `eval-audit` | Audit an LLM eval pipeline for problems such as unvalidated judges, vanity metrics, and missing error analysis. |
| `evaluate-rag` | Evaluate retrieval-augmented generation pipelines, including retrieval quality, generation faithfulness, and relevance. |
| `generate-synthetic-data` | Create diverse synthetic test inputs for LLM evaluation using dimension-based tuple generation. |
| `validate-evaluator` | Calibrate an LLM-as-judge against human labels using metrics like TPR/TNR and bias correction. |
| `write-judge-prompt` | Design LLM-as-judge evaluators for subjective criteria that code-based checks cannot handle. |

## Prompts

Dino includes reusable slash-command prompts for common verification workflows.

### Choosing a browser automation prompt

For browser-based tasks, **prefer the `playwright-cli` prompts**. They are more token-efficient because the agent reads compact terminal output instead of full MCP tool-result payloads.

**Use the Playwright MCP prompts only as a fallback** when the `playwright-cli` prompt fails, the page cannot be reached via the CLI, or you need richer live-browser introspection.

| Prompt | What it does | Example usage |
|--------|--------------|---------------|
| `/create-pr` | Create a GitHub pull request for changes in the current workspace against a given repository URL, installing `gh` and authenticating from an optional auth env file. | `/create-pr https://github.com/octocat/repo github-auth.env` |
| `/generate-bdd-from-url` | Inspect a page and generate exhaustive Gherkin BDD scenarios for that page only. | `/generate-bdd-from-url https://example.com tests/features` |
| `/generate-pom-from-url-cli` | Generate Playwright Page Object Models from a live URL using `playwright-cli` (**preferred**). | `/generate-pom-from-url-cli https://example.com tests/pom` |
| `/generate-pom-from-url` | Generate Page Object Models from a live URL using the Playwright MCP bridge (**fallback only**). | `/generate-pom-from-url https://example.com tests/pom` |
| `/generate-spec-from-feature` | Generate a Playwright test spec from a Gherkin feature file, reusing existing POMs. | `/generate-spec-from-feature https://example.com tests/features/login.feature tests/pom tests/specs` |
| `/heal-playwright-failures` | Run the Playwright suite, detect failures, diagnose root causes, apply safe fixes, and re-run to verify. | `/heal-playwright-failures` or `/heal-playwright-failures tests/login.spec.ts 3` |
| `/setup-playwright` | Set up a vanilla Playwright project in the current workspace with TypeScript, config, and a starter test. | `/setup-playwright` |

## Prerequisite

Dino requires the `pi` CLI to be installed. If `pi` is not present, install it globally via npm:

```bash
npm install -g @earendil-works/pi-coding-agent
```

Verify the installation:

```bash
pi --version
```

## Install

Dino is installed **per-project** using the `-l` flag. This records the package entry in the project's `.pi/settings.json` instead of the global `~/.pi/agent/settings.json`, so Dino is only loaded when you start `pi` inside that project.

### Why install locally instead of globally?

We recommend installing Dino as a **local** pi package rather than a global one for two important reasons:

1. **Avoid overwriting other pi agents.** Installing Dino globally would replace the active global `pi` agent, which can overwrite or conflict with existing pi packages. Installing locally keeps Dino scoped to the current project without affecting your global `pi` setup or other agents.

2. **Allow postinstall scripts to run.** Dino relies on npm `postinstall` scripts to set up required runtime dependencies — for example, installing the Playwright browser binaries and writing extension/configuration files needed by the test verification agent. When Dino is installed locally in a project, its own `npm install` runs fully and these postinstall steps execute as intended. Global installation paths or shared package managers may suppress or skip these scripts, leaving browser tooling incomplete.

In short, a local install keeps Dino isolated to the project that needs it, protects your other `pi` agents, and ensures the Playwright-based verification tooling is fully configured.

### Example: install Dino into a `YourTesting` project

Assume you are starting fresh with an empty project folder. The goal is to have this layout:

```
~/work/
├── YourTesting/        # your project
└── dino/                # Dino package clone
```

1. Create your project folder and navigate into it:

   ```bash
   mkdir -p ~/work/YourTesting
   cd ~/work/YourTesting
   ```

2. Clone the Dino repository next to your project folder:

   ```bash
   git clone https://github.com/testdino-hq/dino.git ../dino
   ```

   Your folder structure should now look like:

   ```
   ~/work/
   ├── YourTesting/
   └── dino/
   ```

3. Install Dino's npm dependencies. Pi does **not** run `npm install` automatically for local-path packages, so you must do this once in the Dino directory:

   ```bash
   cd ../dino
   npm install
   cd -  # returns you to ~/work/YourTesting
   ```

   This installs Dino's dependencies (including `@playwright/mcp`, `@playwright/cli`, and the MCP SDK) and runs the package `postinstall` script.

4. Install Dino into the current project only:

   ```bash
   pi install ../dino --approve -l
   ```

   The `-l` flag tells pi to write the package reference to `.pi/settings.json` inside `YourTesting`.

5. Verify the install. You should now have a `.pi/settings.json` file in your project, and Dino will load automatically the next time you start pi from `~/work/YourTesting`:

   ```bash
   pi list
   ```

> **Note:** If you want to try Dino once without persisting it anywhere, use `--extension` (or `-e`) for a temporary install:
> ```bash
> pi -e ../dino
> ```

## Remove

If Dino was installed with the `-l` flag, remove it from the current project's `.pi/settings.json`:

```bash
pi uninstall dino -l
```

Or remove the local path entry manually from `.pi/settings.json`.

## Playwright MCP bridge

Dino can expose the official Microsoft Playwright MCP server (`@playwright/mcp`) as pi.dev tools. This lets the agent navigate, snapshot, click, fill, and otherwise control a browser — useful for live page inspection when generating Page Object Models or verifying web behavior.

### Setup

1. In the Dino directory, install the package dependencies (this pulls in `@playwright/mcp` and the MCP SDK and also installs the Chromium browser via the `postinstall` script):

   ```bash
   cd /path/to/dino
   npm install
   ```

2. If the Chromium binary was not installed by the `postinstall` script, install it manually:

   ```bash
   npx playwright install chromium
   ```

3. Make sure Dino is installed as a project-local pi package so the extension is loaded (run this from your project directory, e.g. `YourTesting`):

   ```bash
   pi install ../dino --approve -l
   ```

   If already installed, reload the package:

   ```bash
   pi update --extension ../dino
   # or just reinstall
   pi install ../dino --approve -l
   ```

4. Start pi in a project where you want to use the browser tools. On startup you should see:

   ```
   [playwright-mcp] Registered 24 Playwright MCP tool(s).
   ```

### Available tools

Once loaded, tools such as these are available to the model:

- `browser_navigate` — navigate to a URL
- `browser_snapshot` — capture accessibility tree of the current page
- `browser_click`, `browser_type`, `browser_fill_form` — interact with elements
- `browser_take_screenshot` — capture a screenshot
- `browser_console_messages`, `browser_network_requests` — inspect page activity

### Using with the `playwright-pom` skill

The `playwright-pom` skill now includes an MCP-driven workflow:

1. User asks for a POM for a page/URL.
2. The agent calls `browser_navigate` to open the page.
3. The agent calls `browser_snapshot` to inspect the accessibility tree.
4. The agent generates a page object using resilient locators (`getByRole`, `getByLabel`, `getByTestId`).

See `skills/playwright-pom/SKILL.md` for the full workflow and example.

## mabl MCP bridge

Dino can expose the official [mabl MCP server](https://help.mabl.com/hc/en-us/articles/47299375773844-mabl-MCP-overview) as pi.dev tools using OAuth authentication. This lets the agent query mabl workspaces, tests, plans, runs, results, and other mabl resources directly from the conversation.

### Setup

1. In the Dino directory, install the package dependencies (this pulls in `@mablhq/mabl-cli` and the MCP SDK):

   ```bash
   cd /path/to/dino
   npm install
   ```

2. Make sure Dino is installed as a project-local pi package so the extension is loaded (run this from your project directory, e.g. `YourTesting`):

   ```bash
   pi install ../dino --approve -l
   ```

   If already installed, reload the package:

   ```bash
   pi update --extension ../dino
   # or just reinstall
   pi install ../dino --approve -l
   ```

3. Start pi in a project where you want to use the mabl tools. The extension no longer runs the OAuth check automatically on startup. To check authentication, run the OAuth flow, and load the mabl MCP tools, use the extension command:

   ```
   /mabl-auth
   ```

   If you are not already authenticated with mabl, the command runs `mabl auth login --auto` to perform the OAuth flow. Follow the on-screen prompts (open the authorization URL in a browser and approve access). After authentication succeeds you should see:

   ```
   [mabl-mcp] Registered N mabl MCP tool(s).
   ```

   > **Note:** `mabl auth login --auto` is designed for headless/agent environments. It captures the OAuth authorization code automatically, but you still need to approve the login in a browser the first time.

### Manual OAuth (if the automatic flow is blocked)

If the automatic OAuth flow cannot complete inside pi (for example, because no browser is available), authenticate outside of pi and then run the command inside pi:

```bash
cd /path/to/dino
npx @mablhq/mabl-cli auth login --auto
# approve the login in your browser, then:
pi start
```

Once pi is running, invoke `/mabl-auth` to load the mabl MCP tools.

### Available tools

Once loaded, the mabl MCP tools exposed by the server become available to the model. Tool names and capabilities depend on the current mabl MCP server version; examples include:

- Querying workspaces, applications, environments, and plans
- Listing tests and test runs
- Retrieving run results and diagnostics

The extension discovers the tool list dynamically from the server, so new tools are automatically available after updating `@mablhq/mabl-cli`.

### Authentication note

The extension stores no credentials itself. It relies on the mabl CLI's own OAuth token storage (managed by `@mablhq/mabl-cli`). To sign out, run:

```bash
npx @mablhq/mabl-cli auth clear
```

### Command reference

- `/mabl-auth` — Check mabl OAuth status, run `mabl auth login --auto` if needed, and start the mabl MCP server so the model can use the mabl tools.

## Dino API Server

Dino ships with a Fastify API server that exposes every prompt workflow as REST endpoints. This lets you trigger Dino prompts from CI pipelines, web UIs, or any HTTP client.

### Quick start (fresh system)

```bash
# Windows — double-click or run:
scripts\setup-dino-api.bat

# macOS / Linux — run:
chmod +x scripts/setup-dino-api.sh
./scripts/setup-dino-api.sh
```

This single script:
1. Checks that Node.js is installed
2. Installs `pi` globally if missing
3. Installs `@assertrx/dino` via `pi install` if missing
4. Starts the API server on port 4003

### Manual setup

```bash
# 1. Install pi (if not already installed)
npm install -g @earendil-works/pi-coding-agent

# 2. Install Dino via pi
pi install npm:@assertrx/dino --approve

# 3. Set an API key (any supported provider works)
export ANTHROPIC_API_KEY=sk-ant-...
# e.g. for OpenAI instead: export OPENAI_API_KEY=sk-...

# 4. Start the server
node ~/.pi/agent/npm/node_modules/@assertrx/dino/scripts/api-server.mjs
```

### Endpoints

| Method | Path | Description |
|--------|------|-------------|
| `GET` | `/api/health` | Health check |
| `GET` | `/api/providers` | List all providers with auth status and model counts |
| `GET` | `/api/models` | List models available with current credentials (`?provider=<id>` to filter) |
| `POST` | `/api/models` | List models for given credentials — body accepts `provider`+`apiKey` or `apiKeys` (validates the key and unlocks that provider's catalog) |
| `GET` | `/api/prompts` | List all available prompts with parameters |
| `GET` | `/api/prompts/:name` | Get raw markdown content of a prompt |
| `POST` | `/api/prompts/:name` | **Execute** a prompt via the pi agent |

### Providers, models & API keys

The server supports every pi provider (Anthropic, OpenAI, Google, DeepSeek, Groq, Mistral, xAI, OpenRouter, and many more). Credentials can come from three places, in this order of precedence:

1. **Per-request keys** — pass `apiKeys` (a map of provider → key) or `apiKey` (+ optional `provider`) in the POST body. These keys are scoped to that single request's session.
2. **Environment variables** — any provider's standard env var (e.g. `ANTHROPIC_API_KEY`, `OPENAI_API_KEY`, `GEMINI_API_KEY`).
3. **Stored credentials** — logins created via `pi /login` in `~/.pi/agent/auth.json`.

Use `GET /api/providers` to discover which providers are configured and `GET /api/models` to list models available to the server right now.

**Selecting a model:** pass `model` in the POST body as either `"provider/model-id"` (e.g. `"openai/gpt-5.2"`) or `{ "provider": "openai", "id": "gpt-5.2" }`. Without `model`, the session uses the default model resolution (configured provider defaults). Requesting a model whose provider has no credentials returns a clear error.

### Prompt execution endpoints

All POST bodies accept auth/model options, `cwd` (workspace directory, defaults to current), and `stream` (default: `true` for SSE):

| Body param | Description |
|------------|-------------|
| `apiKeys` | Map of provider → API key, e.g. `{"anthropic": "sk-...", "openai": "sk-..."}` |
| `apiKey` | Single API key. Pair with `provider` to target one explicitly; otherwise applied to the first available provider (anthropic fallback) |
| `provider` | Provider id (e.g. `"openai"`) to pair with a single `apiKey` |
| `model` | Model selection: `"provider/model-id"`, `"model-id"`, or `{"provider": ..., "id": ...}` |

| Prompt name | Required params |
|-------------|---------------|
| `setup-playwright` | _none_ |
| `create-pr` | _none_ (optional: `repoUrl`, `authEnvFile`) |
| `generate-pom-from-url` | `url` |
| `generate-pom-from-url-cli` | `url` |
| `generate-bdd-from-url` | `url` |
| `generate-spec-from-feature` | `url`, `featureFile`, `pomDir` |
| `heal-playwright-failures` | _none_ (optional: `testPattern`, `maxIterations`) |

### Example requests

```bash
# List all prompts
curl http://localhost:4003/api/prompts

# Discover providers and their auth status
curl http://localhost:4003/api/providers

# List models available to the server (optionally per provider)
curl http://localhost:4003/api/models
curl "http://localhost:4003/api/models?provider=anthropic"

# Execute a prompt (SSE streaming, auth from env vars)
curl -X POST http://localhost:4003/api/prompts/setup-playwright \
  -H "Content-Type: application/json" \
  -d '{"cwd": "/path/to/workspace"}'

# Execute with a single provider key + model selection
# (env: export OPENAI_API_KEY=sk-... then:)
curl -X POST http://localhost:4003/api/prompts/setup-playwright \
  -H "Content-Type: application/json" \
  -d '{"cwd": "/path/to/workspace", "model": "openai/gpt-5.2"}'

# Execute with multiple per-request API keys and an explicit model
curl -X POST http://localhost:4003/api/prompts/setup-playwright \
  -H "Content-Type: application/json" \
  -d '{
    "cwd": "/path/to/workspace",
    "apiKeys": {"anthropic": "sk-ant-...", "openai": "sk-ov-..."},
    "model": "anthropic/claude-sonnet-4-5"
  }'

# Execute with URL parameter (non-streaming JSON response)
curl -X POST http://localhost:4003/api/prompts/generate-pom-from-url-cli \
  -H "Content-Type: application/json" \
  -d '{"url": "https://example.com", "outputDir": "tests/pom", "stream": false}'
```

### Response formats

**SSE streaming** (`stream: true`, default):
```
event: start      → {"prompt": "..."}
event: text       → {"delta": "..."}        # response tokens
event: thinking   → {"delta": "..."}        # thinking output
event: tool_start → {"tool": "bash"}        # tool execution started
event: tool_end   → {"tool": "bash", ...}   # tool execution finished
event: done       → {}                      # session complete
event: error      → {"message": "..."}      # error
```

**Non-streaming** (`stream: false`):
```json
{
  "prompt": "setup-playwright",
  "response": "...full agent response text...",
  "toolCalls": [{"name": "bash", "status": "success"}]
}
```

### Environment variables

| Variable | Default | Description |
|----------|---------|-------------|
| `PORT` | `4003` | Server port |
| `HOST` | `0.0.0.0` | Server host |
| Provider env vars | — | Any provider's standard env var (e.g. `ANTHROPIC_API_KEY`, `OPENAI_API_KEY`, `GEMINI_API_KEY`); per-request `apiKeys`/`apiKey` override these |