# @runhuman/mcp-server

[![npm version](https://img.shields.io/npm/v/@runhuman/mcp-server.svg)](https://www.npmjs.com/package/@runhuman/mcp-server)
[![License: ISC](https://img.shields.io/badge/License-ISC-blue.svg)](https://opensource.org/licenses/ISC)
[![MCP](https://img.shields.io/badge/MCP-1.0-green.svg)](https://modelcontextprotocol.io/)

A [Model Context Protocol (MCP)](https://modelcontextprotocol.io/) server that enables AI agents to orchestrate on-demand human QA testing through the [Runhuman](https://runhuman.com) service.

> **Note:** This is the MCP server package. For the main CLI tool, see [`runhuman`](https://www.npmjs.com/package/runhuman) (coming soon).

## What is this?

This MCP server allows AI agents (like Claude, GPT, or any MCP-compatible AI) to request human testing of web applications. The AI defines what to test and what data format it needs back, and real humans perform the testing with structured results extracted automatically.

**Perfect for:**
- AI coding agents that need human verification of their work
- Automated workflows requiring human-in-the-loop testing
- CI/CD pipelines with human QA gates
- Visual/UX testing that requires real human judgment

## Quick Start

### Claude Desktop

1. Get your API key at [runhuman.com/dashboard](https://runhuman.com/dashboard)

2. Add to your Claude Desktop config:

**macOS:** `~/Library/Application Support/Claude/claude_desktop_config.json`
**Windows:** `%APPDATA%\Claude\claude_desktop_config.json`

```json
{
  "mcpServers": {
    "runhuman": {
      "command": "npx",
      "args": ["-y", "@runhuman/mcp-server", "--api-key=rh_xxxxxxxxxxxxx"]
    }
  }
}
```

3. Restart Claude Desktop

That's it! Claude can now request human testing:

> "Can someone test my checkout page at https://myapp.com/checkout and verify the payment flow works?"

### Other MCP Clients

For other MCP clients (VS Code extensions, custom agents, etc.):

```bash
# Install globally
npm install -g @runhuman/mcp-server

# Run with your API key
runhuman-mcp --api-key=rh_xxxxxxxxxxxxx
```

Or use npx without installation:

```bash
npx @runhuman/mcp-server --api-key=rh_xxxxxxxxxxxxx
```

## Features

### Available Tools

The MCP server exposes a comprehensive set of tools for orchestrating human QA testing. The five most commonly used tools are documented in detail below; the rest are summarized in the [Additional Tools](#additional-tools) section. The full reference (with input schemas and examples) lives at [runhuman.com/docs/mcp](https://runhuman.com/docs/mcp).

#### `create_job`
Create a new human QA testing job with custom instructions.

**Parameters:**
- `url` (string, optional): The URL to test
- `description` (string, required if no template): Natural language instructions for the human tester
- `template` (string, optional): Template name to use as base configuration
- `outputSchema` (object, optional): JSON schema defining the structured output format
- `targetDurationMinutes` (number, optional): Time limit for tester (default: 30, range: 1-60)
- Plus additional options (screen size, GitHub repo, etc.)

**Example:**
```typescript
{
  url: "https://myapp.com/checkout",
  description: "Test the checkout flow: add item to cart, proceed to checkout, and verify payment options are displayed correctly.",
  outputSchema: {
    type: "object",
    properties: {
      checkout_loaded: { type: "boolean" },
      payment_options_visible: { type: "boolean" },
      any_errors: { type: "string" }
    }
  },
  targetDurationMinutes: 30
}
```

**Returns:** `jobId` for tracking the job

#### `run_template`
**⭐ Templates are important!** Create a job from a pre-configured template.

Templates let you reuse test configurations without writing full descriptions every time. Perfect for common test scenarios.

**Parameters:**
- `template` (string, required): Template name (get from `list_templates`)
- Any parameter from `create_job` to override template defaults

**Example:**
```typescript
{
  template: "checkout-flow",
  url: "https://staging.myapp.com"  // Override template's default URL
}
```

**Returns:** `jobId` for the created job

#### `wait`
**Idiomatic polling** - Wait for a job to complete and return results automatically.

Polls job status every 10 seconds until completion, timeout, or failure. Blocks until terminal state is reached (just like CLI `wait` command).

**Parameters:**
- `jobId` (string, required): Job ID from `create_job` or `run_template`
- `timeoutSeconds` (number, optional): Maximum wait time (default: 600, max: 3600)

**Returns when complete:**
- `result`: Structured test results matching your schema
- `status`: "completed"
- `costUsd`: Exact cost in USD
- `testDurationSeconds`: Time spent by tester
- `testerResponse`: Raw natural language feedback
- `testerName`/`testerAlias`: Tester identification
- Full job details

**Example workflow:**
```typescript
// Create job
const job = await create_job({
  url: "https://myapp.com",
  description: "Test the login page"
});

// Wait automatically polls until completion
const result = await wait({ jobId: job.jobId });
// Returns immediately when done - no manual polling needed!
```

#### `get_job`
Get current job status without waiting (quick status check).

Use this for:
- Checking if a job is still running
- Getting quick updates
- Manual polling with your own timing
- Checking multiple jobs in parallel

**Parameters:**
- `jobId` (string, required): Job ID to check

**Returns:** Current job status and details (if completed, includes results)

#### `list_templates`
List available templates for your project.

**Parameters:**
- `limit` (number, optional): Max templates to return (default: 50)

**Returns:** Array of templates with names, URLs, and descriptions

### Additional Tools

These cover job lifecycle, discovery, member management, artifact retrieval, tester knowledge-base notes, enhanced-video rendering, and the full template / schedule CRUD surface. See [runhuman.com/docs/mcp](https://runhuman.com/docs/mcp) for input schemas, example payloads, and role/permission requirements.

**Job lifecycle (beyond the five above):**
- `rerun_job` — Re-dispatch an existing job (#5933), reusing all its settings; override individual fields (`url`, `commitSha`, `description`, `targetDurationMinutes`, `deviceClass`, `outputSchema`). Returns a new `jobId` — follow up with `wait`. The canonical use is re-pointing the same test at a fresh build via `url`.

**Discovery:**
- `list_organizations` — List orgs the authenticated user belongs to (no params).
- `list_projects` — List accessible projects, optionally filtered by `organizationId`.

**Organization member management:**
- `list_organization_members` — List members of an org with role / status / owner flag / join date. Params: `organizationId`.
- `invite_organization_member` — Invite by email. Roles: `admin` / `contributor` / `viewer` (default `contributor`). Requires admin permission.

**Artifact retrieval (paid-tier gated, except `recording`):**
- `get_test_artifacts` — Fetch rich session data for a completed job. Artifact types: `structured-output`, `transcript`, `console-logs`, `network-requests`, `conversation`, `events`, `key-moments`, `recording`. Use after `wait` / `get_job` to investigate why a test passed or failed. `wait` / `get_job` now report a `Recording:` readiness line so you know when the `recording` artifact is fetchable.

**Template CRUD (beyond `list_templates`):**
- `create_template` — Create a reusable template (params: `projectId`, `name`, optional `testDescription`, `url`, `outputSchema`, `targetDurationMinutes`, `deviceClass`, GitHub repos, `autoCreateGithubIssues`).
- `update_template` — Update any field on an existing template.
- `delete_template` — Permanently delete a template (existing schedules using it will fail on next execution).

**Schedule management:**
- `list_schedules` — List recurring schedules for a project.
- `create_schedule` — Create a recurring schedule. Required: `projectId`, `name`, `templateId`, `scheduledHour`, `scheduledMinute`, `timezone`. Frequencies: `daily` / `weekly` / `biweekly` / `monthly` / `once`.
- `update_schedule` — Pause/resume (`status`), retime, swap templates, or change frequency.
- `delete_schedule` — Permanently delete (use `update_schedule` with `status: "paused"` to temporarily stop without deleting).

**Tester knowledge-base notes:**
- `list_tester_notes` — Search/list tester-facing notes (credentials, setup, known issues, …). Filter by `organizationId`, `projectId`, `search`, `tags`.
- `create_tester_note` — Add a note (params: `title`, `body`, `organizationId`, optional `projectId`, `tags`).
- `update_tester_note` — Edit an existing note.
- `delete_tester_note` — Archive (soft-delete) a note.

**Enhanced video:**
- `render_baked_demo_video` — Trigger the enhanced (baked) demo-video render for a completed job and, by default, wait for it to finish. Returns the video URL and a public share path.

## Configuration

### API URL (Optional)

By default, the server connects to the production API at `https://runhuman.com`. For local development or testing:

```json
{
  "mcpServers": {
    "runhuman": {
      "command": "npx",
      "args": [
        "-y",
        "@runhuman/mcp-server",
        "--api-key=rh_xxxxx",
        "--api-url=http://localhost:3000"
      ]
    }
  }
}
```

### Environment Variables

Alternatively, use environment variables:

```bash
export RUNHUMAN_API_KEY=rh_xxxxxxxxxxxxx
export RUNHUMAN_API_URL=https://runhuman.com  # optional
```

## Pricing

Runhuman uses pay-per-second pricing:
- **$0.0085 per second** of tester time
- Duration is rounded up to the nearest second
- Example: 220 seconds of testing = $1.87

Costs are calculated exactly and returned in the API response. No monthly fees, no minimums.

## How It Works

1. **AI creates job** - Your AI agent calls `create_job` or `run_template` with test instructions
2. **Job posted to testers** - Request goes to the human tester pool via Slack
3. **Human performs test** - A real person tests the application (video/screenshots recorded)
4. **Human reports findings** - Tester describes what they observed in natural language
5. **AI extracts data** - GPT-4o processes feedback into structured JSON matching your schema
6. **AI gets results** - `wait` automatically polls and returns clean, typed data ready for automation

## Example Use Cases

### AI Coding Agents
```
User: "Can you update my checkout page and verify it works?"
Agent: [Updates code, deploys, then uses run_template to verify with checkout template]
Agent: "✅ Updated and verified by human tester. Payment flow works correctly."
```

### CI/CD Pipelines
```yaml
- name: Human QA Gate
  run: |
    # Deploy preview
    # Call Runhuman API via MCP
    # Fail build if test fails
```

### Visual Testing
```
Agent: "Please verify the new homepage design looks good on mobile"
[Human tester provides UX feedback]
Agent: "Human feedback: Design looks great, minor spacing issue noted..."
```

## Development

### Local Development

```bash
# Clone the repo
git clone https://github.com/volter-ai/runhuman.git
cd runhuman

# Install dependencies
npm install

# Build the MCP server
npm run build --workspace=@runhuman/mcp-server

# Run with local API
cd packages/mcp-server
node dist/index.js --api-key=rh_test_key_123 --api-url=http://localhost:3000
```

### Testing

```bash
# Run tests
npm run test --workspace=@runhuman/mcp-server

# Use MCP Inspector (interactive testing)
npm run test:inspector --workspace=@runhuman/mcp-server
```

### Project Structure

```
packages/mcp-server/
├── src/
│   ├── index.ts                 # CLI entry point (stdio)
│   ├── mcp-server-factory.ts    # Core server factory
│   ├── lib.ts                   # Library exports
│   ├── types.ts                 # TypeScript types
│   └── tools/                   # Tool implementations (one *.tool.ts per tool)
│       ├── create-job.tool.ts
│       ├── create-job.schema.ts # create_job schema + contract-drift guard
│       ├── rerun-job.tool.ts
│       ├── wait.tool.ts
│       ├── get-job.tool.ts
│       └── …                     # discovery, templates, schedules, notes, artifacts, video
├── dist/                        # Compiled output
├── tests/                       # Test suite
├── docs/                        # Developer documentation
├── package.json
├── tsconfig.json
└── README.md                    # This file
```

## Documentation

- [Model Context Protocol](https://modelcontextprotocol.io/) - Learn about MCP
- [Runhuman Website](https://runhuman.com) - Product information
- [API Documentation](https://runhuman.com/docs) - REST API reference
- [GitHub Repository](https://github.com/volter-ai/runhuman) - Full source code

### Developer Docs

- [How Agents Use MCP](docs/HOW-AGENTS-USE-MCP.md) - Understanding MCP tool discovery
- [Tool Response Best Practices](docs/TOOL-RESPONSE-BEST-PRACTICES.md) - Writing effective tool responses
- [API Authentication](docs/API-AUTHENTICATION.md) - Authentication details

## Integration Examples

### Claude Desktop Usage

Once installed, Claude can naturally use the tools:

**User:** "Test my app at https://staging.myapp.com and check if the login works"

**Claude:**
```
I'll create a QA job to test your login page.
[Calls create_job tool]
Job created! Waiting for a human tester...
[Calls wait tool - automatically polls until complete]
✅ Test complete! The login page works correctly. The tester was able to log in successfully with test credentials.
```

### Programmatic Usage (Custom MCP Client)

```typescript
import { createMcpServer } from '@runhuman/mcp-server/factory';

const server = createMcpServer({
  mode: 'direct',
  apiUrl: 'https://runhuman.com',
  apiKey: 'rh_xxxxx'
});

// Use with your MCP client
await server.connect(transport);
```

## Troubleshooting

### "Error: API key is required"

Make sure you've:
1. Created an account at [runhuman.com](https://runhuman.com)
2. Generated an API key in the dashboard
3. Added the key to your config with the correct format (`--api-key=rh_...`)

### "Invalid API key format"

API keys must start with:
- `rh_` for production
- `rh_` for testing

Get a valid key at [runhuman.com/dashboard](https://runhuman.com/dashboard)

### Connection Issues

If you're having trouble connecting:

1. Check your API URL is correct (default: `https://runhuman.com`)
2. Verify your network connection
3. Test the API directly:
```bash
curl https://runhuman.com/api/jobs \
  -H "Authorization: Bearer rh_xxxxx" \
  -H "Content-Type: application/json" \
  -d '{"url":"https://example.com","description":"test","outputSchema":{}}'
```

### Claude Desktop Not Detecting Server

1. Verify config file location is correct for your OS
2. Check JSON syntax is valid (use a JSON validator)
3. Restart Claude Desktop completely
4. Check Claude's logs for errors

## Contributing

This package is part of the [Runhuman monorepo](https://github.com/volter-ai/runhuman). Contributions welcome!

```bash
# Clone the full repo
git clone https://github.com/volter-ai/runhuman.git
cd runhuman

# Install dependencies
npm install

# Make changes to packages/mcp-server/

# Build and test
npm run build --workspace=@runhuman/mcp-server
npm run test --workspace=@runhuman/mcp-server

# Submit PR
```

## Support

- **Email:** hey@runhuman.com
- **Issues:** [GitHub Issues](https://github.com/volter-ai/runhuman/issues)
- **Website:** [runhuman.com](https://runhuman.com)

## License

ISC License - See [LICENSE](../../LICENSE) for details

## About Runhuman

Runhuman is an AI-orchestrated human QA testing service. Part of the [Volter AI](https://volter.ai) ecosystem.

**Key Features:**
- On-demand human testing (no hiring/managing)
- AI-powered orchestration and result extraction
- Pay-per-second pricing ($0.0085/sec)
- Custom output schemas for structured results
- Browser automation with video/screenshot recording
- GitHub Actions integration
- REST API and MCP server for any workflow

---

**Built with ❤️ by the Volter AI team**