# ElevenLabs Agents Integration

A complete toolkit for integrating ElevenLabs Conversational AI agents with Claude Code and other AI assistants. This repository contains both a Claude Code skill and an MCP server for seamless voice AI interactions.

## Overview

This repository provides two complementary components:

1. **Claude Code Skill** (`skill/`) - Pre-built skill definition for Claude Code with usage examples and API reference
2. **MCP Server** (`mcp-server/`) - Model Context Protocol server that runs on Vercel as a serverless function

Together, these enable AI assistants to interact with ElevenLabs voice agents, initiate conversations, make outbound phone calls, and retrieve conversation transcripts.

## Quick Start

### For Claude Code Users

Install the skill in your Claude Code configuration:

```bash
# Copy skill to Claude Code skills directory
cp -r skill ~/.claude/skills/elevenlabs-agents
```

Then add the MCP server to your Claude Code MCP configuration:

```json
{
  "mcpServers": {
    "elevenlabs-agents": {
      "url": "https://your-app.vercel.app/mcp",
      "transport": "http"
    }
  }
}
```

### For Developers

Deploy the MCP server to Vercel:

```bash
cd mcp-server
npm install
vercel --prod
```

**Important**: Set `ELEVENLABS_API_KEY` as a Vercel environment variable in your project settings (not in `.env` files).

## Features

### Agent Management
- **List Agents**: Get all available ElevenLabs agents in your account
- **Get Agent Details**: Retrieve detailed information about specific agents

### Conversation Capabilities
- **Initiate Agent Calls**: Start conversations with agents and wait for completion (default 50s timeout, configurable up to 300s)
- **Outbound Phone Calls**: Make phone calls to any number using an agent
- **Status Tracking**: Monitor call progress and completion
- **Transcript Retrieval**: Get full conversation transcripts with metadata

### Call Outcomes
The system handles multiple call scenarios:
- **Completed**: Successful conversations with full transcripts
- **Failed**: Calls that encountered errors
- **No Answer**: Calls that were initiated but never answered
- **In Progress**: Active or processing calls

## Repository Structure

```
elevenlabs-agents/
├── README.md                    # This file
├── CLAUDE.md                    # Project instructions for Claude Code
├── skill/                       # Claude Code skill files
│   ├── SKILL.md                 # Skill definition and usage guide
│   ├── examples.md              # Comprehensive usage examples
│   └── reference.md             # API reference documentation
└── mcp-server/                  # MCP server implementation
    ├── README.md                # Detailed MCP server documentation
    ├── api/                     # Vercel serverless functions
    ├── lib/                     # Core implementation
    │   ├── elevenlabs/          # ElevenLabs API client
    │   └── mcp/                 # MCP server logic
    ├── test/                    # Test scripts
    ├── package.json
    ├── tsconfig.json
    └── vercel.json              # Vercel configuration
```

## Prerequisites

- **ElevenLabs API Key**: Get one from [ElevenLabs](https://elevenlabs.io/api)
- **Vercel Account**: For deploying the MCP server
- **Node.js 18+**: For local development
- **(Optional) Phone Numbers**: For outbound calling features

## Setup Guide

### 1. Get Your ElevenLabs API Key

1. Sign up at [ElevenLabs](https://elevenlabs.io)
2. Navigate to your API settings
3. Generate or copy your API key

### 2. Deploy MCP Server to Vercel

```bash
cd mcp-server
npm install

# Install Vercel CLI if needed
npm i -g vercel

# Login to Vercel
vercel login

# Deploy to production
vercel --prod
```

### 3. Configure Environment Variables in Vercel

**Critical**: Set environment variables in Vercel dashboard, not in `.env` files:

1. Go to your Vercel project settings
2. Navigate to "Environment Variables"
3. Add: `ELEVENLABS_API_KEY` = `your_elevenlabs_api_key_here`
4. Apply to Production, Preview, and Development environments

**Why Vercel Environment Variables?**
- Secure: Never committed to version control
- Environment-specific: Different keys for dev/staging/prod
- Automatic: Available to all serverless functions
- Standard practice: Vercel's recommended approach

### 4. Add to Claude Code

Once deployed, add the MCP server URL to your Claude Code configuration:

```json
{
  "mcpServers": {
    "elevenlabs-agents": {
      "url": "https://your-app.vercel.app/mcp",
      "transport": "http"
    }
  }
}
```

## Available MCP Tools

### Agent Management
- `list_agents` - Get all agents in your account
- `get_agent` - Get details for a specific agent

### Conversations
- `initiate_agent_call` - Start a conversation and wait for completion
  - Default timeout: 50 seconds (configurable up to 300s)
  - Returns full transcript on completion

### Outbound Calling
- `list_phone_numbers` - Get available phone numbers for outbound calls
- `initiate_outbound_call` - Make a phone call to any number
  - Phone format: E.164 (e.g., +15551234567)
  - Returns conversation_id for tracking

### Status & Transcripts
- `get_call_status` - Check current status of a conversation
  - Statuses: initiated, in-progress, processing, done, failed
- `get_call_transcript` - Retrieve full transcript of any finalized call
  - Works for completed, failed, and no-answer calls
  - Includes conversation turns, timestamps, costs, and metadata

## Usage Examples

### Quick Agent Test

```typescript
// List agents
const agents = await useMCP('elevenlabs-agents', 'list_agents');

// Test an agent
const result = await useMCP('elevenlabs-agents', 'initiate_agent_call', {
  agent_id: 'agent_xxx',
  initial_message: 'Hello, can you tell me about your services?'
});

console.log(result.transcript);
```

### Outbound Phone Call

```typescript
// Check available phone numbers
const numbers = await useMCP('elevenlabs-agents', 'list_phone_numbers');

// Initiate call
const call = await useMCP('elevenlabs-agents', 'initiate_outbound_call', {
  agent_id: 'agent_xxx',
  to_number: '+15551234567'
});

// Poll for status
const status = await useMCP('elevenlabs-agents', 'get_call_status', {
  conversation_id: call.conversation_id
});

// Get transcript when done
if (status.call_outcome === 'completed') {
  const transcript = await useMCP('elevenlabs-agents', 'get_call_transcript', {
    conversation_id: call.conversation_id
  });
}
```

## Common Workflows

### Development Workflow
1. Make changes to `mcp-server/` code
2. Test locally: `cd mcp-server && npm run dev`
3. Deploy preview: `vercel`
4. Test in Claude Code with preview URL
5. Deploy production: `vercel --prod`

### Testing Agent Changes
1. Update agent configuration in ElevenLabs dashboard
2. Use `list_agents` to verify changes
3. Test with `initiate_agent_call` for quick validation
4. Monitor transcripts for quality assurance

## Architecture

### Serverless Design
- **HTTP Transport**: Runs as Vercel serverless function
- **Stateless**: No persistent connections or state
- **Timeout-Aware**: Default 50s, max 300s (Vercel limit)
- **Retry Logic**: Automatic retry with exponential backoff for API calls

### Polling Strategy
- Starts at 2-second intervals
- Exponential backoff (1.5x multiplier)
- Max interval: 10 seconds
- Continues polling on transient errors

### Error Handling
- Network errors: Retry up to 3 times
- API errors: Formatted with status code and message
- Tool errors: Wrapped in MCP error format

## Local Development

### Run MCP Server Locally

```bash
cd mcp-server

# Set environment variable
export ELEVENLABS_API_KEY=your_api_key

# Run with stdio transport (for testing)
node lib/mcp/server.ts

# Or run with Vercel dev server (HTTP transport)
npm run dev
```

Then test at `http://localhost:3000/api/mcp`

### Run Tests

```bash
cd mcp-server
npm test
```

## Troubleshooting

### "Authentication failed" errors
- Verify `ELEVENLABS_API_KEY` is set in Vercel environment variables (not `.env`)
- Check API key is valid in ElevenLabs dashboard
- Ensure key has proper permissions

### Conversation timeouts
- Default timeout is 50s; increase via `timeout` parameter if needed
- Maximum timeout is 300s (Vercel serverless limit)
- For longer conversations, use outbound calling with status polling
- Check Vercel function logs for errors

### "No phone numbers available"
- Verify phone numbers are configured in ElevenLabs account
- Check phone number permissions and status
- Use `list_phone_numbers` to see available numbers

### Outbound calls not connecting
- Ensure phone number is in E.164 format (+15551234567)
- Check recipient phone can receive calls
- Verify Twilio integration is active in ElevenLabs

## Security Best Practices

- **Never commit API keys**: Use Vercel environment variables only
- **CORS configured**: Properly set up for MCP client access
- **Environment isolation**: Separate keys for dev/staging/prod
- **Regular rotation**: Update API keys periodically

## Documentation

### Skill Documentation
- [`skill/SKILL.md`](skill/SKILL.md) - Skill definition and usage guide
- [`skill/examples.md`](skill/examples.md) - Comprehensive usage examples
- [`skill/reference.md`](skill/reference.md) - API reference documentation

### MCP Server Documentation
- [`mcp-server/README.md`](mcp-server/README.md) - Detailed server documentation
- [`CLAUDE.md`](CLAUDE.md) - Project instructions for Claude Code

### External Resources
- [ElevenLabs API Documentation](https://docs.elevenlabs.io)
- [Model Context Protocol](https://modelcontextprotocol.com)
- [Vercel Documentation](https://vercel.com/docs)

## Contributing

Contributions welcome! Please ensure:

1. TypeScript types are properly defined
2. Error handling is comprehensive
3. Tests pass: `cd mcp-server && npm test`
4. Documentation is updated
5. Environment variables are documented

## Limitations

- **Timeout**: Max 300 seconds (5 minutes) per request due to Vercel limits
- **Stateless**: No background job processing or persistent state
- **Synchronous**: All operations complete within single request lifecycle

For conversations exceeding 5 minutes, use the outbound calling pattern with status polling.

## License

MIT

## Support

- **Issues**: [GitHub Issues](https://github.com/taskcrew/elevenlabs-agents/issues)
- **ElevenLabs**: [ElevenLabs Documentation](https://docs.elevenlabs.io)
- **MCP Protocol**: [MCP Documentation](https://modelcontextprotocol.com)
- **Vercel**: [Vercel Documentation](https://vercel.com/docs)