# ⚠️ DEPRECATED — This package is obsolete

**Use [pi-multimodal-proxy](../pi-multimodal-proxy) instead.** As of v1.5.0, pi-multimodal-proxy natively handles **video and audio files** by routing them to a multimodal model (e.g. Grok 4.3, Gemini 2.5 Pro) that can transcribe speech, identify speakers, describe visual scenes, read on-screen text, and reason about the content — all in a single unified call.

No extra installation needed. If you already have pi-multimodal-proxy installed, video/audio support is automatic.

### Migration

1. **Uninstall** this package: `pi uninstall pi-video-transcribe`
2. **Ensure** pi-multimodal-proxy ≥ 1.5.0 is installed: `pi install npm:pi-multimodal-proxy`
3. **Set** the video model (if not using the default): `/multimodal-proxy video-model x-ai/grok-4.3`
4. **Done.** Just reference a video/audio file in your prompt — it works automatically.

### Why the switch?

| | pi-video-transcribe (this package) | pi-multimodal-proxy v1.5.0 |
|---|---|---|
| **Visual scene analysis** | ❌ No visual processing | ✅ Describes scenes, objects, people, actions |
| **OCR / text in video** | ❌ Cannot see visuals | ✅ Reads text, diagrams, charts on screen |
| **Speech transcription** | ✅ AssemblyAI (specialized) | ✅ Via frontier multimodal model |
| **Speaker diarization** | ✅ Word-level speaker IDs | ✅ Word-level speaker IDs |
| **Reasoning about content** | ❌ Only reasons about spoken words | ✅ Conversational Q&A about everything it sees and hears |
| **Extra API key needed** | ✅ AssemblyAI API key required | ❌ Uses your existing model provider key |
| **Extra dependency** | ✅ ffmpeg for local video | ❌ No ffmpeg needed |
| **Pricing** | ~$0.13/20min + LLM Gateway tokens | Included in model token costs (often cheaper) |

> **Still need AssemblyAI?** If you require PII redaction, content moderation, specialized 99+ language models, compliance SLAs, or streaming real-time STT with turn detection, AssemblyAI remains the better choice. This package will remain available for those niche use cases, but is no longer recommended for general video understanding.

---

# pi-video-transcribe *(legacy)*

Video transcription with **speaker diarization** for [pi](https://pi.dev). Transcribe any video or audio file and get per-speaker labeled text, summaries, and chapter breakdowns.

Powered by [AssemblyAI](https://www.assemblyai.com/) (Universal-3 Pro model) with summaries and chapters via LLM Gateway.

## Features

- **Speaker diarization** — "who said what" with timestamps
- **Summary** — bullet-point summary via LLM Gateway (Claude Sonnet)
- **Chapter breakdown** — logical chapters with headlines and summaries via LLM Gateway
- **99+ languages** — auto-detected or specify manually
- **URLs & local files** — HTTP(S) URLs or local file paths
- **30+ video/audio formats** — mp4, mkv, avi, mov, webm, mp3, wav, m4a, flac, ogg, and more
- **Skill included** — guides pi to build custom AssemblyAI integrations (streaming, voice agents, etc.)

## Prerequisites

### 1. AssemblyAI API Key

**Step-by-step:**

1. **Sign up** at [assemblyai.com/dashboard/signup](https://www.assemblyai.com/dashboard/signup)
2. **Copy your API key** from the dashboard (shown right after login under "Your API Key")
3. **Set it as environment variable:**

**Bash / ZSH** (add to `~/.bashrc` or `~/.zshrc`):
```bash
export ASSEMBLYAI_API_KEY=your_api_key_here
```

**PowerShell** (run once, persists across restarts):
```powershell
[Environment]::SetEnvironmentVariable("ASSEMBLYAI_API_KEY", "your_api_key_here", "User")
```

**.env file** (if you use dotenv in your project):
```
ASSEMBLYAI_API_KEY=your_api_key_here
```

> **Free tier:** 5 hours of transcription per month — enough to get started. No credit card required.

### 2. ffmpeg (for local video files)

Required only when transcribing local video files (mp4, mkv, avi, etc.). Not needed for URLs or audio-only files (mp3, wav — AssemblyAI handles those directly).

**If ffmpeg is missing, the tool will print platform-specific install instructions.**

| Platform | Install |
|----------|---------|
| **Windows** | `winget install Gyan.FFmpeg` |
| **macOS** | `brew install ffmpeg` |
| **Linux (Debian/Ubuntu)** | `sudo apt install ffmpeg` |
| **Linux (Fedora)** | `sudo dnf install ffmpeg` |
| **Linux (Arch)** | `sudo pacman -S ffmpeg` |

Verify: `ffmpeg -version`

## Installation

```bash
# From local path
pi install ./path/to/pi-video-transcribe

# From npm (once published)
pi install npm:pi-video-transcribe

# From git
pi install git:github.com/YOUR_USERNAME/pi-video-transcribe
```

After installation, the `transcribe_video` tool is available to the LLM automatically.

## Usage

Just ask pi to transcribe a video:

```
> Transcribe ./meeting.mp4 and tell me what each person said

> What are the main topics discussed in this video? ./interview.mp4

> Summarize ./presentation.mp4 per speaker

> Transcribe https://storage.example.com/recording.mp3 with speaker labels
```

### How it works

1. pi recognizes the intent and calls the `transcribe_video` tool
2. For local video files, audio is extracted via ffmpeg
3. Audio is sent to AssemblyAI for transcription + diarization
4. You get back a structured result with per-speaker transcript, summary, chapters

### Parameters (LLM uses these automatically)

| Parameter | Default | Description |
|-----------|---------|-------------|
| `source` | *(required)* | Local file path or URL |
| `speakers_expected` | auto | Exact speaker count (only if certain) |
| `speaker_min` / `speaker_max` | auto | Speaker count range |
| `include_summary` | `true` | Generate bullet-point summary via LLM Gateway |
| `include_chapters` | `true` | Generate chapter breakdown via LLM Gateway |
| `language_code` | auto | BCP-47 code (`de`, `en`, `fr`, ...) |

### Example output

```
# Video Transcription

- **Duration:** 15:42
- **Language:** de

## Speakers

1. **Speaker A**
2. **Speaker B**

## Transcript

**Speaker A** [0:00 – 2:15]:
> Willkommen zu heutiger Diskussion über...

**Speaker B** [2:15 – 5:30]:
> Vielen Dank für die Einladung...

## Summary

• Die Diskussion behandelt die Einführung neuer Richtlinien
• Speaker A betont die Bedeutung von Transparenz
• Speaker B schlägt einen schrittweisen Ansatz vor
...

## Chapters

### Einleitung und Begrüßung [0:00 – 2:15]
Die Moderatorin eröffnet die Diskussion...

### Hauptthema: Neue Richtlinien [2:15 – 10:45]
Ausführliche Diskussion der vorgeschlagenen Änderungen...
```

## Pricing

AssemblyAI charges per minute of audio. LLM Gateway usage is billed separately by token.

| Feature | Cost |
|---------|------|
| Transcription (Universal-3 Pro) | ~$0.0065/min |
| Speaker Diarization | included |
| Summary via LLM Gateway | token-based |
| Chapters via LLM Gateway | token-based |

A **20-minute video** with transcription and diarization costs roughly **$0.13**. LLM Gateway adds a few cents for summary and chapters.

## Troubleshooting

| Issue | Fix |
|-------|-----|
| "Set ASSEMBLYAI_API_KEY" warning | Export the env variable and restart pi |
| ffmpeg not found | Install ffmpeg (see table above) and restart your terminal |
| File not found | Use absolute or relative-to-CWD paths |
| Transcription failed | Check your API key and AssemblyAI dashboard quota |
| Summary/Chapters failed | LLM Gateway error -- check AssemblyAI dashboard for LLM Gateway access |
| EU data residency | Set `ASSEMBLYAI_EU=true` env var to use EU endpoints |

## License

MIT