README โข โ๏ธ License
๐จ๐ณ ไธญๆ โข ๐ฏ๐ต ๆฅๆฌ่ช โข ๐ฐ๐ท ํ๊ตญ์ด โข ๐ช๐ธ Espaรฑol โข ๐ฉ๐ช Deutsch โข ๐ซ๐ท Franรงais โข ๐ง๐ท Portuguรชs โข ๐ท๐บ ะ ัััะบะธะน โข ๐ธ๐ฆ ุงูุนุฑุจูุฉ โข ๐ฎ๐น Italiano โข ๐ณ๐ฑ Nederlands โข ๐น๐ท Tรผrkรงe โข ๐ป๐ณ Tiแบฟng Viแปt โข ๐ฎ๐ณ เคนเคฟเคจเฅเคฆเฅ
Quick Start โข How It Works โข MCP Tools โข Transcription โข Configuration โข Troubleshooting โข License
Popcorn enables AI agents to watch and understand long-form videos by extracting transcripts, detecting scene changes, and returning key frames. Everything runs locallyโno external APIs, no fees, complete privacy.
--- ## Quick Start ```bash # Install FFmpeg (required) brew install ffmpeg # macOS sudo apt install ffmpeg # Ubuntu/Debian # Install Popcorn git clone https://github.com/anthropics/popcorn.git cd popcorn && npm install && npm run build # Optional: Install a transcription backend pip install mlx-whisper # Apple Silicon (fastest) pip install openai-whisper # Any platform ``` Add to Claude Desktop config (`~/Library/Application Support/Claude/claude_desktop_config.json`): ```json { "mcpServers": { "popcorn": { "command": "node", "args": ["/path/to/popcorn/dist/index.js"] } } } ``` **Key Features:** - **Scene Detection** โ Captures frames at visual transitions, not fixed intervals - **Local Transcription** โ 4 backend options (mlx-whisper, faster-whisper, whisper-cpp, whisper) - **Inline Images** โ Returns key frames directly in MCP responses - **Smart Presets** โ Auto-configures for screencasts, presentations, movies, interviews - **Zero Config** โ Just pass a video path and it works - **Privacy First** โ Everything runs locally, no data leaves your machine --- ## Documentation ### Getting Started - **[Quick Start](#quick-start)** โ Installation & setup - **[Tutorial](docs/tutorial.md)** โ Step-by-step usage guide - **[MCP Tools](#mcp-tools)** โ Available tools reference ### Guides - **[Transcription Backends](#transcription-backends)** โ Choose the best backend for your system - **[Video Types & Objectives](#video-types)** โ Presets for different content - **[Configuration](#configuration)** โ Advanced parameters ### Reference - **[Troubleshooting](docs/troubleshooting.md)** โ Common issues & solutions - **[Agent Skill](skills/popcorn-video-analysis/SKILL.md)** โ Instructions for AI agents - **[API Reference](#mcp-tools)** โ Tool schemas & responses --- ## How It Works **Core Components:** 1. **FFprobe** โ Extracts video metadata (duration, resolution, codecs) 2. **FFmpeg Scene Detection** โ Finds visual transitions using `select='gt(scene,N)'` filter 3. **Parallel Frame Extraction** โ Captures JPEGs at scene change timestamps 4. **Multi-Backend Transcription** โ Whisper variants convert audio to timestamped text 5. **Analysis Bundle** โ Results saved to `.popcorn/` directory 6. **MCP Response** โ Returns metadata + inline base64 images ``` Video File โโโถ FFprobe โโโถ FFmpeg โโโถ Whisper โโโถ Analysis Bundle โ โ โ โ โผ โผ โผ โผ metadata frames transcript MCP Response ``` --- ## MCP Tools | Tool | Description | |------|-------------| | `popcorn_analyze` | Main analysis โ extracts frames, transcribes audio, returns results | | `popcorn_suggest` | Probe video metadata and get recommended settings | | `popcorn_presets` | List available video types and objectives | | `popcorn_backends` | Detect your system and show transcription options | | `popcorn_read` | Read transcript slices with time filtering | ### Basic Usage ```json { "tool": "popcorn_analyze", "arguments": { "path": "/path/to/video.mp4" } } ``` ### With Presets ```json { "tool": "popcorn_analyze", "arguments": { "path": "/path/to/video.mp4", "videoType": "screencast", "objective": "detailed" } } ``` --- ## Video Types | Type | Best For | Scene Detection | |------|----------|-----------------| | `screencast` | Tutorials, coding sessions, UI demos | Low threshold | | `presentation` | Slides, lectures, keynotes | Slide transitions | | `movie` | Films, TV shows | Balanced | | `interview` | Podcasts, talking heads | Transcription priority | | `surveillance` | Security footage, dashcam | High threshold | | `sports` | Live events, fast action | High frame rate | ## Objectives | Objective | Use When | |-----------|----------| | `summary` | Quick overview needed | | `detailed` | Don't miss anything | | `find_moment` | Searching for specific content | | `transcribe` | Audio/speech is most important | | `visual_only` | Only care about visuals | | `quick_scan` | Fast preview needed | --- ## Transcription Backends Popcorn auto-detects your system and recommends the best backend. ### Backend Comparison | Backend | Speed | Best For | Install | |---------|-------|----------|---------| | **mlx-whisper** | Fastest | Apple Silicon (M1/M2/M3/M4) | `pip install mlx-whisper` | | **faster-whisper** | Fast | NVIDIA GPUs | `pip install faster-whisper` | | **whisper-cpp** | Moderate | Cross-platform | `brew install whisper-cpp` | | **whisper** | Slow | Most compatible | `pip install openai-whisper` | ### Processing Times (60-min video) | Backend | Time | |---------|------| | mlx-whisper | 3-8 min | | faster-whisper | 5-10 min | | whisper-cpp | 10-20 min | | whisper | 30-60 min | ### Force a Backend ```json { "tool": "popcorn_analyze", "arguments": { "path": "/path/to/video.mp4", "backend": "mlx-whisper" } } ``` --- ## Configuration ### All Parameters | Parameter | Type | Description | |-----------|------|-------------| | `path` | string | **Required.** Absolute path to video file | | `videoType` | string | Video type preset | | `objective` | string | Analysis objective preset | | `transcribe` | boolean | Enable/disable transcription | | `backend` | string | Transcription backend | | `model` | string | Whisper model (`tiny`, `base`, `small`, `medium`, `large`) | | `language` | string | Language code (e.g., `en`, `es`, `fr`) | | `frameMode` | string | `scene` or `interval` | | `sceneThreshold` | number | Scene sensitivity (0-1) | | `maxFrames` | number | Maximum frames to extract | | `inlineFrames` | number | Frames to return as base64 | ### Output Structure ``` .popcorn/