# MCP Video Recognition Server An MCP server that describes images, transcribes audio, and summarizes video from local files. It talks to Google Gemini by default, or to any OpenAI-compatible endpoint such as OpenRouter. Video Recognition Server MCP server ## Features - Pick your provider: Google Gemini (default) or an OpenAI-compatible endpoint - Three MCP tools for local images, audio, and video - Optional Gemini model fallback, plus a final OpenAI-compatible backup - Opt-in multi-perspective parallel inference via `PARALLEL_PROMPTS` (see the [Configuration Reference](docs/CONFIGURATION.md#parallel-prompts-and-merging-methods)) Model support varies by provider. Picking the OpenAI-compatible provider does not mean every endpoint or model handles every media type. And the server never swaps in a different model or provider on its own. ## Prerequisites - Node.js **18.0.0 or later** - An API key for your provider: - Gemini: `GOOGLE_API_KEY` - OpenAI-compatible: `OPENAI_COMPATIBLE_API_KEY` ## Install ```bash git clone https://github.com/yourusername/mcp-video-recognition.git cd mcp-video-recognition npm install npm run build ``` ## Quickstart Add the server to your MCP client config and point it at the built `dist/index.js`: ```json { "mcpServers": { "video-recognition": { "command": "node", "args": ["/path/to/mcp-video-recognition/dist/index.js"], "env": { "GOOGLE_API_KEY": "your_google_api_key" } } } } ``` On Windows, use forward slashes or doubled backslashes (`\\`) in the path. Save the file and reconnect your MCP client. Environment variables must be present in the MCP client's `env` configuration; the server process does not automatically inherit values merely because a `.env` file exists beside it. For OpenRouter or another OpenAI-compatible endpoint, set `RECOGNITION_PROVIDER=openai-compatible` and fill in its variables. [Configuration](docs/CONFIGURATION.md) has a ready-made example. ## Standalone Client The repository includes a client that builds the project, spawns the server over stdio, performs MCP initialization, and lists the tools. It loads `.env` from the repository root when that file exists; already-set process environment variables take precedence. ```bash # Connect and list tools npm run client # Connect and call a tool npm run client -- --tool image_recognition --args '{"filepath":"C:/media/example.png"}' ``` Use `--help` for server-path, working-directory, env-file, and timeout options: ```bash npm run client -- --help ``` A tool error exits with status 2 and prints the complete MCP error result. Connection or configuration failures exit with status 1 and include the spawned server's stderr diagnostics.
Other install options (FLUJO) With [FLUJO](https://github.com/mario-andreschak/FLUJO/): 1. Click **Add Server**. 2. Paste the GitHub URL. 3. Click **Parse, Clone, Install, Build and Save**.
## Configuration The server reads environment variables. These are the ones you'll touch most: | Variable | Default | Purpose | |---|---|---| | `RECOGNITION_PROVIDER` | `gemini` | `gemini` or `openai-compatible` | | `GOOGLE_API_KEY` | none | Gemini API key | | `GEMINI_MODEL` | `gemini-3.5-flash` | Gemini model to use | | `OPENAI_COMPATIBLE_API_KEY` | none | OpenAI-compatible API key | | `OPENAI_COMPATIBLE_BASE_URL` | none | Endpoint base URL | | `OPENAI_COMPATIBLE_MODEL` | none | Model to use | | `ALLOWED_MEDIA_ROOTS` | none | Media directories for the OpenAI-compatible provider and Gemini backup | | `TRANSPORT_TYPE` | `stdio` | `stdio` or `streamable-http`; `sse` is a legacy alias | | `MCP_HOST` | `127.0.0.1` | Bind address for Streamable HTTP | | `PORT` | `3000` | Bind port for Streamable HTTP | | `LOG_LEVEL` | `info` | Diagnostic threshold; all logs go to stderr | A bad value stops startup. Nothing gets fixed silently. The **[Configuration Reference](docs/CONFIGURATION.md)** has the full variable list, validation rules, an OpenRouter example, and the supported media types. For Gemini model fallback and the final backup, read the **[Provider Recovery Reference](docs/RECOVERY.md)**. ## Tools You get three MCP tools. Each takes a local `filepath`, an optional `prompt` (default `Describe this content`), and an optional `modelname` override. - `image_recognition` - describe an image - `audio_recognition` - transcribe or describe audio - `video_recognition` - describe a video Example: ```json { "name": "video_recognition", "arguments": { "filepath": "/path/to/video.mp4", "prompt": "Describe what happens in this video" } } ``` ## Security - HTTPS is required by default. Plain HTTP only works for a local endpoint you explicitly enable. - The OpenAI-compatible provider and the Gemini backup read media only from directories listed in `ALLOWED_MEDIA_ROOTS`. Containment is recursive, so specifying a parent folder (e.g. `C:\Projects` or `${workspaceFolder}`) covers all repositories, subfolders, and media files inside it. - Keys stay in the process environment. Don't commit real keys. The **[Security Reference](docs/SECURITY.md)** covers endpoint rules, resource limits, and incident response. ## MCP Inspector and Streamable HTTP For stdio Inspector use, build first and make sure the API key is in Inspector's server environment. In PowerShell: ```powershell $env:GOOGLE_API_KEY = "your_google_api_key" npm run debug ``` For HTTP, start the server separately: ```powershell $env:GOOGLE_API_KEY = "your_google_api_key" $env:TRANSPORT_TYPE = "streamable-http" npm start ``` Then select **Streamable HTTP** in Inspector and connect to `http://127.0.0.1:3000/mcp`. Do not select legacy SSE: the backwards-compatible `TRANSPORT_TYPE=sse` spelling still starts a Streamable HTTP endpoint. Listing tools only proves the MCP handshake succeeded. It does not contact the recognition provider or read a media file. A later call can still fail because the key, model, network, or `filepath` is invalid. The path must identify a file visible to the spawned server process; paths from a different container or host will not work. ## Development ```bash # Build, connect through the standalone client, and list tools GOOGLE_API_KEY=your_api_key npm run client # Build and run the provider foundation tests npm run verify:provider-foundation ``` ### Project Structure - `src/index.ts`: entry point and provider construction - `src/server.ts`: MCP server and transport - `src/tools/`: the three recognition tools - `src/services/`: Gemini and OpenAI-compatible providers - `src/types/`: shared types - `src/utils/`: helpers ## License MIT