The fastest local AI engine for Apple Silicon.
Drop-in OpenAI / Anthropic API · 2–4× faster than Ollama · Runs on any M-series Mac.

_{rapidmlx.com · Docs · Model mirror · Desktop app}

/install.sh -o install.sh && shasum -a 256 install.sh && bash install.sh` > - **Skip the shell script entirely** — use Homebrew, `uv`, or `pip` below. See [Alternative install methods](#alternative-install-methods) for the non-curl paths. **2. Chat with a model right now:** ```bash rapid-mlx chat ``` Defaults to `qwen3.5-4b-4bit`. First run downloads the weights (~2.5 GB) with a progress bar and drops you into a REPL. Type `/help` for slash commands, `/exit` to quit. **3. Or serve it for use from other apps:** ```bash rapid-mlx serve qwen3.5-4b-4bit ``` Starts an OpenAI-compatible HTTP server bound to `http://localhost:8000`. Point any OpenAI SDK / client (Cursor, Aider, LangChain, OpenCode, PydanticAI, your own scripts) at **`http://localhost:8000/v1`**; Claude Code / Anthropic SDK uses **`http://localhost:8000`** (the Anthropic messages route lives at `/v1/messages` under the same host). ```bash curl http://localhost:8000/v1/chat/completions \ -H "Content-Type: application/json" \ -d '{"model":"default","messages":[{"role":"user","content":"Say hello"}]}' ``` ```python from openai import OpenAI client = OpenAI(base_url="http://localhost:8000/v1", api_key="not-needed") print(client.chat.completions.create( model="default", messages=[{"role": "user", "content": "Say hello"}], ).choices[0].message.content) ``` **4. Or wire up your coding agent — one command:** ```bash rapid-mlx launch claude-code ``` With a server running (step 3), this patches Claude Code's local config (`~/.config/claude/settings.json`) to route at `http://localhost:8000` — no manual env vars, no editing JSON by hand. You get a fully local Claude Code: `$0` per token, nothing leaves your Mac. Swap in `cursor`, `cline`, or `continue-dev` for the other IDE clients, or run `rapid-mlx launch list` to see what's detected on this machine. > **Vision / audio / video / diffusion models?** Base install is text-only (~460 MB). Vision, audio (TTS, STT, voice cloning), video generation, embeddings, and DFlash speculative decoding ship as opt-in extras. → [Optional extras](https://rapidmlx.com/docs/extras.html) > **Not into the terminal?** [**Rapid-MLX Desktop**](https://rapidmlx.com/desktop) bundles the same engine inside a one-click Mac app. --- ## Video generation Run text-to-video or image-to-video locally through the OpenAI-compatible Videos API. Three backends ship — **Wan 2.1 / 2.2**, **CogVideoX-Fun** and **LTX-2.3** — across 8 registered checkpoints. `wan2.2-ti2v-5b-q8` is the recommended starting point: smallest of the Wan set, and TI2V means one checkpoint does both text-to-video and image-to-video. Requires Python 3.11+ (the video runtime does not support 3.10; core text and audio still do) and `ffmpeg` for the final MP4 mux. ```bash pip install 'rapid-mlx[video]' brew install ffmpeg rapid-mlx serve wan2.2-ti2v-5b-q8 ``` Create and download a clip: ```bash curl http://localhost:8000/v1/videos \ -F model=wan2.2-ti2v-5b-q8 \ -F 'prompt=A fox running through fresh snow, cinematic tracking shot' \ -F seconds=1 \ -F size=832x512 # Poll until GET /v1/videos/VIDEO_ID reports "status": "completed", then: curl http://localhost:8000/v1/videos/VIDEO_ID/content -o output.mp4 ``` The create call returns a job immediately. Poll `GET /v1/videos/VIDEO_ID` until `status` is `completed`. Add `-F input_reference=@start.png` for image-to-video. Generation is serialized — one clip at a time — because two diffusion pipelines resident at once will exhaust unified memory. Expect minutes of compute per second of footage, not real time. → [Every checkpoint, RAM requirement and tuning knob](https://rapidmlx.com/docs/models/families/video.html) --- ## Audio: speech, transcription, voice cloning 41 audio aliases behind the OpenAI-compatible `/v1/audio/*` endpoints — any OpenAI SDK works unchanged. ```bash pip install 'rapid-mlx[audio]' # Text to speech rapid-mlx serve kokoro curl http://localhost:8000/v1/audio/speech \ -H "Content-Type: application/json" \ -d '{"model":"kokoro","input":"hello from rapid-mlx"}' --output hello.wav # Transcription (Whisper / Parakeet / SenseVoice) rapid-mlx serve whisper-large-v3-turbo curl http://localhost:8000/v1/audio/transcriptions \ -F file=@hello.wav -F model=whisper-large-v3-turbo ``` Beyond the basics, three things you may not expect to run locally: - **Zero-shot voice cloning** from a reference clip. `indextts` is the only one that takes the clip alone; `qwen3-tts-clone`, `f5-tts-zh` and `chatterbox` all require `ref_text` (the clip's exact transcript) paired with `ref_audio`, and the request is rejected before generation if it is missing. - **Voice design** — `qwen3-tts-voicedesign` has no named speakers at all. Describe the voice you want in natural language via `instructions` (timbre, gender, age, accent, emotion, prosody) and it synthesises it. - **Forced alignment** — `qwen3-aligner` takes audio *plus the transcript you already have* and returns per-character timings. It never guesses at the words, so it cannot mis-hear them; that is what karaoke captions and beat-synced editing need. Also: word-level timestamps on transcription, and local text-to-music at `/v1/audio/music`. → [All 41 aliases across 12 families](https://rapidmlx.com/docs/models/families/audio.html) --- ## Why Rapid-MLX | | | |---|---| | **Apple-Silicon-native** | Pure MLX kernels — no llama.cpp fallback, no Metal shim. Continuous batching, prompt cache (radix + DeltaNet RNN snapshots), and a quantized live KV cache (int4/int8 on the continuous-batching cache + TurboQuant K8V4 codec) run at native MLX bandwidth on M1 → M4. | | **Drop-in OpenAI / Anthropic API** | `/v1/chat/completions`, `/v1/responses` (Codex CLI), `/v1/messages` (Anthropic SDK / Claude Code), `/v1/embeddings`, `/v1/audio/*`, `/v1/videos` — same wire as ChatGPT / Claude, no client adapter. | | **First-class ecosystem coverage** | 11 agent CLIs and 3 Python frameworks are wire-verified against real weights every release (4 are Tier-1, re-verified on current binaries) — Codex CLI, Claude Code, OpenCode, Qwen Code, OpenHands, Hermes Agent, Aider, Kilo Code, GitHub Copilot, Factory Droid, Moonshot Kimi Code + LangChain, PydanticAI, smolagents. | → [Full feature breakdown](https://rapidmlx.com/docs/index.html) --- ## Use Cases | | | | |---|---|---| | **Chat in the terminal** | `rapid-mlx chat qwen3.5-9b-4bit` | Streaming REPL, `/help` for slash commands, `--think` / `--no-think` to control CoT. | | **OpenAI server for your apps** | `rapid-mlx serve qwen3.5-9b-4bit` | Point Cursor, Aider, LibreChat, Open WebUI, LangChain at `http://localhost:8000/v1`. | | **Agent backends** | `rapid-mlx serve qwen3.6-35b-8bit &`
`rapid-mlx agents codex --setup && codex` | 8 agents auto-configure via `agents --setup` once the server is up (11 wire-verified total, 4 Tier-1) — see [Agent support](#agent-support). | | **Benchmark your Mac** | `rapid-mlx bench qwen3.5-9b-4bit --submit` | Standardized B=1 bench, opens a PR to publish your row on [rapidmlx.com](https://rapidmlx.com). | → [One-shot IDE setup](https://rapidmlx.com/docs/cli.html#launch) with `rapid-mlx launch ` --- ## Agent Support All 11 agents below are wire-verified against real weights every release via their own integration-test cell. Of these, four are **Tier-1** — **Claude Code, Codex CLI, Hermes, and Aider** — re-verified end-to-end against the *current* client binary every release, with one guardian per API wire (Anthropic `/v1/messages`, OpenAI `/v1/responses`, and `/v1/chat/completions` covered for both tool-calling depth and reach). The other seven are **Tier-2**: wire-verified in the matrix and configured on-demand. The first eight agents each ship a `rapid-mlx agents --setup` config template (except Claude Code, which is one env-var); GitHub Copilot, Factory Droid, and Moonshot Kimi Code plug in through their own documented BYOK config (auth-gated, so the matrix cell is a wire smoke). **Tier-1 (4):** Claude Code · Codex CLI · Hermes · Aider — last re-verified end-to-end 2026-07-28 on current binaries (claude 2.1.211, codex 0.145.0, hermes 0.9.0, aider 0.86.2) against rapid-mlx 0.11.1. **Tier-2 (7):** OpenCode · Qwen Code · OpenHands · Kilo Code · GitHub Copilot · Factory Droid · Moonshot Kimi Code. | Agents (11) | Frameworks (3) | |---|---| | [Codex CLI](https://github.com/openai/codex) · [Claude Code](https://www.anthropic.com/claude-code) · [OpenCode](https://github.com/sst/opencode) · [Qwen Code](https://github.com/QwenLM/qwen-code) · [OpenHands](https://github.com/All-Hands-AI/OpenHands) · [Hermes Agent](https://github.com/NousResearch/hermes-agent) · [Aider](https://aider.chat) · [Kilo Code](https://github.com/Kilo-Org/kilocode) · [GitHub Copilot](https://github.com/features/copilot) · [Factory Droid](https://factory.ai) · [Moonshot Kimi Code](https://github.com/MoonshotAI/kimi-cli) | [LangChain](https://langchain.com) (+ [LangGraph](https://langchain-ai.github.io/langgraph/)) · [PydanticAI](https://ai.pydantic.dev) · [smolagents](https://github.com/huggingface/smolagents) | Also compatible with any OpenAI-compatible client via `http://localhost:8000/v1` — Cursor, LibreChat, Open WebUI, and more plug in with a single URL change. → [Full 11-agent + 3-framework matrix (test cells + xfail reasons)](https://rapidmlx.com/docs/matrix.html) → [Codex CLI](https://rapidmlx.com/docs/matrix.html#agent-codex-cli) · [Claude Code](https://rapidmlx.com/docs/matrix.html#agent-claude-code) · [OpenCode](https://rapidmlx.com/docs/matrix.html#agent-opencode) · [Qwen Code](https://rapidmlx.com/docs/matrix.html#agent-qwen-code) · [OpenHands](https://rapidmlx.com/docs/matrix.html#agent-openhands) · [Hermes](https://rapidmlx.com/docs/matrix.html#agent-hermes-agent) · [Aider](https://rapidmlx.com/docs/matrix.html#agent-aider) · [Kilo Code](https://rapidmlx.com/docs/matrix.html#agent-kilo-code) · [Copilot](https://rapidmlx.com/docs/matrix.html#agent-copilot) · [Droid](https://rapidmlx.com/docs/matrix.html#agent-droid) · [Kimi Code](https://rapidmlx.com/docs/matrix.html#agent-kimi-code) --- ## Choose Your Model The installer's RAM detector picks a sensible default. If you want to shop the full catalog: `rapid-mlx models` lists every alias, `rapid-mlx info ` shows the per-alias profile (parser, MoE / hybrid flags, KV codec eligibility, speculative-decoding gates). | RAM | Recommended | One-shot | |---|---|---| | **8–23 GB** MacBook Air/Pro | `qwen3.5-4b-4bit` | `rapid-mlx serve qwen3.5-4b-4bit` | | **24–47 GB** MacBook Pro / Mac Mini | `gpt-oss-20b-mxfp4-q8` | `rapid-mlx serve gpt-oss-20b-mxfp4-q8` | | **48–95 GB** Mac Studio | `qwen3.6-35b-8bit` | `rapid-mlx serve qwen3.6-35b-8bit` | | **96 GB+** Mac Studio / Pro | `gpt-oss-120b-mxfp4-q8` | `rapid-mlx serve gpt-oss-120b-mxfp4-q8` | → [Full RAM tier map + serve flags per tier](https://rapidmlx.com/docs/hardware-tiers.html) → [Every alias, quant, and family (165 text + 41 audio + 8 video aliases, 214 total)](https://rapidmlx.com/docs/aliases.html) · interactive at [models.rapidmlx.com](https://models.rapidmlx.com/) --- ## Alternative install methods The two paths above cover most users — reach for these only if you already manage Python yourself.

Homebrew — Mac-native, one command, prebuilt bottle from homebrew/core

```bash brew install rapid-mlx ``` Ships in homebrew-core since 0.10.12 — no tap, no trust prompt. Upgrade with `brew upgrade rapid-mlx`. If you previously installed from the legacy `raullenchai/rapid-mlx` tap, switch once: `brew uninstall rapid-mlx && brew untap raullenchai/rapid-mlx && brew install rapid-mlx`.

uv — isolated tool install, auto-manages Python

```bash uv tool install rapid-mlx@latest ``` Don't have uv yet? `curl -LsSf https://astral.sh/uv/install.sh | sh`. Upgrade with `uv tool upgrade rapid-mlx`.

pip — requires Python 3.10+ (macOS ships 3.9)

```bash python3.12 -m pip install rapid-mlx ``` If `pip install rapid-mlx` says "no matching distribution", your Python is too old. `brew install python@3.12` first. Upgrade with `pip install -U rapid-mlx`. For image-input / VLM models (Qwen-VL, true multimodal), install the vision extra: `pip install 'rapid-mlx[vision]'` — see [Optional extras](https://rapidmlx.com/docs/extras.html).

--- ## Command Reference ```bash rapid-mlx --help # top-level command list rapid-mlx --help # per-subcommand flags ``` Covers chat, serve, share, agents (setup / test), bench, models, pull, rm, ps, info, doctor, upgrade, telemetry, launch, and jlens. → [Full CLI reference with every flag](https://rapidmlx.com/docs/cli.html) --- ## Troubleshooting Run the built-in self-check first: ```bash rapid-mlx doctor ``` Top three things that go wrong: - **Much slower than expected.** Qwen3.5 / 3.6 default to thinking-on — add `--no-think` to skip chain-of-thought. → [Slow tok/s](https://rapidmlx.com/docs/troubleshooting.html#issue-slow-tps) - **Out of memory.** Model too big for your RAM — pick a smaller quant from [Choose Your Model](#choose-your-model) or the [full tier map](https://rapidmlx.com/docs/hardware-tiers.html). → [OOM guide](https://rapidmlx.com/docs/troubleshooting.html#issue-oom) - **Tool calls arriving as plain text.** Auto-recovery handles most cases; if not, set `--tool-call-parser` explicitly for your model. → [Tool-call recovery](https://rapidmlx.com/docs/troubleshooting.html#issue-tool-call-text) → [All troubleshooting entries](https://rapidmlx.com/docs/troubleshooting.html) (OOM, empty responses, slow TTFT, port taken, shell completion, HF cache, and more) --- ## See it in action

## Community & Contributing - **Report a bug or request a model:** [Issues](https://github.com/raullenchai/Rapid-MLX/issues/new/choose) - **Report a security issue:** [Private advisory](https://github.com/raullenchai/Rapid-MLX/security/advisories/new) — see [SECURITY.md](SECURITY.md) - **Ask a question or share a build:** [Discussions](https://github.com/raullenchai/Rapid-MLX/discussions) - **Contribute code, aliases, or docs:** [CONTRIBUTING.md](CONTRIBUTING.md) - **Add your hardware to the public benchmark:** `rapid-mlx bench --submit` opens the PR for you Rapid-MLX ships **opt-in anonymous telemetry** (off by default; explicit `rapid-mlx telemetry enable` required). No prompts, completions, paths, IPs, or API keys are ever collected. → [What we do and don't collect](https://rapidmlx.com/docs/telemetry.html) ### 🚀 Contributors Every avatar here shipped something in rapid-mlx — model support, tool-call parsers, fixes, docs, and benchmark submissions. Thank you. ### Star History --- ## License Apache 2.0 — see [LICENSE](LICENSE).