# work-with-llamacpp

> 🌐 中文: [README.md](./README.md)

A [pi](https://github.com/earendil-works/pi) extension that bridges your local [llama.cpp](https://github.com/ggml-org/llama.cpp) server to the agent: **vision, translation, and general text tasks** — all powered by your local models, zero cloud tokens.

## ✨ Features

- 🔍 **Auto-detection**: probes the llama.cpp server at session start and before every turn (HTTP `/props`, falls back to process lookup)
- 🧠 **Model type detection**: auto-detects vision models (with mmproj) vs text models; can be forced manually
- 🖼️ **Vision** (`llama_cpp_vision`): sends images (local path or URL) to your local vision model
- 🌐 **Translation** (`llama_cpp_translate`): translation-optimized prompt, multi-language support
- ⚙️ **General text tasks** (`llama_cpp_chat`): summarization, rewriting, correction, keyword extraction, etc.
- 💉 **Stealth status injection**: one-line model status injected into the system prompt each turn (visible to the LLM, not to the UI, no cumulative context cost)
- 🔑 **apiKey support**: compatible with `llama-server --api-key`; requests carry auth headers automatically
- 🖥️ **Status bar + notifications**: persistent status bar, state changes notify only once (no spam)

## 📦 Installation

```bash
# From npm (recommended)
pi install npm:work-with-llamacpp

# Or from git
pi install git:github.com/kikilloy/work-with-llamacpp
```

Run `/reload` after installing.

## 🚀 Usage

**Commands**

| Command | Description |
|---------|-------------|
| `/work-llamacpp-status [refresh]` | Show/refresh llamacpp detection status |
| `/work-llamacpp-mode [auto\|vision\|text\|full]` | Switch model mode (no arg = cycle) |
| `/work-llamacpp-config` | Interactive setup of port and apiKey |
| `/work-llamacpp-config port 8080` | Set port directly (0 = auto-probe) |
| `/work-llamacpp-config apikey <key>` | Set apiKey directly (`-` clears it) |

**Tools** (called automatically by the agent as needed)

| Tool | Purpose |
|------|---------|
| `llama_cpp_vision` | Analyze images with the local vision model |
| `llama_cpp_translate` | Translate text with the local text model |
| `llama_cpp_chat` | General text tasks with the local text model |

## ⚙️ Configuration

Config file: `~/.pi/agent/extensions/work-with-llamacpp/config.json`

```jsonc
{
  "port": 0,                 // port; 0 = auto-probe commonPorts
  "commonPorts": [8080, 8081, 1234, 8000],
  "cacheMs": 30000,          // detection cache in ms
  "probeTimeoutMs": 2500,    // per-port probe timeout in ms
  "modelType": "auto",       // auto | vision | text | full (full = vision+translate+chat)
  "capabilities": [],        // vision/translate/chat; empty = inferred by type (mode command clears this)
  "apiKey": "",              // password set via llama-server --api-key
  "maxImageBytes": 20971520, // max image size in bytes (default 20MB)
  "maxTokens": 8192          // max output tokens (tools can override with max_tokens param)
}
```

Config priority: config file > env var `LLAMACPP_PORT` > built-in defaults.

> 💡 **Modes**: `/work-llamacpp-mode` switches auto / vision / text / full — auto auto-detects (mmproj → vision); vision forces vision (vision+chat); text forces text (chat+translate); full enables everything (vision+translate+chat). The status bar shows the current type and capability set.

## 🔧 Starting llama.cpp

```bash
# Text model (translation / general tasks)
llama-server -m /path/to/model.gguf

# Vision model (requires mmproj)
llama-server -m /path/to/model.gguf --mmproj /path/to/mmproj.gguf

# With password
llama-server -m /path/to/model.gguf --api-key your-password
```

## ⚠️ Security Note

Pi extensions run with full system permissions — only install packages from trusted sources. The apiKey is stored in plain text in the local config file; keep it safe.

## 📄 License

[MIT](./LICENSE)

### made by DEEPSEEK