# Llamafiles provider extension for pi

A pi extension that supervises locally-run llamafile-style model servers
(or any OpenAI-compatible server you can launch from a binary). It
registers a `llamafiles` provider with pi, starts the configured binary
when you pick one of its models, and stops it when you switch away or
quit.

## Status

Implementation complete; covered by 55 automated tests (42 unit + 13
integration via the SDK driver). `npm test` runs in ~15s and exits 0.

See [`SPECS.md`](SPECS.md) for the behavioral contract, [`PLAN.md`](PLAN.md)
for the implementation plan, and [`NOTES.md`](NOTES.md) for the pi API
findings that informed the design.

## Features

- **Process supervision** — starts the configured binary on `/model`,
  waits for `/v1/models` to respond, then reports ready. One process per
  pi session.
- **Per-model port** — each model declares its own `port`; pi sends
  requests there. Default `8080`.
- **`{{port}}` substitution in `args`** — `port` is the single source of
  truth; reference it in your binary's arg list as `{{port}}`.
- **Adoption** — if a compatible server is already running on the port,
  pi adopts it instead of spawning a duplicate.
- **Foreign-port safety** — if a different server holds the port, pi
  surfaces the conflict and tells you to free it manually. It never kills
  processes it did not start.
- **Visible quit progress** — when you exit pi while it owns a running
  process, the extension prints "Stopping llamafile ..." and
  "Stopped llamafile ..." to stderr so the user can see the wait.
- **Transparent reload** — `/reload` does **not** prompt; the process
  keeps running and the freshly loaded extension instance re-adopts it.
- **Cleanup-by-design** — adopted processes are never stopped without
  your consent.

## Installation

Three options, in order of recommendation:

**From npm** (most convenient, gets gallery indexing):

```bash
pi install npm:@aittalam/pi-llamafile
```

**From git** (pins to a tag, no npm account needed by you or by pi):

```bash
pi install git:github.com/aittalam/pi-llamafile@v1.0.0
```

**From source** (for hacking on the extension):

```bash
git clone https://github.com/aittalam/pi-llamafile ~/.pi/agent/extensions/pi-llamafile
cd ~/.pi/agent/extensions/pi-llamafile
npm install
```

In all cases, use `/reload` from a running pi session, or restart pi, to pick
up changes.

## Configuration

Define your llamafile models in `~/.pi/agent/models.json` under the
`llamafiles` provider:

```jsonc
{
  "providers": {
    "llamafiles": {
      "models": [
        {
          "id": "qwen3-9b",
          "name": "Qwen3 9B",
          "command": "sh",
          "args": [
            "/path/to/qwen3-9b.llamafile",
            "--server",
            "--port",
            "{{port}}",
            "--jinja"
          ],
          "port": 8080,
          "reasoning": false,
          "input": ["text"],
          "contextWindow": 32768,
          "maxTokens": 4096
        }
      ]
    }
  }
}
```

Optional per-model fields beyond pi's standard set:

| Field     | Description                                          |
|-----------|------------------------------------------------------|
| `command` | Executable to spawn. Required.                        |
| `args`    | Argument list. `{{port}}` is substituted at spawn.    |
| `port`    | TCP port the server listens on. Default `8080`.       |
| `env`     | Extra environment variables for the spawned process.  |
| `cwd`     | Working directory for the spawned process.            |

See [`SPECS.md` §3.2](SPECS.md) for the complete schema and defaults.

## Usage

1. `pi --list-models` should show your llamafile models under
   the `llamafiles` provider.
2. Use `/model` to pick one. The extension spawns the binary, waits
   for readiness, and reports `<name> is ready`.
3. Switch with `/model` again. The current process is stopped and the
   new one started.
4. Switch to any non-llamafile model: the running llamafile is stopped.
5. `/quit` (or `Ctrl+D`): the running llamafile is stopped. Two lines
   appear on the terminal (stderr): `Stopping llamafile "<name>" ...`
   then `Stopped llamafile "<name>".` Adopted servers (started outside
   pi) are left running silently.
6. `/llamafiles`: prints the current state.

## Development

```bash
npm install        # install dev deps
npm test           # 56 tests, unit + integration
npm run test:unit  # 45 unit tests, sub-second
npm run dev        # pi -e . — smoke-load against your real HOME
```

Logs from spawned binaries are appended to `~/.pi/llamafile_logs/<modelId>.log`.

## Layout

```
SPECS.md          # behavioral contract (single source of truth)
PLAN.md           # implementation plan
NOTES.md          # pi API findings
index.ts          # thin wiring: events, commands
src/
  config.ts       # models.json loader
  template.ts     # {{port}} substitution
  process.ts      # LlamafileSupervisor
  log.ts          # file log streams
  notify.ts       # notification text
  types.ts        # shared types
tests/
  unit/           # 3 files, 42 tests
  integration/    # 13 files, all SDK-driven
  helpers/        # fake-server, harness, port allocator
  MANUAL.md       # checklist for things automation cannot reach
```

## License

MIT.