# @jiajun0413/pi-provider-manager

**Roundrobin failover engine for the [pi coding agent](https://pi.dev).**

Registers a virtual `roundrobin` provider that pools multiple real model candidates behind one virtual model, with sticky failover, idle-timeout guards, and per-candidate cooldown.

## Attribution

This package is a **fork and rebuild** of `@arcaneorion/pi-provider-manager@0.3.9` (upstream npm package; no public Git repo). The roundrobin failover engine architecture is the original author's work, used here under MIT.

This fork fixes a mid-stream idle-timeout bug (a candidate stream that starts then goes silent no longer hangs forever), hardens failover against unstable relays (upstream `aborted` is now a cooldown-able failure, not a turn-killing cancel; auth lookup is bounded by the idle guard), and strips the visual config panel and on-disk health store, keeping only the failover engine.

## Install

```bash
pi install npm:@jiajun0413/pi-provider-manager
```

Restart pi. (pi loads `.ts` extensions via jiti — no build step.)

## Features

- **Roundrobin failover** — one virtual `roundrobin` provider pools multiple provider/model candidates. Sticky strategy with automatic failover on first-response timeout or stream error. Each preset becomes a selectable virtual model.
- **Mid-stream idle timeout** — a candidate stream that starts then goes silent errors out after `timeoutMs` instead of hanging forever (prevents ghost subagent sessions).
- **No mid-stream replay** — once content or tool calls have been emitted, an error is terminal rather than replayed on another provider, to avoid duplicating output.
- **Upstream-aborted is a failure, not a cancel** — a relay that drops the socket / cancels upstream and surfaces `stopReason:"aborted"` is cooled down and the pool rotates, instead of killing the whole turn. Only a real user abort (the request's own `AbortSignal`) short-circuits.
- **Auth-bounded idle guard** — API-key/header resolution races against the same `timeoutMs` + attempt abort as stream chunks, so a hung key lookup can't stall the turn.
- **Entry-time model population** — virtual models register at load time with placeholder candidates so they appear in `/model` and static lists (pi-web's `/api/models`) before a session starts; `session_start` rebuilds with the real registry.
- **virtualModel inheritance** — the `virtualModel` block is optional; omit it and metadata is inherited from the group's first candidate (see [Inheritance](#inheritance)).

## Usage

1. Create `~/.pi/agent/roundrobin/config.json` (the `"default"` group):

   ```json
   {
     "virtualModel": {
       "name": "GLM-5.2",
       "reasoning": true,
       "input": ["text"],
       "contextWindow": 1000000,
       "maxTokens": 128000,
       "thinkingLevelMap": { "max": "max" },
       "compat": { "thinkingFormat": "deepseek" }
     },
     "candidates": [
       { "provider": "k", "model": "glm-5.2" },
       { "provider": "wg", "model": "glm-5.2" }
     ],
     "timeoutMs": 16000,
     "cooldownMs": 120000,
     "strategy": "sticky",
     "log": true
   }
   ```

2. (Optional) Add more groups as `*.json` files under `presets/`. Each file's stem (name without `.json`) becomes a `roundrobin/<name>` model.
3. In pi, use `/model` to select `roundrobin/default` (or any preset name).
4. Requests now flow through the failover engine. If the current candidate times out or errors before the stream starts, the next candidate is tried automatically. If every candidate fails, the engine waits for the earliest cooldown and retries from the preferred candidate — bounded to 3 passes before the turn ends with the last error.

A mid-stream error is returned immediately rather than replayed (content or tool calls may already have been emitted).

**Sticky strategy** — stays on the current candidate until it fails. After cooldown, returns to the preferred (first) candidate automatically.

### Multi-preset routing

Each preset under `~/.pi/agent/roundrobin/` registers as an independent virtual model whose model id equals the preset file stem (the `.json` suffix is stripped). `config.json` is the reserved `"default"` group; every `*.json` file under `presets/` is an additional group. In `/model` all virtual models appear at once — pick `roundrobin/<name>` to route to that group's pool. Health and `currentIndex` are isolated per preset.

## Configuration

### `~/.pi/agent/roundrobin/config.json`

| Field | Default | Meaning |
|---|---|---|
| `virtualModel` | *(optional)* | Virtual model metadata: `name`, `reasoning`, `input`, `contextWindow`, `maxTokens`, `thinkingLevelMap`, `compat`. `id` is forced to the group name. Omitted fields are **inherited from the first candidate** (see [Inheritance](#inheritance)); omit the whole block for homogeneous groups. |
| `candidates` | `[]` | `{ provider, model }[]` referencing entries in `models.json`. Unknown pairs are skipped (if all are invalid the group is `enabled=false`). |
| `timeoutMs` | `30000` | Idle timeout per stream event — first byte **and** every mid-stream chunk race against this. |
| `cooldownMs` | `60000` | How long a failed candidate is skipped before it can be retried. |
| `strategy` | `sticky` | `sticky` / `round-robin` / `primary`. |
| `log` | `true` | Append failover events to `~/.pi/agent/roundrobin/roundrobin.log`. |

### Presets

`*.json` files under `~/.pi/agent/roundrobin/presets/<name>.json` — each is a standalone config (same shape as `config.json`) and becomes a `roundrobin/<name>` virtual model (id = `<name>`, the `.json` suffix stripped). `bak.*` files are skipped. A corrupt preset is skipped silently; it never breaks the engine.

### Inheritance

The `virtualModel` block is **optional**. When you omit a field (or the whole block), the engine fills it from the group's **first resolved candidate** — the `Model` from `modelRegistry.find(provider, model)` in your `models.json`. Precedence is three-tier:

1. **Explicit** `virtualModel.<field>` in the preset wins.
2. Else **inherit** from the first resolved candidate's `Model`.
3. Else a built-in **default**.

A homogeneous group (multiple providers of the same model) needs no `virtualModel` at all:

```json
{
  "candidates": [
    { "provider": "k", "model": "glm-5.2" },
    { "provider": "wg", "model": "glm-5.2" }
  ],
  "timeoutMs": 16000,
  "cooldownMs": 120000,
  "strategy": "sticky"
}
```

For heterogeneous groups (mixed models, e.g. a "power" pool of Grok + Claude + Deepseek), keep an explicit `virtualModel` to unify behavior across the pool (force a single `compat.thinkingFormat`, or a conservative `contextWindow`).

> Inheritance happens at `session_start`, when the real model registry is available. Pre-`session_start` the engine registers with placeholder candidates so virtual models still show in `/model` and static lists; fields are then filled from real candidates once a session starts.

## Architecture

```text
index.ts                          ← entry: registers the roundrobin provider
extensions/
├── model-roundrobin.ts           ← failover engine (streamRoundRobin → tryCandidates)
└── rr-config.ts                  ← read config.json + presets/ → GroupConfig[]
```

Cooldown (failure tracking) is **process-local in-memory** — a failed candidate's cooldown resets when pi restarts. This is by design: cooldown is an in-process failover notion, not a durable statistic.

## License

MIT