# pi-nvidia-nim-diegovisk

NVIDIA NIM provider for [pi](https://github.com/earendil-works/pi-coding-agent) with the full chat model catalog, per-family thinking/request transforms, and aggressive rate-limit retry for uninterrupted free-tier development.

Forked from [pi-extension-nvidia-nim](https://github.com/Tibbee/pi-nvidia-nim-provider) (MIT, Tibbee) with merged [pi-nvidia-rate-limit-retry](https://github.com/Diegovisk/pi-nvidia-rate-limit-retry) (MIT, Diegovisk).

## Why this fork

Pi's built-in `nvidia` provider ships ~20 models with no per-model thinking transforms. The upstream `pi-extension-nvidia-nim` package registers `nvidia-nim` with ~82 chat models and 8+ thinking handlers, but does not retry 429s aggressively. This fork combines both.

## Recommended free-dev stack

| Role | Model |
|------|-------|
| Auto failover (sticky router) | `nvidia-nim/nvidia-router` |
| Primary (tools + vision) | `nvidia-nim/minimaxai/minimax-m3` |
| Reasoning fallback | `nvidia-nim/nvidia/nemotron-3-super-120b-a12b` |
| Fast fallback | `nvidia-nim/meta/llama-3.3-70b-instruct` |

Set `defaultModel` to `nvidia-router` to let the extension pick from the chain above and only switch on rate-limit exhaustion or context overflow.

## Install

### npm (recommended)

```bash
pi install npm:@diegovisk/pi-nvidia-nim@1.1.0
```

Or add to `~/.pi/agent/settings.json`:

```json
{
  "packages": ["npm:@diegovisk/pi-nvidia-nim"]
}
```

> **Note:** Published as `@diegovisk/pi-nvidia-nim` because unscoped `pi-nvidia-nim` is taken on npm.

### git

```bash
pi install git:github.com/Diegovisk/pi-nvidia-nim@main
```

### Local path (development)

```bash
pi install file:C:/Users/diego/projects/pi-nvidia-nim
```

### Configure API key

Set one of:

```bash
export NVIDIA_NIM_API_KEY="nvapi-..."
# or (compatible with pi /login and auth.json "nvidia" entry)
export NVIDIA_API_KEY="nvapi-..."
```

### Switch default provider

In `~/.pi/agent/settings.json`:

```json
{
  "defaultProvider": "nvidia-nim",
  "defaultModel": "nvidia-router",
  "packages": ["file:C:/Users/diego/projects/pi-nvidia-nim"]
}
```

Or pin a concrete model:

```json
{
  "defaultProvider": "nvidia-nim",
  "defaultModel": "minimaxai/minimax-m3"
}
```

Remove `npm:pi-extension-nvidia-nim` and the standalone `~/.pi/agent/extensions/nvidia-rate-limit-retry.ts` to avoid double-wrapping.

Then `/reload` in pi.

## Features

- **~82 curated chat models** from scraped NIM metadata + 44 regex families
- **Per-model request transforms** via `before_provider_request` (DeepSeek V4, GLM, MiniMax M3, Nemotron, Kimi K2.6, Seed OSS, etc.)
- **Content normalization** for legacy models that reject structured content arrays
- **Rate-limit retry** — up to 20 attempts with exponential backoff (env-tunable); covers `nvidia`, `nvidia-nim`, and OpenRouter `nvidia/*`
- **Compaction-safe summarization** — auto-compact and `/compact` calls force thinking **off** so MiniMax/NIM does not 400 during context overflow recovery
- **NVIDIA Router** — virtual `nvidia-router` model with sticky per-session routing across the free-dev fallback chain; re-selects only on 429 exhaustion or context overflow
- **Catalog refresh** — `/nim-refresh` caches live API model list; `npm run refresh-metadata` merges new IDs into metadata

## Commands

| Command | Description |
|---------|-------------|
| `/nim-models` | List registered models by family + recommended stack |
| `/nim-refresh` | Refresh `~/.pi/nvidia-nim-cache.json` from NIM API |
| `/nim-router` | Show sticky routed model, fallback chain, and eligibility |
| `/nim-router reset` | Clear sticky router state (re-pick from chain head) |
| `/nim-retry-status` | Show rate-limit retry stats for this session + audit log path |
| `/nim-retry-status reset` | Clear retry session counters |

Retry events are appended to `~/.pi/nvidia-nim-retry.log` (JSONL). Disable with `NVIDIA_RETRY_AUDIT=0`.

## Router env vars

| Variable | Default | Meaning |
|----------|---------|---------|
| `NVIDIA_ROUTER_CHAIN` | `minimaxai/minimax-m3,nvidia/nemotron-3-super-120b-a12b,meta/llama-3.3-70b-instruct` | Comma-separated fallback order |
| `NVIDIA_ROUTER_MARGIN` | 1.15 | Context headroom multiplier for eligibility filter |

Router behavior:

- Picks one concrete model per session (sticky) from the chain.
- Only advances on **429 retry budget exhausted** or **context overflow**.
- Filters candidates by estimated context size and vision requirements.
- Advertises 1M context on the virtual model so pi does not pre-reject long sessions.

Verify after `/reload`:

```
[nvidia-router] loaded — sticky failover router for nvidia-router.
```

## Rate-limit env vars

| Variable | Default | Meaning |
|----------|---------|---------|
| `NVIDIA_RETRY_MAX` | 20 | Max attempts per request |
| `NVIDIA_RETRY_BASE_MS` | 2000 | Base backoff |
| `NVIDIA_RETRY_CAP_MS` | 60000 | Max single delay |
| `NVIDIA_RETRY_JITTER` | 0.2 | Jitter fraction |
| `NVIDIA_RETRY_AUDIT` | 1 | Set `0` to disable JSONL audit file |
| `NVIDIA_RETRY_AUDIT_LOG` | `~/.pi/nvidia-nim-retry.log` | Audit log path |

Verify after `/reload`:

```
[nvidia-rate-limit-retry] loaded — NVIDIA NIM retries: max 20, base 2000ms, cap 60000ms, jitter 20%.
```

## Debug

```bash
NIM_DEBUG=1 pi
```

Inspect `~/.pi/nim-debug.log` for final request payloads per model.

## Validation matrix

Smoke-test these families at thinking levels `off`, `medium`, `high`:

| Family | Example model | Pass criteria |
|--------|---------------|---------------|
| `minimax-inline` | `minimaxai/minimax-m3` | Tool calls; thinking toggles |
| `nemotron-3-super-effort` | `nvidia/nemotron-3-super-120b-a12b` | Reasoning streams; no 400 |
| `deepseek-v4` | `deepseek-ai/deepseek-v4-flash` | `chat_template_kwargs` in debug log |
| `qwen-chat-template` | `qwen/qwen3.5-122b-a10b` | Native pi compat sufficient |
| `glm` | `z-ai/glm-5.2` | `enable_thinking` / `clear_thinking` correct |
| Plain | `meta/llama-3.3-70b-instruct` | No structured-array 400 |
| Rate limit | any free model | `[nvidia-rate-limit-retry]` logs; succeeds by attempt 2–3 |

Run unit tests:

```bash
npm test
```

## Upstream

- [Tibbee/pi-nvidia-nim-provider](https://github.com/Tibbee/pi-nvidia-nim-provider) — base provider + handlers
- [Diegovisk/pi-nvidia-rate-limit-retry](https://github.com/Diegovisk/pi-nvidia-rate-limit-retry) — **deprecated**; retry is bundled in this repo

## License

MIT — see [LICENSE](LICENSE).