# Chat model notes and provider comparison

These notes were moved from the README to keep setup instructions easier to find. Dates and model counts describe the recorded probes and comparison snapshots, not a fresh check of the hosted endpoints.

[Back to the README](../README.md#chat-models-and-thinking-support)

## MiniMax context trial 2026-10-25

Version 1.18.1 registers MiniMax M3.1 Flash with exactly **1,000,000 tokens**
of context at the user's request, following the model card, so it can be tested
in real work before Pi compacts at the earlier 262,144-token setting. The
generator preserves this explicit choice on refresh. NVIDIA's hosted listing
still advertises 262,144; no request above that threshold has been tested, and
the larger context must not be described as hosted-verified. The output
allowance stays 65,536 and sampling remains unset. The original addition's
evidence below records the earlier configuration.

## Model additions 2026-10-25

Version 1.18.0 adds two chat models, bringing the catalog to **18 chat models
and two image models**. The existing 52 generated metadata rows were retained;
only the two new entries were generated with the maintenance tool.

| Model and public ID | Pi inputs | Hosted context | Extension output allowance | Thinking control |
|---|---|---:|---:|---|
| MiniMax M3.1 Flash, `minimaxai/minimax-m3.1-flash` | Text, image | 262,144 | 65,536 | low / medium / high / xhigh / max |
| Nemotron 3.5 Bolt E4B, `nvidia/nemotron-3.5-bolt-e4b` | Text | 65,536 | 32,768 | off / on (Pi high) |

[MiniMax's family card](https://build.nvidia.com/minimaxai/minimax-m3.1-flash/modelcard)
describes an architecture with up to 1M context. The embedded hosted build
specification advertises 262,144, which version 1.18.0 registered; version
1.18.1 overrides it for the user-selected trial described above. Its hosted
request schema caps completions at 65,536. The family card documents five effort
levels, and top-level `reasoning_effort` low/medium/high/xhigh/max requests all
returned HTTP 200. The hosted request schema omits this field, so the transport
evidence is recorded separately. Samples returned separate `reasoning_content`;
some simple tasks returned none. These observations do not establish a general
ordering of reasoning depth, and no off/minimal mode is exposed.

[Bolt's API reference](https://docs.api.nvidia.com/nim/reference/nvidia-nemotron-3_5-bolt-e4b-infer)
mentions a `stg/` prefix for standalone requests. Both the public and staged IDs
returned HTTP 200 on the same short off-mode prompt, reporting the public ID in
the response. The extension uses the public ID as listed by `/v1/models`, with
no routing rewrite or fallback. Its schema allows 65,536 output tokens, equal
to its shared context; the generator retains a 32,768 operating allowance.
`chat_template_kwargs.enable_thinking=false` produced zero reasoning characters;
true produced a separate reasoning trace. A specific family selects Pi's native
boolean serializer and prevents a generic Nemotron reasoning budget from being
injected. No unsupported effort ladder is advertised.

Hosted checks used synthetic prompts, no sampling overrides and no automatic
retries. Through installed Pi 1.4.0, both models streamed thinking/text and
completed a forced `record_number(value=7)` call followed by a synthetic tool
result and a normal final answer. The tool had no external effects. MiniMax
also identified red in a programmatically generated 64x64 PNG held only in
memory. Video is documented by the hosted API, but Pi's current chat input
catalog remains text/image; no video adapter is added here. Tool checks do not
establish every tool schema or long-running agent workflow.

One exploratory MiniMax template-low request returned 429; the supported
implementation uses the verified top-level effort field. An invalid top-level
effort received 400. No raw reasoning, API keys, user images, probe artifacts or
backup files are retained. Successful short requests establish allowance
admission, not generation of a full-length response. Assistant reasoning replay
is retained by the serializer; its effect on reasoning quality remains unmeasured.

The other recently listed models (Nemotron 3.5 Parse, Rerank 8B, Embed 1B and
Embed 8B) use parsing, reranking or embedding APIs. An agentic harness can call
them through dedicated tools/adapters, but they are not general chat models to
select for the agent loop. They are not added to this extension by this release.

## Output budgets and Pi 1.3.0 comparison 2026-10-21

Version 1.17.1 raises Step 5 Preview from 4,096 to **131,072** output tokens
and Llama 3.2 11B Vision from 4,096 to **8,192**. Both were accepted through
`https://integrate.api.nvidia.com/v1/chat/completions` with synthetic
`Reply only OK.` prompts, no sampling overrides, a 25-second timeout and no
retries. Step returned HTTP 200 in 1.23 seconds; Llama returned HTTP 200 in
0.58 seconds. Both stopped normally with short answers. These checks establish
admission of the requested allowance, not completion of a full-length response.
The generator regenerated only these two rows; all other metadata is preserved.

[Step's hosted reference](https://docs.api.nvidia.com/nim/reference/stepfun-ai-step-5-preview-infer)
publishes a 4,096 default and a 1,048,576 parameter maximum across the combined
context. Its always-on reasoning can consume a small allowance before the
answer. The extension's 131,072 is a chosen operating budget, not a new claim
about NVIDIA's maximum. [Llama 11B's reference](https://docs.api.nvidia.com/nim/reference/meta-llama-3_2-11b-vision-instruct-infer)
publishes an 8,192 maximum on the model-specific endpoint; the shared-route
check confirms that the extension can use this allowance too.

The installed `@earendil-works/pi-ai` and `pi-coding-agent` versions were both
**1.3.0**. The built-in `nvidia` provider reads generated catalog data from
`dist/providers/data/nvidia.json`, rather than querying NVIDIA for current
limits during a request. Its catalog contains 20 chat models: 13 shared with
this extension, seven built-in-only and three extension-only. Counts describe
this installed snapshot, not all current upstream releases.

| Model | Built-in Pi 1.3.0 output allowance | Extension 1.17.1 |
|---|---:|---:|
| Step 5 Preview | Absent | 131,072 |
| Llama 3.2 11B Vision | 4,096 | 8,192 |
| Llama 3.2 90B Vision | 8,192 | 8,192 |
| Nemotron 3.5 Super VL | 8,192 | 131,072 |
| Nemotron 3.5 Lightning | 262,144 | 32,768 |
| Nemotron 3 Ultra | 65,536 | 32,768 |
| GPT-OSS 20B | 32,768 | 4,096 |
| DeepSeek V4.1 Flash | 384,000 | 262,144 |
| Muse Glimmer 30B | 131,072 | 131,072 |

Both providers use Pi's built-in `openai-completions` implementation. In the
normal `completeSimple`/`streamSimple` path, `buildBaseOptions` starts with
`options.maxTokens ?? model.maxTokens`, then clamps it against the configured
context window minus estimated input tokens and a 4,096-token safety reserve.
The OpenAI serializer sends the result as `max_tokens` for NVIDIA. A smaller
explicit caller cap wins. A larger explicit cap can also exceed the catalog's
`maxTokens` if it fits the context: this path does not enforce the catalog
value as an independent hard server ceiling. The endpoint remains responsible
for validating its actual limit. Direct low-level streaming callers do not
necessarily go through this simple-options clamp.

Consequently, a bundled catalog value is neither a live capability probe nor
proof of a hosted maximum. Keep our hosted evidence where it differs. For
example, Super VL's bundled context is 1,048,576, while its current hosted build
configuration is 262,144; the extension retains the latter. Muse Glimmer's
embedded build schema and API reference disagree about its output ceiling, so
its existing allowance is retained pending focused hosted verification.

No temperature, top-p or other chat sampling defaults are introduced. With no
caller/model sampling settings, these fields are omitted and NVIDIA chooses
its defaults. Explicit caller settings continue through the serializer, so a
future parameter-control extension can supply them without changing this
provider. The unrelated fixed Nemotron reasoning-budget behavior identified in
the audit is not changed by this output-allowance update.

Upstream implementation references (moving `main`; the numeric table above is
from the locally installed 1.3.0 snapshot):

- [NVIDIA provider registration](https://github.com/earendil-works/pi/blob/main/packages/ai/src/providers/nvidia.ts)
- [Generated NVIDIA catalog loader](https://github.com/earendil-works/pi/blob/main/packages/ai/src/providers/nvidia.models.ts)
- [Simple request options and context clamping](https://github.com/earendil-works/pi/blob/main/packages/ai/src/api/simple-options.ts)

Older sections below remain dated historical snapshots. In particular, the
October 17 record's Step 5 4,096 budget was superseded by version 1.17.1.

## Catalog audit 2026-10-17

Version 1.15.0 keeps 16 chat models: add Step 5 Preview and Nemotron 3.5 Super
VL, remove Mistral Nemotron and Kimi K3. An outage, timeout, missing account
entitlement, or generic 404 is **inconclusive**, never retirement evidence.
The existing-model sweep made one 4-token request per model with a 30-second
timeout and no retries. Four responded, two explicitly retired, ten were
inconclusive. These observations do not prove full conversation, tools or vision.

| Existing model | Observation | Action |
|---|---|---|
| DeepSeek V4.1 Flash | 30 s timeout | Keep |
| DiffusionGemma | HTTP 200 | Keep |
| Gemma 4 31B | 30 s timeout | Keep |
| Llama 3.2 11B Vision | HTTP 200 | Keep |
| Llama 3.2 90B Vision | 30 s timeout | Keep |
| Muse Glimmer | 30 s timeout | Keep |
| Mistral Nemotron | Explicit HTTP 410; card notice September 27, unsupported after September 28 | Remove |
| Kimi K3 | Explicit HTTP 410; card notice October 14, unsupported after October 15 | Remove |
| Nemotron 3 Nano Omni | HTTP 503 | Keep |
| Nemotron 3 Super | HTTP 200 | Keep |
| Nemotron 3 Ultra | HTTP 503 | Keep |
| Nemotron 3.5 Lightning | 30 s timeout | Keep |
| GPT-OSS 20B | HTTP 200 | Keep |
| Laguna XS 2.1 | 30 s timeout | Keep |
| GLM-5.3 | 30 s timeout; future notice October 27, unsupported after October 28 | Keep until effective retirement |
| GLM-5.3 Flash | 30 s timeout; no past retirement established | Keep |

NVIDIA notice sources: [Mistral Nemotron](https://build.nvidia.com/mistralai/mistral-nemotron),
[Kimi K3](https://build.nvidia.com/moonshotai/kimi-k3), and
[GLM-5.3](https://build.nvidia.com/z-ai/glm-5-3). The generator's explicit
retired-ID list prevents both removed entries from returning during a refresh.
It does not blacklist GLM-5.3's future date or any inconclusive model.

### New chat models

- [Step 5 Preview](https://build.nvidia.com/stepfun-ai/step-5-preview): NVIDIA
  hosted configuration is 1,048,576 context tokens; text/image input. Its
  request maximum spans that combined window, so the extension uses the
  hosted 4096-token default as a conservative output budget. The hosted
  checkpoint always reasons; `high` is the sole visible on level, no effort
  parameter or disable switch. Streaming returned HTTP 200 with separate
  reasoning and final-answer deltas (8.5 s). Preserve assistant reasoning
  during replay per the hosted schema. Tool/vision support is documented,
  not live-tested by this probe.
- [Nemotron 3.5 Super VL](https://build.nvidia.com/nvidia/nemotron-3.5-super-vl-120b-a12b):
  the hosted build configuration is 262,144 context tokens, although the
  upstream architecture card describes 1M. Hosted output maximum is 131,072.
  Text/image input and exact off/low/high controls; playground snippets
  translate these to `enable_thinking` and `low_effort`. All three streamed
  HTTP 200; off yielded 0 reasoning characters, low 175 and high 276 on one
  short prompt. Those samples verify transport/on-off behavior, not a general
  depth ordering. Do not inherit generic Nemotron's 32768 reasoning budget.
  Video and complete tool round trips are outside this extension's new
  verification; Pi's catalog declares text/image input only.

Local reports: `tools/output/refresh-2026-10-17/chat-retirement-audit.json`,
`public-card-audit.json`, and `new-chat-probes.json`. They contain scoped
status/count/settings evidence, not API keys or generated reasoning text.
Older sections below remain dated historical snapshots.

## Comparison with Pi's built-in `nvidia` provider

**Historical comparison snapshot (Pi 0.85.1), not the supported runtime baseline.** That bundled provider exposed 20 NVIDIA models in the comparison (21 at runtime, which additionally listed `z-ai/glm-5.3-flash`), versus this extension's 16. Counts below describe that snapshot. Generate an updated comparison against your installed Pi with:

```bash
npm run compare:pi -- \
  --json-output=tools/output/pi-nvidia-compare.json \
  --markdown-output=tools/output/pi-nvidia-compare.md
```

That historical report contains:

- 11 shared model IDs
- 9 official-only models
- 5 extension-only models
- 9 shared models with at least one parameter or compatibility difference
- 2 exact shared matches: both Llama 3.2 vision models

| Aspect | Built-in `nvidia` | This extension `nvidia-nim` |
|--------|-------------------|-----------------------------|
| Models | 20 curated | 16 curated |
| Shared catalog | 11 models | 11 models |
| Thinking control | Basic metadata; DeepSeek has a native `deepseek` map, but no NIM-specific Kimi, GLM, or Nemotron transforms | Family-specific thinking maps and request transforms |
| Compatibility baseline | `supportsStrictMode: false`, `supportsLongCacheRetention: false`, `supportsStore: false` | Same baseline on every model, plus NIM-specific flags |
| Request normalization | No | Content arrays, `max_tokens`, reasoning replay, and model-specific extras |
| API key | `NVIDIA_API_KEY` | `NVIDIA_NIM_API_KEY` with `NVIDIA_API_KEY` fallback |

The comparison includes headers, reasoning/input declarations, cost, context window, output limit, reasoning budget, thinking-level maps, example request extras, and every compatibility key. Provider-level fields such as `api`, `provider`, and `baseUrl` are intentionally reported separately rather than treated as per-model differences.

The remaining differences are not all defects. Pi's official catalog supplies its own limits and prices for models such as Kimi K3, Nemotron 3 Super/Ultra, and GPT-OSS 20B. The extension retains card-derived or probe-verified values where the hosted NIM request contract differs. It also adds `reasoningBudget`, `thinkingFormat`, `requiresReasoningContentOnAssistantMessages`, `supportsUsageInStreaming`, and `exampleRequestExtra` fields that the official catalog does not carry.

Official-only models are Gemma 3 4B/12B, Mistral 7B Instruct v0.3, Kimi K2.6, Cosmos Reason2 8B, and the two legacy Llama 3.1 Nemotron entries — every one of them answers 404 "Function not found for account", so the built-in catalog advertises entries that cannot be called. Extension-only models are DeepSeek V4.1 Flash, DiffusionGemma 26B, Gemma 4 31B, and Mistral Nemotron. The comparison reads pi-ai's bundled catalog snapshot, which now also carries `z-ai/glm-5.3` and `z-ai/glm-5.3-flash`, so those two count as shared.

Use `nvidia-nim/...` when you need the NIM-specific thinking transforms and compatibility flags. Use `nvidia/...` for Pi's native handling of its curated catalog.

### Models with thinking support

DeepSeek V4.1 Flash, GLM-5.3 and GLM-5.3 Flash, Kimi K3, Muse Glimmer, DiffusionGemma, Nemotron (3-Nano Omni, 3-Super, 3-Ultra, 3.5 Lightning), GPT-OSS 20B, and Laguna XS 2.1.

- GLM-5.3 and GLM-5.3 Flash keep thinking permanently on — both build cards state the generation prompt opens a think block unconditionally, and live requests with `enable_thinking: false` or `thinking: {"type":"disabled"}` still returned `reasoning_content`. The effort ladder is `low` / `high` / `max` (default `max`; any other value falls back to `max`), so pi offers exactly those three levels: `off`, `minimal`, `medium`, and `xhigh` map to `null` and are hidden rather than aliased onto a neighbouring effort. `clear_thinking` defaults to `false` in the chat template, so the extension injects `chat_template_kwargs.clear_thinking: true` for chat turns.
- DeepSeek V4: live NIM requests confirmed content-only non-think and separate `reasoning_content` for high and max via `chat_template_kwargs`. Pi exposes only `off`, `high`, and `max` for these models; `reasoning_effort` travels inside `chat_template_kwargs`.
- The extension ships one DeepSeek V4 endpoint: `deepseek-ai/deepseek-v4.1-flash`. The older IDs are all gone — the unsuffixed `deepseek-v4-flash`/`deepseek-v4-pro` and `deepseek-ai/deepseek-v4-pro-0813` reached end of life on 2026-08-07 and 2026-09-14, and `deepseek-ai/deepseek-v4-flash-0731` (shipped by 1.8.1) followed on **2026-09-21**: it now answers `410 Gone` with that date and has dropped from `/v1/models`. Its build card still returns 200, as all three earlier ones do, so only the live sweep proves liveness.
- DeepSeek V4.1 Flash (`deepseek-ai/deepseek-v4.1-flash`, build-listed 2026-09-18) is a text/image Mixture-of-Experts release with a **combined** 1,048,576-token input+output window, OpenAI-format tool calls, and a 262,144-token output cap — the card's own `max_tokens` default. The card publishes `max_tokens.maximum` = 1,048,576, i.e. the whole context; the extension ships the documented default instead, because prompt and completion share that window — a cap equal to it is a parameter bound, not an output budget. Although the release declares no Jinja chat template and its documented schema lists no thinking parameter, the hosted endpoint implements the same `chat_template_kwargs` protocol as V4 Flash: probes on 2026-09-25 showed `thinking: false` suppressing reasoning entirely (0 reasoning chars vs 74 for the unparameterised baseline), `reasoning_effort: "max"` deepening it (122), and `"high"` matching the default, so pi offers the same off/high/max ladder. Top-level `reasoning_effort` — string `"none"` and numeric `100`, DeepSeek's documented 1–100 reference encoding — answered `504` with no usable body. Expect heavy queueing: `/v1/models` answers in ~200 ms, while an accepted completion waited ~198 s for its first byte; when the queue is saturated the gateway answers `504` after roughly 300 s with an empty body, which is admission pressure rather than a parameter problem (requests with a 1,024-token cap fail the same way at the same moment).
- GLM-5.3 Flash has its own build card now (`build.nvidia.com/z-ai/glm-5-3-flash`, published mid-September 2026), which confirms text/image input up to 8 images per request, structured output, OpenAI-format tool calls, and the same 1,048,576-token context and `low`/`high`/`max` ladder as GLM-5.3. Earlier it was shipped from live probes alone, and during that rollout roughly one request in three answered `404 Function ...: Not found for account` — those 404s did not reproduce on 2026-09-17. Both GLM endpoints are currently slow rather than unavailable: re-probes measured 57–123 s to first byte with occasional 150 s+ stalls, so retry long before concluding the endpoint is down (pi does not retry 404).
- Retired on the hosted endpoint: Step-3.7 Flash (2026-08-28), Nemotron 3 Nano 30B (2026-09-01), GPT-OSS 120B (2026-09-03), MiniMax M3 (2026-09-09), DeepSeek V4 Pro 0813 (2026-09-14), DeepSeek V4 Flash 0731 (2026-09-21). All six answer `410 Gone` with an explicit end-of-life date, but their build-page cards still return 200, so the live aliveness sweep — not the card — is the liveness signal.
- Kimi K3 (`moonshotai/kimi-k3`) is the newest Moonshot model on NIM: text/image input, 1,048,576-token context (NVIDIA's official card value), 65,536-token output, OpenAI-format tool calls, a boolean `chat_template_kwargs.thinking` toggle plus card-documented `reasoning_effort` levels (`low` / `high` / `max`), and separate `reasoning_content`. The card became officially listed on the build page on 2026-08-28; before that the extension carried it from live probes alone. **Practical warning: it is near unusable at times** — probe latency ranged from 1 s to 46 s for the same request and the free-tier endpoint repeatedly rate-limits (429) in bursts, so expect intermittent multi-minute-feeling turns; treat it as a capacity-constrained endpoint.
- Muse Glimmer 30B supports text and image input with a 131,072-token context. Hosted NIM accepts top-level `reasoning_effort` and streams separate `reasoning_content`; `none` was accepted but still produced reasoning in live probes.
- Nemotron 3.5 Lightning 30B has a 1,048,576-token context with text input. NVIDIA documents no `reasoning_effort`; thinking is toggled via `enable_thinking` with a top-level `reasoning_budget` (default 16384, max 32768). Live probes confirmed `enable_thinking: false` and `reasoning_effort: none` both stop reasoning, and tool calls work.