# @llmgates_api/pi-llmgates-provider **English** · [简体中文](./README.md) A pi provider extension that connects several **OpenAI-compatible gateways** in parallel — [NewAPI](https://github.com/QuantumNous/new-api), [Sub2API](https://github.com/Wei-Shaw/sub2api), [CLIProxyAPI](https://github.com/router-for-me/CLIProxyAPI), and any generic gateway that implements `GET /v1/models`. Each gateway instance is its own pi provider: it discovers models from `/v1/models`, registers them with pi, and routes each model to the matching inference endpoint. Reference implementation: [@router-for-me/pi-cliproxyapi-provider](https://pi.dev/packages/@router-for-me/pi-cliproxyapi-provider) > The extension's in-product UI (login prompts, pickers) is in Chinese. Literal UI labels are quoted verbatim below, e.g. the `/login` entry is **「LLMGates 网关」**. ## Contents - [Feature overview](#feature-overview) - [Quick start](#quick-start) - [Installation](#installation) - [Command reference](#command-reference) - [Supported gateways](#supported-gateways) - [Adding and managing instances](#adding-and-managing-instances) - [Models and inference endpoints](#models-and-inference-endpoints) - [Usage and cost](#usage-and-cost) - [Upstream response model audit](#upstream-response-model-audit) - [Input history](#input-history) - [Remembering the last used model and thinking level](#remembering-the-last-used-model-and-thinking-level) - [Configuration](#configuration) - [Security](#security) - [Troubleshooting](#troubleshooting) - [Upgrading from 0.2.13 and earlier](#upgrading-from-0213-and-earlier) - [Development and release](#development-and-release) - [Related documents](#related-documents) - [License](#license) ## Feature overview - **One login entry** — 「LLMGates 网关」 in `/login` adds NewAPI / CLIProxyAPI / Sub2API / generic gateway instances; any number of them can coexist. - **One provider per instance** — credentials are validated and the catalog fetched through `GET /v1/models`; models are disambiguated by provider ID in `/model`. - **Per-model endpoint routing** — the gateway's own `inference_endpoint` / `web_chat_endpoint` wins; models that declare nothing go to OpenAI Chat Completions, and any model can be overridden to `messages` / `responses`. Image/video generation models are not registered. - **Balance lookup** — `/balance` probes each instance's quota, and reports *not available* rather than `0` when the gateway exposes no usable endpoint. - **Usage and cost tracking** — TUI status line plus `/calls` breakdown, covering the parent session and both sync and async subagents; cost is estimated from upstream retail rates. - **Upstream response model audit** — compares the model sent with the model the gateway's response reports; a different series adds a red `.xN` to the status line and `/model-audit` shows the details (on by default; records only, never blocks). - **Persistent input history** — what ↑↓ walks through survives across pi processes (on by default, one file per working directory); `/input-history` manages the switch, the scope and clearing it. - **Remembers the last used model and thinking level** — a fresh session starts back on the model *and* the thinking level you used last, instead of the first entry of the `/scoped-models` list (on by default). ## Quick start ```bash # install pi install npm:@llmgates_api/pi-llmgates-provider pi /login ``` Pick **「LLMGates 网关」** in `/login`, then choose the gateway type and fill in the URL and API key. Run `/reload` or restart pi after installing or updating so the extension takes effect. ## Installation **Requirements:** [pi](https://pi.dev), Node **≥ 22.19**, and `@earendil-works/pi-coding-agent` / `@earendil-works/pi-ai` **≥ 0.81.0, < 0.87.0** (baseline 0.81.1 — tests and typecheck run against that version; 0.82.1, 0.83.0, 0.84.0 and 0.84.3 are also verified). The 0.7.1 release gate additionally exercised **pi 0.86.0**: extension load, all seven commands, catalog refresh across two gateways, DeepSeek inference (thinking off/high), the status-line ledger and its quality markers, the three `/calls` views, and subagent usage attribution. `/endpoint`, `/balance`, `/logout`, cross-process input history and the full `restoreLastModel` matrix were **not** re-verified on 0.86.0 — those still rest on the 0.84.3 run. **0.8.0** re-used pi 0.86.0 with the unpacked tarball for `/model-audit` (three-API mismatch and `response≥1`, toggles and equivalents, clear); subagent / lite attribution and the peer 0.81.0 floor were not certified in that 0.8.0 install-package gate. The **0.8.1 pre-release gate** used Node **22.19.0** and installed the unpacked pre-bump `0.8.0` tarball into isolated **pi 0.81.0 / 0.86.0** environments: managed re-login, eight unsuffixed commands, `/reload`, and the 0.86.0 model picker and streaming inference. The 89 loopback turns covered all six GLM thinking levels, cross-vendor alias precedence, three APIs, session restart, and cached-vendor recovery with `compat` removed. This verifies request/cache contracts, **not every real upstream model's capabilities**, and does not extend existing `/model-audit`, subagent / lite, or other feature certification. This extension uses the **native Provider** API and does **not** support pi 0.80.x. ### npm ```bash pi install npm:@llmgates_api/pi-llmgates-provider # first install (latest) pi install npm:@llmgates_api/pi-llmgates-provider@0.8.2 # pinned version pi install -l npm:@llmgates_api/pi-llmgates-provider # this project only (otherwise ~/.pi/agent/) ``` **Upgrade with `pi update`.** A plain `pi install` with no version does **not reliably** upgrade an existing copy: inside `~/.pi/agent/npm/` pi runs `npm install `, and npm resolves that against the `^` already saved in that `package.json`. This extension is still 0.x, so `^0.5.0` only covers `0.5.x` — a patch release is picked up, a minor bump (0.5 → 0.6) is not, and both cases print `Installed`: ```bash pi update npm:@llmgates_api/pi-llmgates-provider # move to the latest release (not held back by that range) ``` `pi install …@x.y.z` does install that exact version, but it also **pins** the entry in `settings.json`; from then on `pi update` skips the package — **it still prints `Updated` while the version stays put**. To follow latest again: run the version-less `pi install` once to drop the pin, then `pi update`. ### From source / local development ```bash git clone https://github.com/ax128/pi-llmgates.git cd pi-llmgates npm install npm run build # compile extensions/ -> dist/ pi install . # one-off run without touching the global config pi -e npm:@llmgates_api/pi-llmgates-provider ``` > **`pi install git:…` is unsupported as of 0.2.7.** The published artifact is the compiled `dist/` (not committed to the repo), and pi's git install only runs `npm install --omit=dev`, so it never gets `dist/` and the extension silently fails to load. Install from source with `pi install .` above (run `npm run build` first). ## Command reference | Command | Description | | --- | --- | | `/login` | Pick 「LLMGates 网关」 to add an instance | | `/login ` | Reconfigure an existing instance's base URL and API key (pick the oauth login item when the auth-method chooser appears; the "Sign in with an API key" item only reports that credentials are managed) | | `/logout` | Select an instance's display name in the picker to delete it (type the instance ID to search) | | `/model` | Choose a registered gateway model (disambiguated by provider ID, e.g. `grok-4.5 [work-newapi]`) | | `/balance [instance-id]` | Query gateway quota (all instances when no argument is given) | | `/endpoint [model-id]` | Switch or clear the inference endpoint of **one** model | | `/endpoint-setting` | Interactive multi-select to switch endpoints in bulk across instances | | `/calls` | Per-model usage and cost for this turn or session, plus coverage and read-only reconciliation snapshots | | `/model-audit` | Mismatches between the model sent and the series the upstream response reports, for this working directory, plus this session's All / Turn counts | | `/model-audit clear` | After confirmation, clears this working directory's audit records **and the counts of every session in it** | | `/input-history [status]` | Show the input-history switch, scope, file path and how much is stored | | `/input-history on\|off` | Enable or disable persistent input history (writes `config.json`, effective immediately in this pi process) | | `/input-history scope ` | Switch scope: one file per working directory (default) or one shared by all of them | | `/input-history clear` | Delete the current scope's history file and empty this process's in-memory list | | `/input-history help` | Show usage and a summary of what gets recorded | | `/llmgates list` | List instance ID, scheme, base URL and display name (never the key) | | `/llmgates remove ` | Delete an instance along with its registry / auth / endpoint-override records | | `/llmgates help` | Show usage and known limitations | | `/llmgates-reload` | Force-refresh the model catalog of every instance (bypasses the freshness window, rewrites cached thinking levels and other metadata) | | `/reload` | Reload extension code after installing or updating (does **not** refresh the catalog) | ## Supported gateways | Gateway | scheme | Typical use | Source | | --- | --- | --- | --- | | [NewAPI](https://github.com/QuantumNous/new-api) | `newapi` | Self-hosted model aggregation and channel management | [QuantumNous/new-api](https://github.com/QuantumNous/new-api) | | [Sub2API](https://github.com/Wei-Shaw/sub2api) | `sub2api` | Subscription quota distribution and multi-account routing | [Wei-Shaw/sub2api](https://github.com/Wei-Shaw/sub2api) | | [CLIProxyAPI](https://github.com/router-for-me/CLIProxyAPI) (CPA) | `cpa` | Local CLI subscription proxy, default port `8317` | [router-for-me/CLIProxyAPI](https://github.com/router-for-me/CLIProxyAPI) | | Generic OpenAI-compatible gateway | `default` | Any gateway implementing `GET /v1/models`; no specific product assumed, multiple allowed | — | - Multiple instances of the same scheme are fine (e.g. `work-newapi` and `home-newapi`, or two `default` instances on different hostnames), and different schemes coexist. - For `default`, leaving the instance ID blank derives it from the URL hostname; a repeated hostname gets a `-2`, `-3` suffix. - The base URL does not have to include `/v1`; the extension normalizes it before probing `/v1/models`. - **All schemes share one endpoint mapping.** When the gateway declares `inference_endpoint` / `web_chat_endpoint` per model in `/v1/models`, that declaration is used; anything that declares nothing goes to OpenAI Chat Completions. The protocol is never guessed from the scheme or the model name. Override explicitly with `/endpoint` or `/endpoint-setting` — see [Models and inference endpoints](#models-and-inference-endpoints). > Upgrading from 0.2.13 or earlier and previously using the built-in official gateway? See [Upgrading from 0.2.13 and earlier](#upgrading-from-0213-and-earlier). ## Adding and managing instances ### Adding an instance ```bash pi /login ``` Menu path: `/login` → Sign in with an account → **「LLMGates 网关」** → choose the gateway type (**NewAPI** → **CLIProxyAPI** → **Sub2API** → **通用网关**, the generic gateway). The prompt order is **identical for all four gateway types**: instance provider ID → display name (blank uses the ID) → base URL → API key. The only difference is that the **generic gateway (`default`)** allows a blank instance ID. | Field | Description | | --- | --- | | scheme | Only drives the label and URL placeholder hint; the placeholder is **not** a default value | | Instance ID | Used by `/login `, `/model` and `/llmgates remove`; 1–64 characters, starting with a letter or digit, may contain `.` `_` `-`. Derived automatically when left blank for `default`; required for the other three | | Base URL | Must be complete, usually ending in `/v1` | | API key | Must be entered explicitly; stored in `auth.json` as a literal string — `!cmd`, `$ENV` and `${...}` are never expanded | After a successful add: - The instance's models are registered immediately and selectable via `/model`. - The **API key** is stored as a literal string in pi's `auth.json` (as an OAuth credential); instance metadata (ID, display name, scheme, base URL) goes to `~/.pi/agent/llmgates/2api.json`. - A message containing the instance ID is left in the session (for a `default` instance with a blank ID, that message — or `/llmgates list` — is the only place to read the derived ID). Credential validation makes at most **5 attempts** (covering invalid URLs and network/HTTP/JSON errors) before the login is aborted. Remote HTTP is rejected; you can correct it to HTTPS or loopback HTTP within those 5 attempts. The instance registry is written to `~/.pi/agent/llmgates/2api.json`. Both it and `auth.json` are written with mode `0600`, guarded by a cross-process file lock, an in-lock re-read, and atomic replacement. ### Removing an instance - `/llmgates remove ` — deletes the instance along with its registry / auth / endpoint-override records. - `/logout` — select the instance's display name in the picker and pi deletes the `auth.json` credential; this extension watches that file and asynchronously drops the matching registry record, stops the provider, and removes the endpoint override. The instance is not retained as a restorable config; if the watcher cannot start — or starts but silently misses the event — a low-frequency reconciliation every 60s finishes the cleanup within one interval, and `/reload` or a restart still triggers one right away. ### Per-gateway walkthroughs #### NewAPI 1. Deploy an instance following the [NewAPI docs](https://docs.newapi.pro/en/docs) (Docker or binary). 2. Create an API key in the NewAPI console and confirm `GET /v1/models` is reachable. 3. In pi, run `/login` → **「LLMGates 网关」** → **NewAPI**, then fill in: - Instance ID: e.g. `work-newapi` - Display name: e.g. `Work NewAPI` (optional) - Base URL: e.g. `https://your-newapi-host/v1` - API key: the key issued by the console 4. Confirm with `/llmgates list`, then pick a model with `/model`. #### Sub2API 1. Deploy following `deploy/` in the [Sub2API repository](https://github.com/Wei-Shaw/sub2api) (the service port is commonly `8080`). 2. Generate an API key in the Sub2API admin console. 3. In pi, run `/login` → **「LLMGates 网关」** → **Sub2API**, then fill in: - Instance ID: e.g. `team-sub2api` - Base URL: e.g. `https://sub2api.example.com/v1` (locally `http://127.0.0.1:8080/v1`) - API key: the key generated in the console 4. Pick a model with `/model` and start chatting. #### CLIProxyAPI (CPA) 1. Start the local proxy following the [CLIProxyAPI README](https://github.com/router-for-me/CLIProxyAPI) (listens on `http://127.0.0.1:8317` by default). 2. After the CLI OAuth login, confirm `GET http://127.0.0.1:8317/v1/models` returns a model list. 3. In pi, run `/login` → **「LLMGates 网关」** → **CLIProxyAPI**, then fill in: - Instance ID: e.g. `local-cpa` - Base URL: `http://127.0.0.1:8317/v1` (loopback HTTP is allowed) - API key: whatever your CPA instance is configured with (must be non-empty; if the gateway has no bearer auth, follow your deployment) 4. Pick one of the models CPA exposes with `/model`. ### Balance lookup There is no standard balance API, so `/balance` probes each instance in this order: 1. `GET {baseUrl}/dashboard/billing/subscription` + `/dashboard/billing/usage` (the OpenAI-compatible billing endpoints of NewAPI / one-api) → shows `remaining / total` 2. `GET {baseUrl}/user/balance` → reads the balance field out of the payload When neither is available (CLIProxyAPI, for instance, does no billing at all), the output says *balance is not available from this gateway* — never `0`. A gateway that falls unmatched routes back to its frontend page (200 + HTML, the one-api default) also counts as "does not offer this endpoint" and probing continues; only timeouts, aborts and network errors are reported as errors. Only currency-denominated fields are read (`balance` / `remaining` / `remaining_usd` / `credit` / `credits`, with the unit taken from `unit` or `currency`, defaulting to USD). one-api's internal quota units (`quota` / `remain_quota`, where 500000 = 1 USD by default) are **not** treated as an amount — showing "not available" beats showing a number five orders of magnitude too large. For those gateways the real balance comes from the billing endpoints in step 1. ### Known limitations - Pi's `/logout` offers no extension cleanup callback; this extension watches `auth.json` for changes to clean up the registry, provider and endpoint override of a logged-out instance. The watcher is the immediate path; when it cannot start — or starts and then silently misses events, which Node does not rule out on network filesystems and some mounts — a low-frequency reconciliation every 60s is the backstop. It only compares file metadata (it never reads or parses the credentials) and triggers the same cleanup when that metadata changes. `/reload` or a restart still triggers one right away, but is no longer the only way to recover. The instance is not kept as a restorable config. - If `auth.json` is missing entirely or temporarily corrupt (a manual credential reset, a sync tool mid-write), that cleanup round is skipped so instances are not wrongly deleted; cleanup resumes once the file is readable again. - After `/llmgates remove `, the instance's models disappear immediately; because of pi extension API limits, `/logout` may still briefly list the removed ID until `/reload`. - Orphan auth keys in `auth.json` with no matching registry record cannot be handled by `/llmgates remove`; delete the corresponding ID entry from `~/.pi/agent/auth.json` manually. - If `~/.pi/agent/llmgates/2api.json` cannot be parsed (a hand-editing mistake, a duplicate instance ID, …), the extension registers **no providers and no gateway commands** — including the 「LLMGates 网关」 entry in `/login` — and pi shows no hint at all. The startup log prints the exact reason (with the file name); fix or delete the file and run `/reload` to recover. (`/input-history` and remembering the last used model and thinking level have nothing to do with gateways and stay available.) ## Models and inference endpoints ### Model mapping | Gateway field | Pi field | | --- | --- | | `id` | `id` | | `display_name` / `name` | `name` | | `context_window` / `max_model_len` | `contextWindow` | | `max_output_tokens` / `max_tokens` | `maxTokens` | | `capability_tags` (vision) or `input_modalities` | `input`: text + image | | `provider_id` | Vendor hint for pricing and transport compat | Models tagged `image_generation` / `image_edit` / `video_*` are not registered (the coding agent cannot drive them) and never show up in `/model`. The inference endpoint comes from the gateway's own `inference_endpoint` / `web_chat_endpoint`; when the gateway declares nothing (or a value that cannot be recognized) it is `chat_completions`, and it can be overridden per model: | endpoint value | pi `api` | | --- | --- | | `chat_completions` | `openai-completions` | | `messages` | `anthropic-messages` | | `responses` | `openai-responses` | **Priority: per-model override > `defaults` > the gateway's `inference_endpoint` / `web_chat_endpoint` > `chat_completions`** (`inference_endpoint` beats `web_chat_endpoint`; only the three values above and their aliases are recognized, anything else is ignored and falls back to `chat_completions`). No id-shape heuristic is used: a model the gateway says nothing about still goes to `chat_completions`. ### Reasoning effort The extension applies the **same fixed level map to every** model selectable in `/model` and **passes it through verbatim** to upstream — no remapping, no reading of the gateway's `supported_reasoning_levels`, no pi-ai built-in sparse map: | pi level | effort sent upstream | | --- | --- | | `off` | `none` | | `low` | `low` | | `medium` | `medium` | | `high` | `high` | | `xhigh` | `xhigh` | | `max` | `max` | `minimal` is not in the universal map (disabled with `null`) — the vast majority of models, Claude included, have no such level. If a specific OpenAI model needs it, enable it individually via `modelOverrides` below. Every model has `reasoning: true`, so the selector always exposes the levels above. If upstream does not support a level or returns 400, lower it or switch model yourself; the extension does not clamp or remap on your behalf. **Transport-layer compat** is still inherited from pi-ai's exact metadata (Anthropic `forceAdaptiveThinking`, `supportsTemperature: false`, …), which only affects request shape, never the effort string. Restoring from the disk cache also rewrites to the universal map above (a remap from an old cache is not preserved). On top of that, **Moonshot / Kimi** models routed through a gateway lose pi-ai's URL-based compat detection, so the extension supplies that transport compat itself — keyed on the vendor (`moonshotai` / `moonshot` / `kimi-coding`, …) or on an id starting with `kimi-` / `moonshot`. The load-bearing field is `supportsDeveloperRole: false`; without it Moonshot answers `tokenization failed`. This too **only affects request shape**: it changes neither the endpoint choice nor the effort string, so it does not contradict "the protocol is never guessed from the scheme or the model name" above. A Kimi model routed to `messages` shares none of those fields and is deliberately left unstamped. **DeepSeek** models routed through a gateway likewise lose pi-ai's URL-based detection, so the extension supplies DeepSeek transport compat keyed on the vendor (`deepseek` / `deepseek-ai`) or an id starting with `deepseek-`. This sends the system prompt with the `system` role, uses DeepSeek's `thinking` parameter for reasoning requests, and disables `store`. It changes request shape only, not endpoint selection or the effort string; the known vendor hint for aliases is persisted with the model cache so offline restores retain the same compat. **Z.ai / Zhipu GLM** models routed through a gateway receive request-shape compat keyed on the vendor (`zai` / `zai-coding-cn` / `z-ai` / `zhipu` / `zhipuai` / `bigmodel` / `glm`) or the final id segment (`glm` / `glm-*` / `chatglm*`). The system prompt uses the `system` role, the output limit uses `max_tokens`, and `store` is omitted. For Chat Completions, thinking `off` sends `thinking: { type: "disabled" }` without `reasoning_effort`; other levels send `thinking: { type: "enabled", clear_thinking: false }` and pass the effort through verbatim when the endpoint supports it. Known vendor hints for aliases persist with the model cache for offline restoration. Endpoint selection is unchanged, and models routed to `messages` are not stamped with this OpenAI metadata. When choosing a compatibility family, **recognized DeepSeek, Moonshot/Kimi, or GLM vendor hints take precedence over id heuristics**. Id detection is used only for absent or unrecognized vendor hints, so cross-family alias names do not replace the identified upstream's compat. This GLM compat is **not a complete copy of each native Z.ai model's metadata**: model-specific `supportsReasoningEffort` / `zaiToolStream` flags are not copied. Sending `reasoning_effort` still follows pi-ai's endpoint detection (ordinary OpenAI-compatible gateways default to passing it through); this patch does not opt into the native `tool_stream` extension. This does not certify every GLM model and gateway: which efforts an endpoint accepts still needs verification against its actual behavior. **User-level fine-tuning (pi's own hook):** override a single model's thinking levels through `providers..modelOverrides` in `~/.pi/agent/models.json` (top level, merge semantics — only the keys you write are overridden): ```jsonc { "providers": { "work-newapi": { "modelOverrides": { "gpt-5.6-sol": { "thinkingLevelMap": { "xhigh": null, "max": null } } } } } } ``` `thinkingLevelMap` keys are `off` / `minimal` / `low` / `medium` / `high` / `xhigh` / `max`; values are a `string` (the effort sent to the gateway) or `null` (disable that level). This is pi's built-in model override hook and has nothing to do with `apiKey`. ### Switching one model: `/endpoint` ```text /endpoint [model-id] ``` - With `model-id` omitted, only the current model changes; if the current model does not belong to an instance managed by this extension, the command is rejected and you must name a model ID inside an instance. - An explicit ID is matched exactly across **all instances** — no fuzzy matching. If several instances carry the same model name, the command is rejected and the candidates are listed; select the model with `/model` and rerun without an ID. - `chat` → `openai-completions`, `messages` → `anthropic-messages`, `responses` → `openai-responses`. - `auto` only clears that model's per-model endpoint. If `defaults.endpoint` exists, the model falls back to it rather than skipping straight to the gateway's default. - The command first saves the instance's `~/.pi/agent/llmgates/2api-models/.json` atomically, then force-refreshes the catalog over the network, writes the provider store, publishes and validates it; when the target is the current model it also rebinds to the new object in the registry. Success is only reported when all of that completes. - On pi 0.84, persisting the model cache is owned by pi and keyed by refresh generation: if this write is superseded by a newer refresh, the new catalog is still published and effective for this session (the command still reports success) and only the disk cache is deferred to the next refresh — it will not be overwritten by the older cache in the meantime. - With `PI_OFFLINE`, a network failure, a provider that is not ready yet, a failed store write, or a failed rebind of the current model, a warning is shown: the config is saved but not fully active, so retry the command once you are online; a failed rebind can also be fixed by reselecting via `/model`. - This command has no bulk mode — use `/endpoint-setting` for that. ### Bulk switching: `/endpoint-setting` ```text /endpoint-setting ``` - Two steps: first tick the models to change (multi-select across instances), then choose `chat` / `messages` / `responses` / `auto`. - In the TUI the first step is an interactive checklist: `↑↓` to move, space to tick, `Tab` for the whole group, `Ctrl+A` select all, `Ctrl+D` clear (those three act only on the current filter result while filtering), type to filter (`Backspace` / `Ctrl+U` clears the search), `Enter` to confirm, `Esc` to cancel. RPC mode has no component channel and falls back to a text checklist: change `[ ]` to `[x]` in front of the models you want to change. - Covers the models of **every instance**; cancelling either step, or confirming with nothing ticked, writes no files at all. - The list is grouped by provider and shows "model-id · name · current endpoint". A `*` marks a model that has **its own per-model entry** in the override file; models whose endpoint comes only from `defaults.endpoint` are not marked (choosing `auto` on those has no per-model entry to clear). Models from third-party extensions and pi's built-in providers have no `api` write channel, so they are disclosed in the summary but cannot be ticked; writing those IDs into the text checklist by hand is rejected with an explicit reason. - Requires an interactive interface: TUI and RPC modes work; `print` / `json` modes suggest `/endpoint` instead — no error, no file written. - Each provider is locked once, written once and refreshed once, with groups processed serially. - Three-state result: all-success is info; **written but not activated (offline / provider not ready / superseded by a newer refresh / some models did not take effect / the current model failed to rebind) is always a warning**, never a false success; only a write failure on **every** provider is an error. (On pi 0.84, "pi owns persistence and the write was superseded" does not count as not-activated — the catalog is already published for this session and only persistence is deferred; see the previous section.) On partial success across providers, the status is reported per provider; whatever succeeded stays in effect and is not rolled back. - Whether upstream supports `messages` / `responses` depends on your own gateway deployment — this extension neither probes nor blocks. If you pick wrong, choose `auto` in `/endpoint-setting` or run `/endpoint auto `. ### Force-refreshing the catalog: `/llmgates-reload` ```text /llmgates-reload ``` - Force-refreshes the model catalog of **every instance**, bypassing the background freshness window; it fetches `/v1/models` over the network and writes each provider store (including thinking levels and other metadata). - Takes no arguments. Unlike `/reload`, which only reloads extension code and does not refresh the catalog. - Providers run `refreshEndpointForeground()` **concurrently** (each with its own 15s models timeout), so the command takes as long as the slowest one rather than the sum of all timeouts; it waits for the agent to go idle first. - Three-state result: all providers refreshed is **info**; at least one succeeded while the rest were offline / not ready / superseded / threw is a **warning** (the text contains *partial*); **zero** providers refreshed without all of them hard-failing is a **warning** (*did not update any provider*, without *partial*); all providers hard-failing is an **error**. - If the current model's provider refreshed successfully but that model ID is no longer in the new catalog, an extra **warning** asks you to reselect with `/model` (a stale binding is never kept silently). > `/endpoint`, `/endpoint-setting` and `/llmgates-reload` share one in-flight lock: while any of them runs, the others are rejected. The `/endpoint-setting` picker holds that lock the whole time it is on screen (which can be several minutes), so close it first. All three wait for the agent to go idle for at most **120s**; on timeout **no file is written**, the lock is released, and you are asked to retry later. ### Editing the override file by hand Each instance's override lives in **its own file**, `~/.pi/agent/llmgates/2api-models/.json`: ```jsonc { "defaults": { "endpoint": "responses" }, "models": { "gpt-5.6-sol": { "endpoint": "chat_completions" }, "claude-sonnet-4-6": { "endpoint": "messages" } } } ``` - Aliases are accepted: `responses`·`response` / `chat`·`chat_completions`·`chat-completions`·`completions` / `messages`·`message`·`anthropic`. - Instances are isolated from each other: one instance's override never affects a same-named model in another instance. - A missing file (`ENOENT`) means the override is cleared; a valid object replaces the current config; malformed JSON or a malformed root produces a warning and keeps that instance's last-known-good (or no override at all on first load — never one shared from another instance). Other filesystem errors (`EACCES`, `EISDIR`, …) never silently change routing: an explicit refresh fails before requesting the catalog, a background refresh only warns, and old models and cache are kept. Warnings never print the API key, the file contents, or arbitrary underlying error bodies. - A hand edit takes effect on the next successful catalog refresh — no restart needed. Cache-only, `PI_OFFLINE` and a freshness-window skip never remap cached models. Prefer `/endpoint` or `/endpoint-setting`, which trigger a validated foreground refresh. - `/llmgates remove ` deletes the instance's override file too, so recreating an instance under the same ID will not resurrect the old config. A failed delete is reported as partial and does not block the remaining cleanup steps. - **Downgrade note:** going back from 0.2.0 to 0.1.12 leaves non-`openai-completions` models in the provider store cache that the old version's validation rejects, so that instance shows no models **until the first successful online refresh**. The override file is not lost and the old version ignores `2api-models/` — deleting that directory does **not** fix the store; one successful online catalog refresh (or restarting pi) heals it. ## Usage and cost ### Status line and `/calls` The TUI extension status line shows: - While the agent is **running**: only `Turn 17m.19c.~$1.78` (turn elapsed · calls · cost). Parent-session cost is prefixed with `~` when it comes from the local rate table; unknown cost shows `?` rather than a free `$0`. - **After the turn finishes or a cancel settles**: `All(partial) 100c.~$18.10, Turn 30m.20c.~$10.10` (`All` shows calls and cost for confirmed current-session entries and the preserved ledger and **includes this turn's confirmed usage immediately**; `Turn` is the current turn). All no longer shows elapsed time. `partial` discloses recovery in progress or history/identity/budget gaps; it does not replace configuration exclusions or metric-quality markers. The 1s status-line tick stops once the parent settles; late child usage updates the line as each event arrives, without keeping the 1s timer. The tick keeps going, with `↻ 2s` appended, only while the ledger still has a `running` / `provisional` producer. When the exact call count cannot be confirmed (e.g. a subagent result reports tokens but not turns) it is shown as a lower bound `≥N`. The next turn goes back to `Turn` only. A red `.xN` at the end of a segment (e.g. `All(partial) 100c.~$18.10.x3, Turn 30m.20c.~$10.10.x1`) is the [upstream response model audit](#upstream-response-model-audit) mismatch count, unrelated to usage; it is hidden at 0. `/calls` shows the per-model breakdown. This session includes the in-progress turn's confirmed numbers. This turn / This session / Coverage / Reconciliation are all fixed at menu-open time, unaffected by subsequent background updates; live totals stay on the status line. Coverage separately lists the collection start, pending work and configuration exclusions. Opening `/calls` requests a public-entry boundary check from the recovery coordinator; pending recovery suppresses exact residuals without changing an already-open menu. Reconciliation compares plugin All with the raw `cost.total` subtotal from assistant, toolResult, compaction and branch-summary entries in the current Pi session's `getEntries()` snapshot (0.81.1 baseline contract, including other tree branches; not a new cross-version certification). It does not reprice, write observations, scan other sessions or start watchers. Each menu opening parses at most 200 entries / 50ms; the synchronous shallow copy inside `getEntries()` cannot be preempted and may itself block on long sessions. Incomplete scans, invalid costs and unsupported standalone usage entries are explicitly labelled as a Native checked subtotal with partial/unknown status, never a complete Pi total. Pending collection is not drained or used for exact residual decomposition; other differences remain unexplained, not proof of missing usage or a target to match. Scope, local estimates and recorded costs may differ; only stable parent-entry local estimates and proven configuration exclusions are classified. Other differences remain unexplained; incomplete recovery/gaps suppress exact residuals. Unrounded comparisons use `max(1e-9 USD, 1e-9 × max(|a|, |b|))` tolerance. `LLMGATES_TPS=0` does not enumerate history; non-TUI fallback behavior is unchanged. The footer, `/calls` title and model rows use the same metric-quality rules. Missing metrics are not hidden by known values from other records: cost may read `~$0.010 + ?`, tokens `10 + ?`, and an entirely unknown value `?`. Local estimates retain `~`; numeric protocol costs are reported, while legacy costs without provenance stay unknown. All / Turn share the same cost precision; positive costs below $0.0001 use three significant digits (scientific notation if needed) rather than rounding to free. Multi-model `modelAttempts` retain separate model rows; a newer snapshot revision replaces the entire previous model partition set. Recovery uses stable entry/toolCallId identities. Historical or unassigned usage never enters the current Turn; new turn numbers advance beyond restored ones. Limits are 2048 queued tasks, 256 pending associations/30s TTL, and 200 entries / 256KiB / 50ms per slice. Archive merging and projection restoration also check the time budget and owner cancellation, without copying the whole archive for every row. No other session files are scanned. Progress stays out of confirmed All, and model partitions commit atomically; complete checkpoints establish batch durability. Root tool results, including on supported older Pi versions, are finalized only through their public toolResult entries, never once per end event and again per entry. Running root tools retain only ID/launch-turn scalars in a separate map capped at 256; taking longer than 30s does not discard their origin, and overflow reports `active-tool-capacity`. Completion releases that running record and starts the 30s result-to-entry association deadline. Repeated end events still re-arm entry discovery and reset the deadline, without waiting for the next model reply or settle. Only identity/model scalars omitted by Pi are retained temporarily; associated entries and session shutdown release the corresponding state. The entry index keeps the newest 10,000 entries. Once full, full-history backfill and idle probes stop, in-flight associations still complete, and later parent messages and root tools are still counted. Final usage, details and nested evidence always come from the entry. With persistence enabled, existing v1 `coveredCallIds` retain entry/execution associations across model-partition and terminal replacements. Older archives without provable links keep their known subtotal and report `tool-entry-overlap-unresolved` rather than adding a possible alias. With persistence disabled, event-only model attribution and local estimates are not guaranteed to survive reload. Historical subagent/Task results with only a tool-local `tool:` fallback identity and no matching archive or entry link are quarantined as `tool-entry-overlap-unresolved` and **excluded from the confirmed subtotal**, even with persistence off and an empty archive. Otherwise Pi's omission of an event-only run ID lets replay and a late completion/meta become two charges. Public run identities and reliable archive associations still restore normally; ordinary tools' stable `toolusage:` identities are unaffected. Equal amounts are never used to infer duplicates. Persistence is not enabled automatically, and Pi history is not rewritten. Async/foreground completion events validate session ownership before checking each child's metadata separately. The whole event still shares limits of 200 results, 256KiB and 50ms for validation; per-child node, depth and string limits remain unchanged. Ordinary parallel results no longer fail merely because their combined nodes exceed the single-entry limit. A genuinely oversized event is rejected without granting run ownership and immediately marks Coverage and the idle footer `metadata-budget-exceeded` / partial, preserving known subtotals. Validated queued usage batches, including async completions and bg_wait, commit in slices between independent executions instead of hiding the entire child batch inside one synchronous task. Model partitions count toward the shared 200-row / 256KiB / 50ms slice budget; each execution remains atomic, and the combined bytes of several executions may span slices. The whole batch is rejected with `live-queue-overflow` when the queue cannot hold every task, or when one execution cannot fit in an empty slice — without consuming deduplication identities or granting that completion's run trust. A completion with no usage still keeps its identity. Arrival order survives yields, so background meta and later public entries cannot claim the remaining completions first. Views update once per slice, and slicing alone does not mark All partial. Shutdown/reload keeps the same sliced order—queued live completions, final public entries, then background meta—before checkpointing, so late entries cannot claim remaining children first; an invalidated owner stops. The time budget is checked between execution groups, not a promise to preempt a single synchronous commit or file I/O. When a live completion arrives before its launching tool result, it first affects All/unassigned. Once trusted launch evidence arrives, its original turn is backfilled, including separately identified children from session-verified completions. Parent/child links do not require usage in the event: an async completion that passes session and metadata-budget checks may supply only run identities, with spend arriving later through `_meta.json`, and still retain its proven launch turn. Without parent-run launch evidence it remains unassigned, never borrowed from the current turn. Only origin and local receipt revision change, not cost, call identity or model partitions. Confirmed origins are never overwritten, and historical/restored records cannot claim a new turn this way. Pending parent-child links and unassigned execution indexes are each capped at 10,000 and belong only to the current collector; this enables no extra persistence and changes no v1 fields. Backfill runs asynchronously through the recovery coordinator, sharing its 200-row / 50ms slice budget. Model partitions count toward the row limit and each execution commits atomically; processing yields between groups when the budget is reached instead of rewriting the ledger on the launch-event stack. Only execution IDs survive between slices, so each slice reads current records. Pending work remains visible as pending/recovering, normal shutdown drains it before checkpointing, and an invalidated session owner cannot continue backfill or UI updates. If replay reaches a `bg_wait` result before all its launch evidence, even when some completions can already be counted, at most 256 entry IDs are deferred for one budgeted retry after the same snapshot is processed. Completions deduplicate by stable child identity; a partial archive link does not cover the entire result. Children still lacking ownership after retry report `completion-ownership-unresolved` and remain excluded, as do foreign-session results. Public leaf traversal continues across bounded slices without repeatedly copying full history; tool entries claim spend before queued background meta scans, preserving cross-granularity exclusion. Meta and tool revisions are checked within their own inlet, then accepted snapshots replace the shared ledger contribution in session arrival order. Millisecond mtimes are never compared with tool counters. Session-ownership-checked completion events atomically replace a restored snapshot of the same execution even without a source revision. Repeated completions and later meta files cannot replace that final result; old model partitions do not survive the replacement. Parallel tool progress retains each child's identity; each usage-bearing progress snapshot replaces that tool's previous progress, and the final result replaces progress again. Coverage sequences advance independently per producer. At the 10,000-observation memory limit, if no provisional record can be evicted, Coverage shows `partial · memory-exhausted` separately from disk `storage-exhausted`. Capacity prechecks and atomic-batch rejections likewise downgrade All, Turn and model rows to known lower bounds/unknown values; a turn with no successfully recorded usage shows `?`, not a free `$0.000`. Unchanged ledger projections are cached for the 1s footer refresh. | Mode | `/calls` | | --- | --- | | TUI | Interactive menu (This turn / This session / Coverage / Reconciliation) | | rpc | A text summary; when there are no records it appends *Usage is tracked in the interactive session only.* rather than staying silent | | `-p` / json | No UI channel (pi binds no `uiContext`, `ctx.hasUI === false`), so nothing is printed and script stdout stays clean | ### What is counted - Parent-session assistant usage is counted at `message_end`, after skipping this plugin's `preprocessAssistantMessage` zero-fill. Zeros already filled by the Pi SDK stay unknown; missing fields are not treated as reported. - Subagent and tool usage with trusted launch evidence stay on the launching parent turn, not the arrival turn. UUID case/hyphens are normalized consistently for binding and lookup; children with distinct IDs inherit the proven parent-run origin. Completions without launch evidence affect All/unassigned only. Runs explicitly launched before the first turn still belong to that first turn. - Synchronous pi `subagent` / Cursor `Task` tool results and `_meta.json` summaries feed the same counter; scanned directories are `.pi/subagents/artifacts` (pi-subagents ≥ 0.49), the legacy `.pi-subagents/artifacts`, and `subagent-artifacts/` next to the session file. - async / background subagents are collected through the `subagent:async-complete` / `subagent:foreground-complete` event bypass: the numbers come from the `usage` / `modelAttempts` / `totalCost` / `tokens` the event carries, and from the `_meta.json` files in the directories above when it carries none. pi-subagents 0.69 `bg_wait` is a management projection: only completion children whose run IDs were already observed through the current session's trusted path are accepted; pooled top-level `usage` and `details.results` are ignored, and a wait result cannot bind a new run. Parent-run plus flat child index forms the normal identity; an indexless meta file maps to index 0 only when that parent/agent has no indexed sibling, otherwise it is fail-closed. In-flight `tool_execution_update` payloads that already carry usage are ingested too; `tool_execution_end` for the same `toolCallId` replaces that in-flight progress even when the two payloads mint different sourceKeys. A newer `_meta.json` (mtime advanced) replaces **that file's** previous snapshot — a sibling dropped for cross-granularity reasons is not brought back because the file grew. **A completion event that already occupies the same `sourceKey` is not replaced by a later meta file** (first-wins, to avoid double counting); meta is waited on only when the event carried no numbers. **The `status.json` under pi-subagents' temp root and the child `session.jsonl` are not read** — in the default layout both sit outside the workspace (asyncDir under `os.tmpdir()`, child sessions under `~/.pi/`), and those two fallbacks were only ever allowed to read paths inside the workspace, so they never fired; they and that guard have been removed. - Indexless uniqueness is evidence from the current scan, not a fact that survives restart. The existing `source.runner` distinguishes indexed, indexless and completion provenance without adding v1 fields. On recovery, indexless snapshots and legacy child-zero snapshots without that evidence stay archived but are quarantined as `indexless-origin-unproven` until fresh trusted evidence permits accounting. Reloads and source toggles cannot resurrect revoked guesses; explicit indexed and completion records restore normally. - The `sessionId` in those events may be a bare ID, the full session file path, or its basename (pi-subagents identifies a session with `getSessionFile() ?? getSessionId()`); all three identity forms are matched. - A subagent row carries a **cost** only when the upstream reported money: the first of `usage.cost`, the sum of `modelAttempts[].usage.cost`, and `totalCost.costUsd` that has a value is taken as-is. When only tokens came through (`tokens` / `totalTokens`), **the tokens are counted and cost is shown as unknown (`?`)** rather than estimated at the parent model's rate. A subagent row with a low cost in `/calls` is therefore expected, not a lost token count. - Any tool that follows pi's convention of hanging a top-level `usage` on its result is counted — this is not tied to any one extension. A result that names its own model gets a `/` row, merging with the parent model's row when the name matches; one that names no model lands under `tool/`, with `?` for cost unless it reports a verifiable amount. Pi-compatible numeric `usage.cost` and a complete `{input, output, cacheRead, cacheWrite, total}` object (including an explicit zero total) are reported as producer-supplied cost; a missing cost is estimated only when a known pricing rule/registry entry matches. Unknown model/provider IDs never receive a default rate, and malformed or partial cost objects remain unknown. Tool names already claimed by the subagent path, or whose usage would double-count, are excluded: `subagent`, `task`, `bg_wait`, `subagent_wait`, `subagent_supervisor`, `intercom`, plus `@tintinweb/pi-subagents`' `Agent` / `get_subagent_result` / `steer_subagent`. - Two deliberate under-counts come with that. The three `@tintinweb/pi-subagents` names stay excluded; `subagents:completed` / `subagents:failed` are fail-closed probes that mark Coverage `unavailable` and **do not enter All**. If you have manually enabled that package's `reportUsage` (off by default), its usage is not counted in the totals. And a tool result that pools several LLM calls without reporting a count keeps **calls unknown** (the parser's defaulted `1` is not treated as an exact count); its tokens and cost are unaffected. Under-counting is the safe direction; double counting is not. - The LLM call behind a context compaction or a branch summary is counted under a `compact/` row (pi bills it to the session too; we used to miss it). Automatic compaction, manual `/compact`, overflow-recovery compaction and branch summaries are all covered. Deferred accounting uses the model ID/provider captured for that entry's event, not a later model selection. Historical entries or expired associations without model evidence never borrow the current model for pricing. A compaction owned by another extension (pi flags it `fromHook`) lands under `compact/unknown` and is charged only the cost it reports itself — the model it ran on is invisible to us, so its tokens are never priced at the session model's rate; one that reports no usage at all still cannot be counted. - **Pi 1.0.4 usage-only gate** (not broader package peer support): enabled only on the verified SDK version. Ordinary nested tools claim the parent's final pooled usage once, never child events. Incomplete/inconsistent nested records or dedicated/management/excluded children reject the whole pool as partial; no subtraction guesses a remainder. Independently identifiable subagent/Task results and trusted bg_wait completions saved in the parent's `details` still recover through their dedicated identities and policy gates. They never fall back to rejected top-level usage or infer child costs from nestedCalls. Nested dedicated results require stable run identities shared across inlets; tool-local fallback identities are isolated as `nested-dedicated-identity-unresolved`, not counted separately through parent details and child events. Root tools wait for their final entries and use the same ownership rules. Valid standalone UsageEntry records, including unknown kinds, use stable entry identities and affect All/unassigned only. They invent no call count and do not add reasoning/cacheWrite1h twice; ambiguous alternate-category exports are isolated. Idle discovery uses a cancellable 2s public leaf probe, not full-history polling, and unchanged probes do not redraw. The entry index keeps the newest 10,000 entries. Exhaustion stops full-history backfill and idle probes for that session, retaining the confirmed subtotal and partial marker rather than repeatedly scanning the dropped older entries; later parent messages and root tools are still counted. Synthetic SDK gates do not certify provider/login or whole-package compatibility. - **Structurally out of reach** (not a bug, and there is no switch for it): extensions that spin up a sub-session inside their own process without hanging a `usage` off a tool result the way pi's convention asks — dynamic-workflows, piolium, pi-goal-x and the like. Their messages never join the parent message stream, so pi's own `/cost` misses them too. The same goes for extensions such as `pi-vision` that call a model directly and write their own session entries, for extensions that spawn child pi processes without reporting their usage back (`@mjasnikovs/pi-task`'s `pi --mode json` workers, `pi-goal-list-loop-audit`'s `pi --mode rpc` auditor), and for pi-subagents **nested / fork / helper LLM calls** (there is no public child-factory usage hook; Coverage stays `partial` and a directory listing is not a support claim). When pi-subagents is configured with `artifactDir: "temp"` (or has no session file to sit next to), its `_meta.json` files land in the temp root, which is not scanned; an indexless metadata file is accepted only when uniqueness within its parent/agent can be proven, otherwise it stays fail-closed. A third-party extension that wants to be counted only has to put a top-level `usage` on its tool result — that lands in pi's `/cost` and here at the same time. - Set `LLMGATES_TPS=0` to turn off **all** usage collection (parent assistant, subagents, compaction, nested tools) and persist writes. Enabled by default. - Set `LLMGATES_TPS_PERSIST=1` or `"tpsPersist": true` in `config.json` to write the usage journal/checkpoint (off by default). Files live under `~/.pi/agent/llmgates/usage//` (`0700`/`0600`). A full disk or cap stops further writes and marks `storage-exhausted`; a corrupt or unknown-version checkpoint is **never overwritten or "repaired"**. Loading that skips damaged rows marks Coverage `partial` (`checkpoint-incomplete` / `journal-truncated`); rows that did parse still enter All. With persistence off, current-session public entries can still be recovered in bounded slices without writing a ledger or repricing historical usage; unrecoverable local estimates are disclosed. With persistence on, the archive is separate from the display projection: disabled categories are retained. Damage, truncation, identity conflicts or incomplete loading make the entire root read-only, protecting both files. Legacy parents without entry identities retain a legacy-window rather than being added to overlapping history. - Set `LLMGATES_TPS_SUBAGENT=0` to turn off the subagent bypass and the meta scan (the parent model and synchronous `subagent` / Cursor `Task` tool results are still counted). - Set `LLMGATES_TPS_COMPACTION=0` to stop counting compaction / branch-summary entries. - Set `LLMGATES_TPS_TOOL_USAGE=0` to stop counting generic tool-result usage (`subagent` / Cursor `Task` are still counted). - Aggregation is ordered through an asynchronous task chain and makes no additional model calls; counting happens only in an interactive parent session (TUI). Without persistence, recovery covers only usage provable from current-session public entries, not side-channel-only history. Third-party runners and external CLIs are **not a support list**: Coverage marks them `unavailable` / unverified (EventBus probes are fail-closed and usage-shaped payloads do not enter All). **An empty session pre-lists three S4 placeholder rows for CLI/job/runs** (not installed runners); npm probes such as tintinweb appear only after their events arrive. Directory hits in the research inventory are not compatibility certifications. Persistence remains opt-in: each append synchronously checks the entire usage directory, and closed roots are not automatically pruned by retention age. When the complete checkpoint exceeds 256KiB, writing is skipped while the old checkpoint/journal is retained; long sessions can eventually reach the 8MiB per-root or 64MiB global limit. Recovery queue/slice limits are described above; persistence write retries are not implemented. These limitations require a separate persistence change before enabling persistence by default; automatic rotation and write recovery are not promised. ### Pricing data The cost shown in the TUI and `/calls` is an **estimate based on upstream retail API rates** and may differ from what the gateway actually charges; check `/balance` or the gateway's own console for real account spend. Pi's built-in footer may still show `(sub)` after an OAuth login — that marker has nothing to do with gateway billing. `~/.pi/agent/llmgates/pricing.json` holds editable USD prices per **1M tokens** (`input`, `output`, `cacheRead`, `cacheWrite`). Keys are `modelId` or `provider/modelId` (e.g. `openai/gpt-5.6-sol`): ```json { "_comment": "overrides always beat rates and auto-sync", "updatedAt": 0, "lastAutoSyncAt": 0, "rates": { "openai/gpt-5.6-sol": { "input": 5, "output": 30, "cacheRead": 0.5, "cacheWrite": 6.25 } }, "overrides": { "anthropic/claude-sonnet-4-6": { "input": 3, "output": 15, "cacheRead": 0.3, "cacheWrite": 3.75 } } } ``` With `pricingAutoUpdate` enabled, every catalog refresh syncs retail model prices from [LiteLLM](https://github.com/BerriAI/litellm) in the background (without blocking the list): a **newly seen** missing key is fetched immediately; a key already confirmed absent from LiteLLM is re-probed at most once an hour within the same process (the record lives in memory only — restarting pi re-probes, and nothing is written to any file); a complete, positively cached table still refreshes every 24h. The round that does fetch the table re-checks **every** model in the current catalog with it, not just the gaps. On a failed sync — or one whose payload is not structurally a price table — the cache and the static rules are kept (`LLMGATES_DEBUG=1` for details). Auto-sync **only writes `rates`** and **never touches `overrides`**. `rates` entries outside the catalog survive a refresh. Every refresh re-reads the file from disk, so hand edits need no restart. The static rules in `extensions/model-pricing.ts` are the offline fallback. After a successful sync, the `cost` field of already-registered models is patched in memory — no extra catalog request. ## Upstream response model audit `/model-audit` checks whether a gateway "swapped the model": for every request sent through one of this plugin's gateway providers it compares **the model actually sent** with **the model the upstream reports in its streamed response**, and records one mismatch when the normalized **model series** differs. - **Detects and records only** — never blocks, retries, or changes request or response bytes, never touches billing, and **stays out of the usage ledger** (`/calls` and cost are unaffected). - **On by default.** `LLMGATES_MODEL_AUDIT=0` turns it off completely: no request wrapping, no history writes, no suffix. It is independent of `LLMGATES_TPS`. Read at every session start; changing it inside a running process needs `/reload`. - **Status line** — a red `.xN` is appended to the matching segment, e.g. `All(partial) 52c.~$6.20.x3, Turn 1m.2c.~$0.236.x1`. All is this session's total, Turn this turn's; hidden at 0. Shown only in the TUI parent session and catches up within about 1–2 seconds. - **`/model-audit`** — this session's All / Turn / unattributed (before the first turn, e.g. a compaction on resume) counts; this process's write failures / quarantines / writes unfinished at exit; per-API observation counters; the equivalence table status; and the most recent mismatches for this working directory (up to 300, newest first). A list in the TUI, a notification in rpc, nothing in `-p` / json. - **`/model-audit clear`** — after confirmation, clears this working directory's records **and the counts of every session in it** (status-line suffixes included); the file itself is kept. ### What counts as a different series Both names go through the same normalization; equal results are the same series and are **neither counted nor written**: - Case and a `vendor/` prefix are ignored; a trailing `-latest`, a date (`-2024-08-06`, `-20241022`, `@20241022`) and a 3–4 digit revision (`-002`, `-0613`, `-2411`) are dropped. - Gateway suffixes — only those proven from gateway source to select the same upstream model with different call parameters: CLIProxyAPI's trailing `(…)` (e.g. `gpt-5(high)`); NewAPI's `@thinking:` / `@effort:` / `@temperature:` / `@topp:` modifier chain, `-thinking` / `-thinking-` / `-nothinking` on `claude-*` / `gemini-*`, `-max|-xhigh|-high|-medium|-low|-minimal|-none` on `gpt-*` / `o1`…`o9` / `claude-*` / `gemini-*` (except `gpt-5.1-codex-max`, a real model), and `-none` / `-max` on `deepseek-v4-*`. - Example: sending `gpt-4o` and receiving `gpt-4o-2024-08-06` does not count; receiving `gpt-4o-mini` counts once. When the response names no model, or the model actually sent cannot be determined, there is no verdict and nothing is counted. ### Equivalence table **Model mappings / aliases configured by the gateway admin** are recorded as mismatches: with NewAPI's `model_mapping` and CLIProxyAPI aliases without `force-mapping`, the response reports the real upstream model. Put pairs you know are fine in `~/.pi/agent/llmgates/model-audit-equivalents.json`: ```json { "version": 1, "equivalents": [["my-sonnet-alias", "claude-sonnet-4-5"]] } ``` Names in one group (after the normalization above) are the same series; overlapping groups merge. **Any format error ignores the whole file**, and `/model-audit` shows `Equivalents: INVALID`. Read at session start and re-read by every `/model-audit`. ### What is and is not visible | pi version | openai-completions | anthropic-messages | openai-responses | | --- | --- | --- | --- | | 0.81–0.82 | ✅ via pi's `responseModel` field | ❌ | ❌ | | 0.83 and later | ✅ reads the response stream | ✅ reads the response stream | ✅ reads the response stream | The table comes from reading the pi-ai sources plus a smoke run on 0.87.0; until the real installed-package gate has run at both ends (0.81.0 and 0.86.0), it does not certify the whole peer range. The `Observed (this process)` line of `/model-audit` lists, per API, `fetch` (the stream side channel was called) / `response` (model read from the response bytes) / `field` (fallback to pi's field) / `none` (nothing read), so you can confirm the side channel works on your pi version. - **Covered by design**: the parent session, compaction and branch summaries; pi-subagents foreground and background; pi-subagents-lite (with this plugin's extension enabled); a pi started from bash inside a session (counted in the current session and turn). When a subagent runs in another working directory (a worktree, say), records still go to the parent session's history file, and a late record counts towards **the turn that started it**. Verification of each subagent combination on a real installed package is part of the pre-publish gate. - **Not covered**: external CLIs, models not served by this plugin's providers, and child processes started with a cleared environment (they become a separate root the parent cannot see). - History file names reuse the input-history directory encoding, so paths such as `/a/b-c` and `/a/b/c` share one file: per-session counts stay separate, but the `/model-audit` record list is mixed. - CLIProxyAPI echoes the requested name to **Responses clients** on Claude / Gemini / OpenAI-chat upstreams, so that path can never show a mismatch; CLIProxyAPI's `auto` model is recorded as a mismatch every time (exempt it with the equivalence table). - Only the model the upstream **reports** is visible: a response without a model name proves nothing, and a gateway that rewrites the response back to the requested name cannot be detected. - Responses whose content-type is not `text/event-stream` are not read (counted as `none`). ### Storage and privacy - History: `~/.pi/agent/llmgates/model-audit/.json`, directory `0700`, file `0600`, one per working directory. Only the latest 300 records are kept; counts are kept per session and are **not affected by the 300-record cap** (up to 100 sessions, the latest 20 turns each). - Each record: time, session and turn ids, gateway instance id, API, model sent, model received. **Never** prompts, response content, API keys, headers or base URLs; model names are stripped of control characters and length-capped. - A damaged file is renamed to `..corrupt` (one copy kept) and recreated; a file from a newer plugin version is read-only. Write failures never affect inference and show up in `/model-audit`'s process failure count; exit waits at most 1.5 seconds for unfinished writes. - The plugin keeps a small JSON in the `LLMGATES_MODEL_AUDIT_ROOT` environment variable (session id, working directory, history file path, current turn id — **no secrets**) so that subagents write back to the parent session. pi's bash tool passes it to child commands; do not set or edit it by hand. ## Input history pi's editor already walks through what you typed with ↑↓ (capped at 100 entries), but it lives **in memory only**: the list hangs off the editor instance, so **quitting pi wipes it**, and it does not distinguish working directories. (Within one pi process `/reload`, `/new` and `/resume` do *not* wipe it — pi never rebuilds that editor instance. This extension changes that; see “Known limitations” below.) This extension adds exactly one thing — it **stores that list on disk** and feeds it back into pi's history on the next start. The ↑↓ triggering rules, draft protection, de-duplication and the 100-entry cap all stay pi's own. **On by default, scoped to the current working directory.** | Scope | File | Meaning | | --- | --- | --- | | `cwd` (default) | `~/.pi/agent/llmgates/input-history/----.json` | One file per working directory, the same granularity pi uses for its own session files | | `global` | `~/.pi/agent/llmgates/input-history/global.json` | **One file shared by every working directory** of this pi user. It is the only option that introduces cross-project visibility, so it is an explicit opt-in and discloses itself once when first enabled | ### What is recorded - Only prompts **actually typed in the TUI** (the `input` event with an interactive source). - **Not recorded**: pi's built-in slash commands (`/model`, `/resume`, …) and extension-registered ones (`/endpoint`, `/input-history`, …), `!bash` / `!!bash`, messages from rpc clients or injected by extensions, and pi's replay of an older session's history. - **Recorded**: `/skill:`, prompt-template invocations and mistyped `/xxx`. Those are not commands — pi sends the whole line to the model as a prompt, so they are stored like any other prompt. - Leaving `!bash` out is a deliberate security trade: `!export TOKEN=…` or `!curl -H "Authorization: Bearer …"` — the inputs most likely to carry a secret — structurally can never reach this file. - There are exactly two caps: the **newest 100 entries** and **8 KiB per entry** (UTF-8). An entry over 8 KiB is skipped whole rather than truncated (a half prompt resurrected by ↑ and submitted is a real hazard); it still works with ↑ for the rest of the session. - On disk it is an MRU list: re-submitting an entry that is already stored **moves it to the front** instead of adding a copy, so habitual prompts ("continue", "run the tests") cannot fill all 100 slots. ### Known limitations - Slash commands and `!bash` never reach the persisted history, so **after a restart ↑ walks through fewer entries than it did inside the session**. - **This extension shortens the in-process lifetime of the history.** pi keeps it on an editor instance it builds exactly once per process, so `/reload`, `/new` and `/resume` normally leave it alone. Prefilling requires a fresh editor instance, and pi copies only the draft text — not the history — when it swaps editors, so after those three actions ↑ walks through **what is on disk**: entries that were never persisted (`!bash`, slash commands) are gone. That is the price of cross-process persistence; `LLMGATES_INPUT_HISTORY=0` gets pi's own behaviour back. - Several pi processes running in the same scope cannot see each other's in-memory history; they converge on the next start's prefill. The merge on disk is complete (cross-process file lock plus read-modify-write). - `/input-history off` and `/input-history clear` **only affect the current pi process**: another pi still running in the same scope recreates the file on its next prompt (with the new entries only). - A history file that fails to parse is treated as empty and is overwritten wholesale by the next write. **It is not a backup — do not hand-write anything into it.** - The prefill goes through pi's custom-editor API: an extension that calls `setEditorComponent` **after** this one (a vim-mode extension, say) replaces our editor and the **prefill silently stops working** (recording is unaffected). That is inherent to pi's extension API. - There is no per-entry delete. A secret that made it to disk can only be removed with `/input-history clear`. ## Remembering the last used model and thinking level pi does **not** store "the model used last". `defaultProvider` / `defaultModel` in `~/.pi/agent/settings.json` is an **explicitly pinned default**: since pi 0.84 both picking a model with Enter in `/model` and Ctrl+P / Ctrl+N cycling switch with `persist: false` (this session only), and only **Ctrl+S** ("set as default") in the `/model` list writes the file. (0.81–0.83 persisted every switch, which is why that file can look like a last-used record on older pi.) Startup does want to use that pinned default — **but only when no model scope is configured**. As soon as `settings.json` carries `enabledModels` (what `/scoped-models` saves) or the command line carries `--models`, pi's startup honors the saved model **only when it is inside the scope** and otherwise falls back to the scope's **first** entry. pi has no setting that reorders this. **The thinking level is the other half of the same problem.** Startup takes the level from `defaultThinkingLevel` in `settings.json` and then **clamps it to whatever model the start landed on** — with a scope's first entry that has a lower ceiling, the level you were using is gone right there. And `setThinkingLevel`, which writes that key, cannot tell "the level you picked with Shift+Tab" from "the level clamped out of a model switch": both go into the same key, so the setting is not a dependable record of the level used last either. This extension adds the two missing halves: 1. **Record** — `model_select` and `thinking_level_select` listeners write whichever model and level were switched to into `~/.pi/agent/llmgates/last-model.json` (one global file, three fields: provider id, model id and thinking level). `/model` with Enter, Ctrl+P cycling, Shift+Tab level changes and extension-driven switches all count. 2. **Restore** — once a fresh session exists, the model is set back to that one **and then** the level is set back. With nothing recorded yet (fresh install, just cleared), it falls back to pi's `defaultProvider` / `defaultModel` / `defaultThinkingLevel`: a scope shadows that pin the same way, so it is the same fix. **Model first, level second — never the other way round.** pi's own `setModel` re-derives the level and clamps it to the incoming model (0.81 from the outgoing model's current level / settings; 0.84 from a per-model override or `defaultThinkingLevel`), so a level applied first would be wiped by our own model switch. **The level is put back even when the model already matches** — the startup level comes from `defaultThinkingLevel`, the scope's first entry has already clamped it on the way in, and "same model" says nothing about the level. **Once a record exists, it outranks every explicitly pinned default in pi.** `defaultProvider` / `defaultModel` / `defaultThinkingLevel` in `settings.json` is read as a seed only while nothing has been recorded; after that, starts follow the record. That includes the pin written by **Ctrl+S** ("set as default") in the `/model` list from 0.84 on — press Ctrl+S, then cycle away with Ctrl+P, and the next start comes back on the Ctrl+P model, not the pinned one — and it includes a hand-written pin in a project's `/.pi/settings.json` (pi merges project settings over global; this extension stops consulting either once it has a record). Turn the feature off if the pin should win — but that only holds **where no scope is configured**: with `enabledModels` set and the pinned model outside it, pi's own startup falls back to the scope's first entry anyway (see above), so turning the feature off does not get the pin back either. **On by default.** It deliberately stays out of the way when: | Case | Why | | --- | --- | | `/resume` / a `/tree` fork / `/reload` | those actions arrive with a reason other than startup/new, so pi keeps the model | | the session already carries a conversation | including an old chat opened with `pi -c` / `--session`. CLI continue still arrives as `reason: "startup"`, so the skip is the conversation, not the `-c` flag | | `--model` / `--models` on the command line | `--model` pins the model for this run and outranks a remembered one; `--models` swaps in a different scope just for this run, so it is skipped too (an `enabledModels` scope stored in `settings.json` is still restored over). This row skips the level as well | | `--thinking` on the command line | the model still restores; the recorded level is not applied (`thinking=cli-thinking`) — the flag's level is what stays in effect, and since restoring the model makes pi re-clamp the level, it is set back afterwards. `--model provider/id:high` takes the row above and skips both | | the saved model is gone, has no credentials, or is already selected | pi's own choice of model is left untouched. **The level is still applied to whichever model is current** (not to the missing one) — it is a preference of its own, and pi's startup reads the very key that clamp overwrote | | the record carries no level (written by 0.5.0 or earlier), or a level pi does not name | only the level step is skipped; the model still restores | That leaves a **cold start** (`pi`), **`/new`**, and an empty continued session (`pi -c` on a session that never sent a message) — pi itself would not restore a model from stamps there, so they are treated the same as a cold start. Known costs and boundaries: - Each restore appends one `model_change` entry to the session, plus a `thinking_level_change` when the level actually moves (pi re-clamps it inside the model switch first, then we set the recorded level back; both steps append only on a real change). On pi 0.81–0.83 it also writes `defaultModel` and `defaultThinkingLevel` in `settings.json` as a side effect (extension-side `setModel` / `setThinkingLevel` persisted unconditionally in those versions); from 0.84 on both are gated on `options.persist`, which neither extension-side call passes, so neither is written. - **On restore, the record holds the level you asked for, not the clamped one.** Restoring `high` onto a model that stops at `low` lands on `low` in the session while `last-model.json` still says `high` — that only suppresses the **restore-triggered** clamp. A live switch onto a lower-ceiling model emits `thinking_level_select` with the clamped value and the record follows it; switching back does not resurrect `high`. - The `thinking_level_select` event carries **no `source`** (the automatic re-clamp inside a model switch is indistinguishable from you changing the level by hand), so what keeps restore-time writes out of the file is an internal latch held until microtasks from `setModel` / `setThinkingLevel` drain, not the event. - With no record on disk yet, changing only the level still writes: it hangs the level on the **current** model. From 0.84 on Shift+Tab is `persist: false`, so `defaultThinkingLevel` cannot cover the next start. Nothing is written only when there is no current model (no ctx). - A level the file names that pi does not **costs the level, not the model**: the model still restores and the level is left to pi. Passing it through would be worse — pi clamps a level it cannot place to `availableLevels[0]`, which is `off` on most models, so one typo would silently turn thinking off. The cost is that a level added by a future pi is not restored until it is listed here. - If the last model is not in the local catalog cache yet (the provider's entry in `~/.pi/agent/models-store.json` was just cleared, or the model is new upstream), this start reports `model-unavailable` and restores no model (the level is still applied to the **current** model); the next start picks it up once the background catalog refresh has landed. - What gets recorded is an **explicit switch**: `/model` with Enter, Ctrl+P cycling, Shift+Tab level changes, extension `setModel` / `setThinkingLevel` (**another extension** swapping the model or the level for one kind of task is recorded too). A model or level named with `--model` / `--thinking` at startup is **not** recorded — pi emits these events only on a switch, and a startup level set to the value it already has emits nothing — so `pi --model … -p …` in CI cannot overwrite the record you have. `--thinking` also **skips restoring the level** on that start (the model still restores). pi currently emits no `model_select` when it restores a session's own model; a future `source: "restore"` event is ignored too. Going straight from an old session to `/new` lands on the pair you last explicitly switched to, not on that session's. - A restore-triggered `model_select` / `thinking_level_select` does not write `last-model.json` back, so it cannot clobber a newer switch recorded by another pi process. - **Non-interactive runs are covered too**: `pi -p "..."` and RPC mode go through the same `session_start`, so scripts and CI get switched to the last used model and level as well. Pin the model with `--model`, the level with `--thinking`, or set `LLMGATES_RESTORE_LAST_MODEL=0`. - If the restored model is **not** in `enabledModels`, the next Ctrl+P jumps to the scope's *second* entry (pi starts from index 0 when the current model is not in the list, so the first is skipped); Ctrl+N jumps to the last one. - With several pi processes open it is last-whole-write-wins: each write replaces the file atomically, but it first reads the other field (a model switch carries the stored level, a level change carries the stored model), so two processes interleaving a model switch and a level change can lose one field. That is last-writer-wins for a preference file; no lock is added. - Recording keeps running even when restoring is turned off — otherwise turning it back on would have nothing to restore. The file holds a provider id, a model id and a thinking level, nothing else. - Turn it off with `"restoreLastModel": false` or `LLMGATES_RESTORE_LAST_MODEL=0`; one switch covers both the model and the level, there is no second key. Startup then behaves exactly as pi does on its own. `LLMGATES_DEBUG=1` prints the decision taken on every start, one part per half (`model=restored thinking=restored`, `model=already-selected thinking=restored`, `model=cli-model thinking=skipped`, `model=restored thinking=cli-thinking`, …), which is the quickest way to see which branch you hit. ## Configuration Gateway URLs and API keys can **only** be configured through `/login`; they are never read from environment variables or config files. ### Config files Config files live under `~/.pi/agent/llmgates/` (older flat files under `~/.pi/agent/` — `llmgates.json`, `llmgates-2api.json`, `llmgates-model-pricing.json` — are migrated automatically when the extension loads): | File | Contents | | --- | --- | | `config.json` | Extension-level switches: `pricingAutoUpdate`, `inputHistory`, `inputHistoryScope`, `restoreLastModel`, `tps`, `tpsPersist`, `tpsExt` | | `2api.json` | Instance registry (ID, display name, scheme, base URL — **no keys**) | | `2api-models/.json` | Per-instance endpoint overrides, see [Editing the override file by hand](#editing-the-override-file-by-hand) | | `pricing.json` | Editable model prices and the LiteLLM sync cache, see [Pricing data](#pricing-data) | | `input-history/*.json` | Persisted input history, one file per scope, see [Input history](#input-history) | | `last-model.json` | The model and thinking level used last (provider id + model id + thinking level), see [Remembering the last used model and thinking level](#remembering-the-last-used-model-and-thinking-level) | | `usage//` | Optional usage journal/checkpoint (not created by default); see [Status line and `/calls`](#status-line-and-calls) | | `model-audit/*.json` | Upstream response model audit history, one per working directory; see [Upstream response model audit](#upstream-response-model-audit) | | `model-audit-equivalents.json` | Optional model equivalence table for the audit (create it by hand); see [Equivalence table](#equivalence-table) | `config.json` (the values below are the **defaults**, in effect whenever the file or a key is missing): ```json { "pricingAutoUpdate": true, "inputHistory": true, "inputHistoryScope": "cwd", "restoreLastModel": true, "tps": true, "tpsPersist": false, "tpsExt": true } ``` - Setting `"pricingAutoUpdate": false` or `LLMGATES_PRICING_AUTO_UPDATE=0` restricts pricing to local/manual values. - `inputHistory` / `inputHistoryScope` are described under [Input history](#input-history). Prefer `/input-history` for changing them — it merges into the file and keeps every other key. A hand edit needs a `/reload` to take effect. - Set `"restoreLastModel": false` or `LLMGATES_RESTORE_LAST_MODEL=0` to stop restoring the last used model and thinking level, see [Remembering the last used model and thinking level](#remembering-the-last-used-model-and-thinking-level). This key is re-read at the start of every session, so an edit applies from the next start on. - `tps` / `tpsPersist` / `tpsExt` are described under [Status line and `/calls`](#status-line-and-calls). Environment variables override the file; when the master switch is off, `tpsPersist` does not write. A hand edit needs a `/reload` to take effect. ### Environment variables | Variable | Effect | | --- | --- | | `LLMGATES_PRICING_AUTO_UPDATE` | Overrides `pricingAutoUpdate` (default `true`; `0` / `false` disables) | | `LLMGATES_INPUT_HISTORY` | Overrides `inputHistory` (default `true`; `0` / `false` turns persistent input history off — the simplest kill switch) | | `LLMGATES_INPUT_HISTORY_SCOPE` | Overrides `inputHistoryScope`: `cwd` (default) or `global` | | `LLMGATES_RESTORE_LAST_MODEL` | Overrides `restoreLastModel` (default `true`; `0` / `false` stops fresh sessions from restoring the last used model and thinking level) | | `LLMGATES_DEBUG` | `1` / `true` / `yes` enables debug logging | | `LLMGATES_BLOCK_PRIVATE_URLS` | `1` / `true` / `yes` rejects private / link-local gateway addresses given as **IP literals** (loopback still allowed); hostnames such as `gateway.local` are not subject to this rule | | `LLMGATES_TPS` | Master usage-collection switch (enabled by default; `0` / `false` / `no` stops every inlet, including persist writes) | | `LLMGATES_TPS_PERSIST` | Usage journal/checkpoint (off by default; `1` / `true` / `yes` enables; overrides `tpsPersist`) | | `LLMGATES_TPS_EXT` | Third-party / external Coverage probes (on by default; `0` does not register EventBus observers. Uncertified sources never enter All) | | `LLMGATES_TPS_EXT_` | Per-source off switch, e.g. `LLMGATES_TPS_EXT_TINTINWEB=0`. Cannot widen the master switch | | `LLMGATES_TPS_SUBAGENT` | Enabled by default; `0` / `false` / `no` turns off the subagent async bypass and the meta scan | | `LLMGATES_TPS_COMPACTION` | Enabled by default; `0` / `false` / `no` stops counting compaction / branch-summary entry usage | | `LLMGATES_TPS_TOOL_USAGE` | Enabled by default; `0` / `false` / `no` stops counting top-level tool-result `usage` (`subagent` / Cursor `Task` are unaffected) | | `LLMGATES_MODEL_AUDIT` | Upstream response model audit (enabled by default; `0` / `false` / `no` stops request wrapping, history writes and the `.xN` suffix; independent of `LLMGATES_TPS`) | | `PI_OFFLINE` | `1` / `true` / `yes` skips network catalog refreshes | All of these parse the same way: `1` / `true` / `yes` / `on` is on, `0` / `false` / `no` / `off` is off, and any other value counts as unset (falling back to the respective default). `LLMGATES_INPUT_HISTORY_SCOPE` only accepts `cwd` / `global`; anything else likewise counts as unset. When one of these is **actually in effect** — a recognized value that really outranks `config.json` — the subcommand it governs **refuses to run** and asks you to unset it first, otherwise the config file would say one thing while the environment kept overruling it. The refusal is **per setting**: with only `LLMGATES_INPUT_HISTORY_SCOPE` set, `scope` is refused while `on` / `off` still work, and vice versa; an unrecognized value (say `LLMGATES_INPUT_HISTORY=maybe`) already falls back to the config/default and refuses nothing. ## Security - API keys are always treated as a **literal string**; `!`, `$`, `${...}`, `$$`, `$!` and friends are never interpreted as shell commands or environment expansion. - Gateway credentials come only from `/login` (written to pi's `auth.json`), never from environment variables or config files, and keys and URLs are never shared between instances. - Remote gateways must use **HTTPS**; HTTP is allowed only for loopback (`localhost`, `127.0.0.0/8`, `::1`, IPv4-mapped loopback). There is no insecure override switch. - The wildcard addresses `0.0.0.0` and `::` are **always rejected** (independent of `LLMGATES_BLOCK_PRIVATE_URLS`): they are listen addresses, not connectable destinations. When a gateway listens on `0.0.0.0:8317`, use `http://127.0.0.1:8317/v1` as the base URL. - Gateway network calls (`/models`, `/balance`, inference) use a whole-operation timeout, a 5 MiB response body cap, and same-origin manual redirects. - With `pricingAutoUpdate` enabled, the retail price sync fetches a fixed LiteLLM JSON from `raw.githubusercontent.com` (background, 30s timeout, 8 MiB cap) without blocking the catalog or inference. Disable it via config or `LLMGATES_PRICING_AUTO_UPDATE=0`. - TPS / cost tracking preprocesses assistant usage on a background queue; malformed usage is skipped or zeroed, and a failure never affects inference (`LLMGATES_DEBUG=1` records the details). - Startup is cache-first; cache-only, offline and freshness-window skips use the routing/thinking metadata straight from the cache. A session start can trigger one background refresh, but there is no periodic refresh timer; a failure warns and keeps the old catalog/cache. - A normal catalog refresh publishes new models only when both the network mapping and the cache write succeed; a network or cache write failure keeps the previous in-memory and on-disk values. A failed cache write right after login is the exception: the login is not undone, the session uses the validated catalog, and the disk keeps the old cache. - Config files are written with mode `0600` and replaced atomically. - The upstream response model audit only reads response bytes: it forwards them untouched and parses on a side channel, never re-encoding or changing the request; the history stores model names and session / instance ids only, never prompts, responses, keys or headers. History files are `0600` in a `0700` directory; only the `llmgates/model-audit` directory itself is checked for not being a symlink and tightened, no other directory is touched. - Input history files are `0600` in a `0700` directory, the same level as `auth.json`. ⚠️ POSIX permission bits offer no real protection on Windows; rely on the access control of the user profile directory there. - Input history is scoped to `cwd` by default: pi already writes every user message into a per-cwd session file (`~/.pi/agent/sessions/`), so the incremental risk of having this on by default is **aggregation** — turning scattered input into one readable list — not "input starts hitting the disk". `global` is the only option that adds **cross-project visibility**, which is why it is an explicit opt-in that discloses itself once. - pi's built-in and extension-registered slash commands, and `!bash`, **structurally never enter** the input history file; `/skill:` and prompt-template invocations do (they are prompts, not commands). See [Input history](#input-history). - Input history does no secret filtering or automatic redaction: incomplete redaction is worse than none, because it suggests safety that is not there. There is no per-entry delete either. - Three ways out: `/input-history off`, `/input-history clear`, `LLMGATES_INPUT_HISTORY=0`. After uninstalling the extension, `rm -rf ~/.pi/agent/llmgates/input-history/` wipes it completely. - **Unsupported / unsafe:** configuring this extension's provider `apiKey` through a `~/.pi/agent/models.json` overlay (pi may re-enable config-value syntax). Do not do this. ## Troubleshooting | Symptom | What to do | | --- | --- | | Extension not loaded after install | `/reload` or restart pi | | The entry is missing from `/login` | Confirm you ran `/reload`; the entry is named 「LLMGates 网关」. If the commands disappeared too, check the startup log — most likely `llmgates/2api.json` failed to parse, see [Known limitations](#known-limitations) | | No models after install | Add an instance with `/login` first; check the key's model permissions on the gateway and that `GET /v1/models` works | | All existing models vanished after an upgrade | You were using the removed built-in official gateway (core provider); see [Upgrading from 0.2.13 and earlier](#upgrading-from-0213-and-earlier) | | `401` / `403` at startup | Reconfigure that instance's key with `/login ` | | `/balance` shows *not available* | That gateway exposes no recognizable quota endpoint (CLIProxyAPI, for example) — expected behaviour | | Kimi / `tokenization failed` | Upgrade this extension and `/reload`; Kimi rejects the `developer` role and the extension injects a compat shim. Starting a fresh session also helps (switching to K3 from another model mid-session is unreliable) | | GLM / `400001` “角色信息不正确” (invalid role) | Upgrade this extension and `/reload`; a GLM id or recognized `provider_id` enables the `system`-role compat patch. Custom aliases need a recognized vendor hint | | A wrong endpoint causes a 400 | Fall back with `/endpoint auto ` or `auto` in `/endpoint-setting` | | Cost does not match the bill | The TUI cost is an upstream retail estimate; check `/balance` or the gateway console for account spend | | Cost shows as a `~` estimate, or prices look stale | The price table could not be fetched (offline, `raw.githubusercontent.com` blocked, or Node `fetch` ignoring `HTTPS_PROXY`), or the response was not structurally a price table (`Implausible LiteLLM pricing table` — usually a proxy or error page in its place); cost falls back to cached or static rates and nothing else is affected. **No warning is printed by default**; `LLMGATES_DEBUG=1` then `/reload` shows `LiteLLM pricing sync failed` with the cause, or edit `~/.pi/agent/llmgates/pricing.json` by hand | | `The agent is still busy` | `/endpoint`, `/endpoint-setting` or `/llmgates-reload` waited more than 120s for the current turn to end; no file was written — rerun after the turn finishes | | `file lock was compromised` | The lock could not be renewed within its window (machine sleep, a long event-loop stall, a network drive). It is released automatically and work continues without affecting writes; if it recurs, check whether `~/.pi/agent/` sits on a network filesystem | | A red `.xN` at the end of the status line | The upstream response reported a different model series than the one sent; see `/model-audit`. Exempt mappings / aliases configured on the gateway with the [equivalence table](#equivalence-table) | | Need debug logs | `LLMGATES_DEBUG=1`, then `/reload` | ## Upgrading from 0.2.13 and earlier This extension used to ship a built-in official LLMGates gateway (a core provider, connected via `/login LLMGates` or the `LLMGATES_API_KEY` / `LLMGATES_BASE_URL` environment variables). That built-in gateway was **removed in 0.3.0**: after upgrading it is no longer registered and its models disappear from `/model`. Migration steps: 1. Re-add it as an ordinary instance: `/login` → 「LLMGates 网关」 → **通用网关** (generic gateway), with the original base URL and API key. 2. Endpoint overrides from the old `llmgates/models.json` are not migrated. Models whose endpoint the gateway declares still route automatically — only models you had **manually reassigned** need to be recreated in `llmgates/2api-models/.json`. 3. Clean up `llmgates/config.json` by hand: its `apiKey` / `baseUrl` / `providerId` / `providerName` are no longer read and are not deleted automatically. Keep only `pricingAutoUpdate`. A leftover `llmgates` entry in `auth.json` is overwritten by the login entry's own inert marker on your **first successful login** (that entry now uses `llmgates` as its provider id), and the plaintext key disappears with it. See the [CHANGELOG](./CHANGELOG.md) (in Chinese) for the full set of changes. ## Development and release ```bash git clone https://github.com/ax128/pi-llmgates.git cd pi-llmgates npm install npm run build # compile extensions/ -> dist/ (pi.extensions points at dist, so rebuild after source changes) npm run check # typecheck + vitest pi install . ``` Design and implementation docs: [docs/README.md](docs/README.md). ### Release (maintainers) **The local pre-publish gate is mandatory** (`npm run gate` → install the `.tgz` → functional verification → `gate-record-pass.sh`); see [docs/pre-publish-gate.md](docs/pre-publish-gate.md). The full npm flow for agents / maintainers (**get the auth link → wait for the operator → publish → hand back the install command**): - [docs/pre-publish-gate.md](docs/pre-publish-gate.md) (the gate, not skippable) - [docs/npm-package.md](docs/npm-package.md) (starts with the standard agent publish dialogue) - [AGENTS.md](AGENTS.md) > Do not `set -a && source .env` up front: the probe script has its own `loadDotEnv()`, and `publish-npm.sh` reads the token only for `npm publish` / `npm view` — the check / build / pack steps must not see it. ```bash node ./scripts/npm-publish-auth-link.mjs # send the link to the operator ./scripts/publish-npm.sh --otp= # run only after they reply ``` ## Related documents | Document | Description | | --- | --- | | [CHANGELOG.md](./CHANGELOG.md) | Version history (Chinese) | | [docs/README.md](docs/README.md) | Internal design specs and source-file index (Chinese) | | [pi docs](https://pi.dev) | Pi extension and Provider API | ## License MIT — see [LICENSE](LICENSE)