# Caching

There are **two distinct kinds of caching** involved in this package. They are
easy to confuse; this document explains both.

## 1. Model catalog cache (pi's `models-store.json`)

**What:** the list of available models per provider, persisted by pi.

**How it works here:**

- The OpenCode providers register a `refreshModels(context)` hook
  (`src/backends/opencode/refresh.ts`).
- pi calls the hook in **two phases** every time the model selector is opened
  (`/model` — this is pi's trigger; see `model-selector.js` in pi-coding-agent):
    1. `allowNetwork = false` — the hook returns `context.stored.models` (the
       previously persisted catalog) so offline startups keep the last live list.
    2. `allowNetwork = true` — the hook fetches `https://opencode.ai/zen/{go,}v1/models`,
       merges live ids over the static baseline, and calls
       `context.publish({ persist: { models, checkedAt } })`, which writes
       `~/.pi/agent/models-store.json` keyed by provider id
       (`oc-zen`, `oc-go`).
- New upstream models **appear automatically**; the static catalog remains the
  offline baseline and supplies full metadata (pricing, context window).

**Behavior matrix:**

| Situation                                | Result                                                            |
| ---------------------------------------- | ----------------------------------------------------------------- |
| `/model` opened, network OK              | live fetch → merged list → persisted                              |
| `/model` opened, network down / API down | `null` → static baseline shown, no error                          |
| Startup (no `/model` opened)             | static baseline (+ cached list restored if a refresh already ran) |

**Command Code** uses its own cache (vendored `models.ts`):
`<agent-dir>/commandcode-models.json` — versioned, atomic write, live → cache →
empty fallback chain.

**Note:** the catalog cache does **not** affect per-request cost. That is the
second concept:

## 2. Usage cache hits (prompt caching while chatting)

**What:** provider-side caching of the conversation prefix. When a new request
shares a prefix with a cached one, the provider bills the shared tokens at the
much cheaper `cacheRead` rate and (sometimes) reports `cacheWrite` for the
cache update.

**How it works here:**

- **OpenCode (oc-zen/oc-go):** requests are delegated to pi-ai built-in
  streamers, which already handle prompt caching:
    - Anthropic transport → `cache_control: { type: "ephemeral" }` on system /
      messages / tools (unless cache retention is `none`)
    - OpenAI transports → prompt-cache keys + session-affinity from
      `options.sessionId`
    - Usage (`cacheRead` / `cacheWrite` tokens) is parsed by pi-ai and costed
      with the rates in `ZEN_PRICING` / `GO_PRICING`
- **Command Code:** the request carries a **stable `threadId`** derived from
  pi's `options.sessionId` (fresh uuid as fallback). A stable thread lets the
  upstream cache the conversation prefix across turns. `cacheReadTokens` /
  `cacheWriteTokens` are parsed from `totalUsage.inputTokenDetails` and
  reported to pi separately, so displayed costs use the discounted rates.

**Pricing tables include cache rates** (e.g. DeepSeek V4 Flash:
input 0.14 → cacheRead 0.028, ~5× cheaper). Models not in the tables display
$0 — that is a display-only placeholder, **not** a guarantee of free billing.

**How to verify cache hits are happening:**

- Open a conversation, send a message, then send a follow-up (same session).
- Check the usage/cost display in pi: the second request should show
  `cacheRead` tokens > 0 if the upstream cached the prefix.
- `~/.pi/agent/models-store.json` entries `oc-zen` / `oc-go` confirm the
  catalog cache is being maintained (concept 1).