# Provider caching

## What it does

Provider caching documents Prism's cache intent surface:

- `ProviderRequestOptions.cache?: PromptCacheHints` for structured, provider-agnostic cache hints.
- Legacy aliases `cacheKey` and `cacheRetention`, still supported for backwards compatibility.
- `PromptCacheBreakpoint` locations for reusable prompt regions.
- `ModelCacheCapabilities` for model/provider cache support metadata.
- Shared helpers: `sanitizeCacheKey`, `mapCacheRetention`, `applyCacheControl`, `resolveBreakpoint`, `canonicalizeJsonSchema`, `cacheHitRate`, `cacheSavings`, and `cacheUsageReport`.

Cache hints are best-effort. They describe intent; providers decide whether their native API can use them. Prism does not guarantee cache hits.

**Kernel defaults (0.5.1).** `applyDefaultProviderRequestOptions` fills `cache.breakpoints` with `{ location: "system_prompt" }` and `{ location: "last_stable_message" }` plus `cacheRetention: "short"` when `model.cache.kind === "cache_control"` or `model.cache.explicitBreakpoints === true`, unless the host set `cache.mode: "off"`, `cacheRetention: "none"`, or a non-empty breakpoint list. Implicit / `none` / host-owned (Azure, Bedrock, Vertex, AI SDK) models get no Prism markers. Session and cache keys are correlation ids — Cache keys must never be credentials.

## When to use it

Use this page when a host or provider package needs to:

- Mark stable system prompts, tools, context, or messages as cacheable.
- Use cache-aware default input ordering so stable instructions, attachments/resources, summaries, and prior history form a reusable prefix before the current user turn.
- Carry a stable cache key across turns without putting provider-specific fields in core.
- Read `ModelConfig.cache` to decide whether to map hints to implicit caching, key-based caching, cache-control breakpoints, provider-specific caching, or no caching.
- Compute normalized cache diagnostics from `Usage.cacheReadTokens` / `Usage.cacheWriteTokens`, including providers that only report reads.
- Understand when **caller-gated model discovery** may fill `ModelConfig.cache` / `ModelConfig.cost` from a live `/models` response (see [Discovery and live cache/cost metadata](#discovery-and-live-cache-cost-metadata)).

Do not use cache keys for credentials, bearer tokens, API keys, OAuth tokens, user secrets, or raw private prompts.

## Inputs / request

```ts
import type {
  ModelCacheCapabilities,
  PromptCacheBreakpoint,
  PromptCacheHints,
  ProviderRequestOptions,
} from "@arnilo/prism";
```

| Type / field | Purpose |
| --- | --- |
| `PromptCacheHints.mode?: "auto" | "on" | "off"` | Host intent. Providers may ignore unsupported modes. |
| `PromptCacheHints.key?: string` | Stable, untrusted cache key. Sanitize before sending to provider APIs. |
| `PromptCacheHints.retention?: "none" | "short" | "long"` | Desired retention. `mapCacheRetention()` downgrades unsupported long retention. |
| `PromptCacheHints.breakpoints?: readonly PromptCacheBreakpoint[]` | Stable prompt locations to mark for cache-control style providers. |
| `PromptCacheBreakpoint.location` | `system_prompt`, `tools`, `stable_context`, `last_stable_message`, `last_user_message`, or `message_id`. |
| `PromptCacheBreakpoint.messageId?` | Required when `location: "message_id"`. |
| `PromptCacheBreakpoint.ttl?` | Generic `short` / `long` hint. Provider packages map to native TTL shape. |
| `ModelConfig.cache?: ModelCacheCapabilities` | Static model/provider cache support metadata. |

`ModelCacheCapabilities.kind` values are generic: `implicit`, `openai_key`, `cache_control`, `provider_specific`, or `none`. Core never branches on provider names; provider packages read the metadata and map it to native requests.

Legacy alias note: `cacheKey` maps to `cache.key`, and `cacheRetention` maps to `cache.retention`. When both are present, structured `cache.key` / `cache.retention` is the authoritative cache intent for providers that read structured hints; legacy fields remain for older adapters.

## Outputs / response / events

Cache helpers return plain data:

| Helper | Output |
| --- | --- |
| `sanitizeCacheKey(value, maxLength)` | Safe key string or `undefined`. |
| `mapCacheRetention(retention, model)` | `"short"`, `"long"`, or `undefined`. |
| `applyCacheControl(messages, breakpoints, options)` | New message array with `cache_control: { type: "ephemeral" }` on selected message anchors. |
| `resolveBreakpoint(messages, breakpoint)` | Message index for a `PromptCacheBreakpoint` (`-1` when unresolved); shared anchor selection for `applyCacheControl` and OpenAI explicit breakpoints. |
| `canonicalizeJsonSchema(value)` | Clone with sorted object keys and `required` names; semantic arrays stay ordered. Used by first-party tool serializers. |
| `cacheHitRate(usage)` | Cached input ratio or `undefined`. |
| `cacheSavings(usage, model)` | Estimated read-token savings or `undefined` without pricing. |
| `cacheUsageReport(usage, model?)` | Normalized reported read/write tokens, hit rate, estimated savings, and currency when available; `undefined` when no cache token field is reported. Missing fields stay absent, never become `0`. |

Cache accounting stays in normalized `Usage.cacheReadTokens` and `Usage.cacheWriteTokens`. Terminal `provider_turn_finished.metadata.cache` carries the same numeric report for a reporting provider; unavailable cache usage stays absent.

For stable-prefix payloads, `inputLayout: "cache_aware"` is the default on the default input builder, `assembleProviderInput()`, `AgentConfig`, and `RunOptions`; set `inputLayout: "legacy"` to restore the prior order. The default prompt builder's cache-aware order is leading system instructions → resolved context blocks → selected/progressively disclosed skill catalogs → fallback text tool declarations → attachments/resources → summaries → prior history → pending tool results → current input → optional session tail. `RuntimeAgentSession` uses that tail for URI resources and loaded skill bodies: first insertion fixes `resource:<uri>` / `skill:<name>` order, so a later skill load appends instead of rewriting its catalog slot. Re-deriving the same id keeps its position; changed bytes explicitly invalidate from that tail segment. Declared tool schemas remain in `ProviderRequest.tools` and are never granted by prompt middleware. First-party tool serializers run `canonicalizeJsonSchema` so property insertion order cannot break that prefix. Context-budget eviction, custom builders/middleware, `toolResultFold`, and attention compilation are explicit invalidation boundaries; folding cannot move to an append-only tail without retaining the raw payload it exists to remove. The prefix is byte-stable only when those stable inputs are unchanged; Prism still does not guarantee provider cache hits.

## Request/response example

```json
{
  "providerRequest.options": {
    "sessionId": "sess_123",
    "cache": {
      "mode": "on",
      "key": "sess_123",
      "retention": "long",
      "breakpoints": [
        { "location": "system_prompt" },
        { "location": "last_user_message" }
      ]
    }
  },
  "model.cache": {
    "kind": "cache_control",
    "maxBreakpoints": 4,
    "minCacheableTokens": 1024,
    "longRetention": true
  }
}
```

## Implementation example

```ts
import {
  applyCacheControl,
  cacheHitRate,
  cacheUsageReport,
  mapCacheRetention,
  sanitizeCacheKey,
  type ModelConfig,
  type PromptCacheHints,
} from "@arnilo/prism";

const model: ModelConfig = {
  provider: "demo",
  model: "demo-large",
  cache: { kind: "cache_control", maxBreakpoints: 4, longRetention: true },
};

const hints: PromptCacheHints = {
  mode: "on",
  key: "workspace:agent#1",
  retention: "long",
  breakpoints: [{ location: "system_prompt" }, { location: "last_user_message" }],
};

const key = sanitizeCacheKey(hints.key, model.cache?.maxKeyLength ?? 128);
const retention = mapCacheRetention(hints.retention, model);
const stamped = applyCacheControl(messages, hints.breakpoints ?? [], { maxBreakpoints: model.cache?.maxBreakpoints });
const hitRate = cacheHitRate({ inputTokens: 1000, cacheReadTokens: 800 });
const report = cacheUsageReport({ inputTokens: 1000, cacheReadTokens: 800 }, model);
// { cacheReadTokens: 800, hitRate: 0.8, ... }

await session.run("Explain this", { inputLayout: "cache_aware" });
```

## Extension and configuration notes

Provider request policies can set `ProviderRequestOptions.cache` or the legacy `cacheKey` / `cacheRetention` aliases. Provider packages decide how to map hints to native payloads:

| `ModelCacheCapabilities.kind` | Typical mapping |
| --- | --- |
| `implicit` | No request mutation; provider caches automatically. Conformance (`assertNoForeignCacheFields`) proves the serialized body carries no cache wire fields. |
| `openai_key` | Send sanitized cache key and mapped retention where supported. |
| `cache_control` | Use `applyCacheControl()` on provider-native message anchors. |
| `provider_specific` | Provider package uses `compat`/native options intentionally. |
| `none` | Do not send cache fields. |

### Per-provider cache behavior

| Provider package | Cache kind | Explicit cache hints | Multi-turn reuse notes | Caveats |
| --- | --- | --- | --- | --- |
| `@arnilo/prism-providers/openai` | `openai_key` | Sends sanitized `prompt_cache_key`; pre-5.6 models emit `prompt_cache_retention: "24h"` when `longRetention`; GPT-5.6+ models (`explicitBreakpoints`) map `cache.breakpoints`/`cache.mode: "on"` to `prompt_cache_options: { mode: "explicit" }` + `prompt_cache_breakpoint` markers (≤4 writes). | Stable cache key + stable prefix can improve reuse; keep selected anchors stable. | Best-effort only; `"short"`/`"none"` omit retention; `"30m"` TTL is the default and never emitted. |
| `@arnilo/prism-providers/anthropic` | `cache_control` | Marks only selected Anthropic message anchors; `system_prompt` breakpoints emit native `system` text blocks with the marker; `"long"` maps to documented `ttl: "1h"`. | Keep selected anchors stable. | Best-effort; never stamp every block. |
| `@arnilo/prism-providers/google` | none | Sends no Prism cache marker. | Host/model may have upstream behavior. | Gemini cache controls are not mapped in this package. |
| `@arnilo/prism-providers/openrouter` | `cache_control` | Kernel defaults (agent session / helper) emit **per-message** `cache_control` on `system_prompt` + `last_stable_message`. Raw `generate` with empty breakpoints may still send top-level automatic `cache_control`. `"long"` may add `ttl: "1h"`. Sticky `session_id` routing. | Breakpoint-stable prefixes can be reused by upstream providers. | Best-effort only; top-level automatic may exclude some backends from routing. |
| `@arnilo/prism-providers/opencode-go` | route-specific | Sends sanitized `x-opencode-session`; Anthropic route applies selected `cache_control` breakpoints; OpenAI route sends none. | Session id + unchanged selected anchors can help route-native caches. | Best-effort and route-dependent. |
| `@arnilo/prism-providers/hyper` | route-specific implicit / `cache_control` | Chat route sends no markers (implicit prefix caching); `qwen3.6-*` messages route applies `cache_control` only to caller-selected `cache.breakpoints` (max 4); **no `ttl`** (undocumented). Opt-in responses route emits OpenAI-standard `prompt_cache_key` from hints only, never retention/options. | Keep selected Anthropic anchors and prior history stable. | Best-effort only; `402 billing_error` when Hypercredits run out. |
| `@arnilo/prism-providers/commandcode` | route-specific implicit / `cache_control` | Chat route sends no markers (implicit prefix caching); `claude-*` messages route applies `cache_control` only to caller-selected `cache.breakpoints` (max 4); **no `ttl`** (undocumented). GPT-5.6 tiers keep docs `cacheWrite` prices in `cost` but stay implicit until the live probe verifies `prompt_cache_key`. | Keep selected Anthropic anchors and prior history stable. | Best-effort only; GPT-5.6 explicit `prompt_cache_key` upgrade pending probe (plan 055 Task 9); OSS models bill at mean per-provider price; DeepSeek off-peak 17h/day, peak 2×. |
| `@arnilo/prism-providers/zai` | `implicit` | No explicit cache payload; GLM context caching is automatic. | Resend unchanged prior history for implicit context-cache reuse. | Best-effort only; cache options do not force hits. |
| `@arnilo/prism-providers/kimi` | implicit by default, optional `cache_control` | Default catalog models send no `cache_control`; hosts may opt in on Anthropic `/messages` models with `ModelConfig.cache.kind: "cache_control"`. | Keep selected Anthropic anchors and prior history stable. | Best-effort and model/route-dependent. |
| `@arnilo/prism-providers/neuralwatt` | `implicit` | No `cache_control`, `cacheKey`, `prompt_cache`, or `cacheRetention` payload; NeuralWatt vLLM prefix caching is automatic. | Full prior history must be resent unchanged with only the new turn appended; `inputLayout: "cache_aware"` keeps stable prefixes first. | Best-effort only; does not promise cache hits; `cacheRetention: "none"` disables Prism hints only, not the implicit backend prefix cache. |
| `@arnilo/prism-providers/ai-sdk` | host-owned | No Prism cache payload; host `LanguageModelV4` owns upstream caching. | Host model/provider decides cache keys, breakpoints, and sticky routing. | Adapter maps `inputTokens.cacheRead`/`cacheWrite` from `finish.usage` only; does not invent cache fields. |
| `@arnilo/prism-providers/alibaba` | implicit by default, optional `cache_control` | DashScope implicit prefix caching is automatic; opt-in `cache_control: {"type":"ephemeral"}` markers only on caller-selected `cache.breakpoints`, capped at 4. | Keep selected anchors and prior history stable; each cached prefix needs ≥1024 tokens and lives ~5 minutes upstream. | Best-effort and model-dependent; `cached_tokens`→read, `cache_creation_input_tokens`→write. |
| `@arnilo/prism-providers/ollama` | `implicit` | No `cache_control`, `cacheKey`, `prompt_cache`, or `cacheRetention` payload; Ollama KV/prefix caching is automatic with no request knob. | Resend unchanged prior history for implicit KV reuse. | Best-effort only; Ollama reports no cached-token count, so `Usage.cacheReadTokens` stays `undefined`. |
| `@arnilo/prism-providers/deepseek` | `implicit` | No `cache_control` / `prompt_cache_key`; tool `parameters` go through shared `canonicalizeJsonSchema`. | Resend unchanged history from token 0; append only the new turn. Thinking-on strips temperature/top_p/penalties so they cannot break the prefix. | Best-effort prefix units (~1024 practical min). `prompt_cache_hit_tokens` → `cacheReadTokens`. |
| `@arnilo/prism-providers/xai` | `implicit` | No `prompt_cache_key`. Package-local `x-grok-conv-id` is `sanitizeCacheKey(cache.key ?? cacheKey ?? sessionId, 128)`. | Same server + unchanged message prefix. Replay `reasoning_content` on reasoning models or the prefix breaks. | Conv-id is never a credential or SuperGrok token. Omitted when `cache.mode` is `off` or `cacheRetention` is `none`. `cached_tokens` → `cacheReadTokens` (inclusive or exclusive reports kept as-is). |
| `@arnilo/prism-providers/clinepass` | `implicit` | No `cache_control` / `prompt_cache_key`. Gateway-owned prefix cache. | Resend unchanged prior history. Stream only. | Best-effort and backend-dependent (`cline-pass/*` slugs). `cached_tokens` / `prompt_cache_hit_tokens` map when present. |
| `@arnilo/prism-providers/azure` | none | No Prism cache mapping. | Endpoint/model-specific. | Host owns Azure cache policy. |
| `@arnilo/prism-providers/bedrock` (`compatible`) | none | No Prism cache mapping. | Endpoint/model-specific. | Host owns Bedrock cache policy. |
| `@arnilo/prism-providers/bedrock` (`converse`) | `cache_control` | Prism breakpoints become standalone `cachePoint` blocks in `system`/message content; long retention adds `ttl: "1h"` when the model allows it. `tools` caching stays host-owned. | Stable prefix in the documented order `tools → system → messages`; changing an earlier section invalidates later ones. | `cacheReadInputTokens`/`cacheWriteInputTokens` map to `Usage.cacheReadTokens`/`Usage.cacheWriteTokens`; `inputTokens` is the non-cached remainder and is never folded. |
| `@arnilo/prism-providers/vertex` | none | No Prism cache mapping. | Endpoint/model-specific. | Host owns Vertex cache policy. |

Detailed first-party provider notes:

- OpenAI Responses (`@arnilo/prism-providers/openai`): `kind: "openai_key"`. Sanitizes/clamps `prompt_cache_key` to 64 chars; pre-GPT-5.6 models (`cache.longRetention: true`) map `"long"` retention to `prompt_cache_retention: "24h"`; GPT-5.6+ models (`cache.explicitBreakpoints: true`) map `cache.breakpoints`/`cache.mode: "on"` to `prompt_cache_options: { mode: "explicit" }` plus `prompt_cache_breakpoint: { mode: "explicit" }` markers on selected message anchors (≤4 writes; the only TTL `"30m"` is the default, so none is emitted). Resolved cache fields win over caller `extra`. `input_tokens_details.cached_tokens`/`cache_write_tokens` map to `Usage.cacheReadTokens`/`Usage.cacheWriteTokens`.
- OpenAI-compatible Chat Completions adapter: minimal scope, sends no cache payload; see [OpenAI-compatible provider](providers/openai-compatible.md).
- Anthropic (`@arnilo/prism-providers/anthropic`): `kind: "cache_control"`; selected Anthropic message anchors receive `cache_control` and eligible long retention maps to `ttl: "1h"`. A `system_prompt` breakpoint serializes `system` as native text blocks carrying the marker (shared `systemCacheControlField()` helper; plain joined string when unmarked). Cache read/create usage maps to normalized cache read/write tokens.
- Google (`@arnilo/prism-providers/google`): sends no Prism cache-control payload. Do not infer cache hits or cache token counts from absent Gemini fields.
- OpenRouter (`@arnilo/prism-providers/openrouter`): `kind: "cache_control"`. Sanitizes/clamps `session_id`/`X-Session-Id` to 256 chars for sticky routing (from `cache.key` ?? legacy `cacheKey` ?? `sessionId`); kernel defaults supply `system_prompt` + `last_stable_message` breakpoints so agent-session requests use per-message markers (not top-level automatic). Raw `generate` with empty breakpoints still emits top-level automatic `cache_control: { type: "ephemeral" }`; `"long"` retention adds `ttl: "1h"` when the model allows it. `prompt_tokens_details.cached_tokens`/`cache_write_tokens` map to `Usage.cacheReadTokens`/`cacheWriteTokens`. Optional `listOpenRouterModels()` may populate `ModelConfig.cache`/`cost` from live pricing.
- OpenCode Go (`@arnilo/prism-providers/opencode-go`): default base `https://opencode.ai/zen/go/v1`; `x-opencode-session` from `cacheKey ?? sessionId`, sanitized to 128 chars; the Anthropic route (MiniMax/Qwen) applies `cache_control` markers only to selected breakpoints (`"long"` → `ttl: "1h"`), the OpenAI route (Grok/GLM/Kimi/MiMo/DeepSeek) sends none and preserves `reasoning_content`. OpenAI route maps `prompt_tokens_details.cached_tokens`/`cache_write_tokens`; Anthropic route maps `cache_read_input_tokens`/`cache_creation_input_tokens`. Caller-gated `listOpenCodeGoModels` against official `GET /zen/go/v1/models`.
- Hyper (`@arnilo/prism-providers/hyper`): `kind: "implicit"` on the chat route, `kind: "cache_control"` on the messages route. Model route selection follows the live catalog's pricing shape: models with explicit cache-write pricing (qwen3.6-*) default to the Anthropic route; the rest stay chat-route implicit with the write fee recorded in `cost.cacheWrite`. The messages route applies `cache_control: { type: "ephemeral" }` only to caller-selected `cache.breakpoints` (shared `applyCacheControl`, max 4) — never every block, and **never a `ttl`** (Hyper documents no TTL values). Chat route maps `prompt_tokens_details.cached_tokens`/`cache_write_tokens` (and `prompt_cache_hit_tokens`); messages route maps `cache_read_input_tokens`/`cache_creation_input_tokens`. `parseHyperUsageCost` surfaces USD/cost/remaining Hypercredits from the OpenAI usage chunk for cost telemetry; `402 billing_error` is the drained-balance signal. Caller-gated `listHyperModels`; operator-gated `getHyperCredits`.
- Command Code (`@arnilo/prism-providers/commandcode`): `kind: "implicit"` on the chat route, `kind: "cache_control"` on the messages route. `claude-*` tiers (the only Anthropic-route models by server-enforced routing) default to `cache_control` with markers only on caller-selected `cache.breakpoints` max 4 — never every block, and **never a `ttl`** (the upstream TTL window is undocumented). GPT-5.6 sol/terra/luna keep the docs cache-write price in `cost.cacheWrite` but stay `implicit` until the live `prompt_cache_key` probe passes (plan 055 Task 9); all other chat-route models are implicit with `prompt_cache_hit_tokens`/`cached_tokens` mapped by the shared OpenAI usage mapping. Messages route maps `cache_read_input_tokens`/`cache_creation_input_tokens`. Billing caveats: OSS models bill at the mean per-provider price; DeepSeek off-peak rates (17h/day) with ~2× peak 01–04 & 06–10 UTC; deals (MiniMax M3, MiMo) auto-applied. Caller-gated `listCommandCodeModels`; optional ZDR (`zdr: true`) adds `x-cmd-zdr: 1` and may route to costlier upstreams or fail `422 cmd_zdr_no_providers`.
- Z.AI (`@arnilo/prism-providers/zai`): `kind: "implicit"`. GLM context caching is automatic; sends no explicit cache payload regardless of cache options. `prompt_tokens_details.cached_tokens`/`cache_write_tokens` map to `Usage.cacheReadTokens`/`cacheWriteTokens`.
- NeuralWatt (`@arnilo/prism-providers/neuralwatt`): `kind: "implicit"`. NeuralWatt prefix caching is automatic; sends no explicit cache payload regardless of cache options. `cacheRetention: "none"` disables Prism cache-control hints only (not the implicit backend prefix cache). `prompt_tokens_details.cached_tokens` maps to `Usage.cacheReadTokens`; NeuralWatt does not report a cache-write token, so `Usage.cacheWriteTokens` is never fabricated (stays `undefined`). NeuralWatt's `/v1/models` catalog advertises exact `cached_input_per_million` rates for cache reads and `cached_output_per_million: null`; static curated aliases do not guess those prices.
- Kimi (`@arnilo/prism-providers/kimi`): default catalog models use implicit caching (no `cache_control`); hosts opt in via `ModelConfig.cache.kind: "cache_control"` on the Anthropic `/messages` route, then `cache_control` markers apply only to selected breakpoints (`"long"` → `ttl: "1h"`); the Moonshot OpenAI route sends none. `cache_read_input_tokens`/`cache_creation_input_tokens` map to `Usage.cacheReadTokens`/`cacheWriteTokens`.
- AI SDK adapter (`@arnilo/prism-providers/ai-sdk`): **host-owned**. Sends no Prism cache payload; the supplied `LanguageModelV4` and its upstream provider own request caching. Maps AI SDK v4 `finish.usage.inputTokens.cacheRead`/`cacheWrite` to `Usage.cacheReadTokens`/`cacheWriteTokens`. No `list*Models()` export.
- Alibaba Cloud (`@arnilo/prism-providers/alibaba`): implicit by default, optional `cache_control`. DashScope implicit prefix caching is automatic (no marker); explicit opt-in `cache_control: {"type":"ephemeral"}` markers apply only to selected breakpoints when `ModelConfig.cache.kind: "cache_control"` and the caller supplies breakpoints, capped at 4 (each prefix ≥1024 tokens, ~5 minute TTL). `prompt_tokens_details.cached_tokens`/`cache_creation_input_tokens` map to `Usage.cacheReadTokens`/`cacheWriteTokens`. Caller-gated `listAlibabaModels` against OpenAI-compatible `GET {base}/models`.
- Ollama (`@arnilo/prism-providers/ollama`): `kind: "implicit"`. Ollama reuses its KV/prompt cache automatically; there is no request knob and no wire marker, so Prism never emits `cache_control`. Ollama reports no cached-token count, so `Usage.cacheReadTokens` is intentionally left `undefined` (not `0`). Caller-gated `listOllamaModels` against OpenAI-compatible `GET {base}/models`.
- DeepSeek (`@arnilo/prism-providers/deepseek`): `kind: "implicit"`. Official disk prefix cache is automatic (byte-identical prefix from token 0). Adapter sends no cache payload; tool `parameters` use shared `canonicalizeJsonSchema` (object keys + unordered `required` only; `enum`/`prefixItems`/`examples` keep caller order). `prompt_cache_hit_tokens` maps to `Usage.cacheReadTokens`. Caller-gated `listDeepSeekModels`.
- xAI (`@arnilo/prism-providers/xai`): `kind: "implicit"`. Automatic prefix cache. Sticky `x-grok-conv-id` is a sanitized session/cache key (128 chars), never an OAuth access token. Reasoning models must replay `reasoning_content`. `prompt_tokens_details.cached_tokens` maps to `Usage.cacheReadTokens`. Caller-gated `listXaiModels`.
- ClinePass (`@arnilo/prism-providers/clinepass`): `kind: "implicit"`. No explicit cache payload; multi-backend gateway may report `cached_tokens` or `prompt_cache_hit_tokens`. Static `cline-pass/*` catalog only — no `listClinePassModels`.
- Azure, Bedrock, and Vertex: their OpenAI-compatible packages intentionally emit no Prism cache fields. Endpoint/model-specific cache controls remain host-owned rather than guessed from another provider family. Bedrock's native `converse` route is the exception: it is a documented cache-control surface (`cachePoint`, shared `applyCacheControl` markers) and maps cache usage fields instead of leaving them host-owned.

### NeuralWatt cache-aware limiter

NeuralWatt (`@arnilo/prism-providers/neuralwatt`) runs a cache-aware backend rate
limiter on top of its implicit vLLM prefix cache. This shapes long-running agent
sessions differently from one-shot chat:

- **Uncached TPM counts cold prefill only.** The tokens-per-minute budget charges the
  prefix that is not already cached. A request whose prefix is fully cached consumes
  far less TPM than a cold request of the same total prompt length.
- **Warm-prefix requests can avoid some `503` fleet-capacity blocks.** Near fleet
  capacity, requests that reuse a cached prefix are more likely to be admitted than
  fully cold requests. Prefix reuse is both an availability and a latency lever.
- **Full prior history is required for multi-turn cache reuse.** The prefix cache is
  keyed by request content, so each follow-up turn must resend the entire prior
  transcript (system prompt + all prior turns) unchanged, with only the new turn
  appended. Use `inputLayout: "cache_aware"` so Prism keeps the stable prefix first.
- Cache behavior is best-effort and **does not guarantee cache hits**. Admission and
  eviction are server-side decisions and vary with fleet load. `cacheRetention:
  "none"` disables Prism cache-control hints only; it does not disable the implicit
  backend prefix cache.

See [NeuralWatt provider](providers/neuralwatt.md) for the package-level cache,
usage, and retry details.

## Discovery and live cache/cost metadata

Caller-gated `list*Models()` helpers (see [Provider packages — Caller-gated model discovery](provider-packages.md#caller-gated-model-discovery)) may map official list-models fields onto `ModelConfig`:

- `cache` — when the provider documents cache kind / long-retention / breakpoint support in model metadata (otherwise keep the package's known default, e.g. NeuralWatt/Z.AI `implicit`, OpenAI `openai_key`).
- `cost` — when the provider documents per-token or per-million rates, including cache-read rates such as NeuralWatt `cached_input_per_million`.

Static featured catalogs remain offline bootstrap and must **not** invent pricing or cache capabilities the official docs do not state. Discovery is never invoked by `create*ProviderPackage()`; hosts that want live `cost`/`cache` pass the returned models into package `models:` (or register them themselves).

## Security and performance notes

- Cache hints are best-effort and do not guarantee cache hits.
- Cache keys are untrusted input; sanitize and truncate with `sanitizeCacheKey()` before provider I/O.
- Cache keys must never be credentials or secrets.
- Provider-owned auth/session/security headers always win over caller headers.
- Helpers are pure, network-free, and O(messages) at most. `cacheUsageReport()` is O(1).
- Cache-aware input ordering does not change resource loading: URI attachments/resources still load only through the caller-provided `ResourceLoader`.
- Cache usage reports contain only usage counts and optional pricing/currency; they do not include prompt text, cache keys, headers, credentials, or provider payloads.
- `applyCacheControl()` returns new message objects for stamped anchors and does not mutate input messages.

## Cache telemetry

### What it does

`createCacheTelemetry()` is a dependency-free aggregator that turns the per-call
`Usage.cacheReadTokens`/`cacheWriteTokens` counters into per-provider/model
statistics hosts can use to tune the `cache_aware` input layout: request count,
cache-read/write token totals, hit rate, and an estimated read-token savings
when the model carries cost metadata.

### When to use it

Use it when you want to observe cache effectiveness per provider/model over a
session, a day, or a run ledger. It is opt-in by construction: importing the
module never collects anything — the host explicitly wires `record()` to its
`usage` `ProviderEvent` stream or to run-ledger usage records.

### Inputs / request

| Input | Meaning |
| --- | --- |
| `usage` (`Usage`) | One usage record: `cacheReadTokens`, `cacheWriteTokens`, `inputTokens` are validated (non-negative safe integers; a violation throws `CacheTelemetryError` and mutates nothing). |
| `model` (`ModelConfig?`) | Attribution key (`provider` + `model`). Omit it for provider-only aggregation into the `unknown` bucket. Cost metadata (`ModelCost.input`/`cacheRead`) enables `estimatedSavings`. |
| `options.maxKeys` | Distinct provider/model keys before excess keys collapse into the `__overflow__` bucket (default `DEFAULT_CACHE_TELEMETRY_CAP` = 256). |

### Outputs / response / events

`report()` returns `{ samples, overflowed, totalRequests, totalCacheReadTokens,
totalCacheWriteTokens }`. Each sample carries `provider`, `model`, `requests`,
`cacheReadTokens`, `cacheWriteTokens`, `inputTokens`, `hitRate` (total reads /
total input — the same math as `cacheHitRate()`), and `estimatedSavings` with
`currency` only when the model has cost metadata. Samples are sorted by
provider then model. `reset()` clears all samples; `size` reports the current
distinct-key count.

### Request/response example

```ts
import { createCacheTelemetry } from "@arnilo/prism";

const telemetry = createCacheTelemetry();
for await (const event of provider.generate(request)) {
  if (event.type === "usage") telemetry.record(event.usage, request.model);
}

const report = telemetry.report();
for (const sample of report.samples) {
  console.log(sample.provider, sample.model, sample.hitRate, sample.cacheReadTokens);
}
```

### Security and performance notes

- Reports carry token counters, rates, currency, and provider/model names only
  — never prompt content, cache keys, headers, credentials, or identity fields
  (redaction-safe by construction).
- Cardinality is bounded: beyond `maxKeys` distinct provider/model keys, excess
  keys accumulate in a single `__overflow__` bucket; memory cannot grow with
  hostile model names (`ponytail:` ceiling — upgrade to host-configurable caps
  or LRU eviction only if a real deployment exceeds it). The `__overflow__`
  bucket aggregates mixed provider/model tokens, so it never carries cost
  metadata: `estimatedSavings`/`currency` are unset there and it reports
  requests and token totals only.
- `record()` is O(1) per usage event; `report()` is O(keys). No secrets or
  cache keys are accepted or stored.

## Related APIs

- [Input and prompt assembly](input-and-prompt-assembly.md): opt-in cache-aware ordering for stable provider payload prefixes.
- [Attention compiler](attention-compiler.md): opt-in per-turn shrink that only rewrites rows *behind* the stable prefix, so cache hits survive.
- [Provider request policies](provider-request-policies.md): set cache hints before provider calls.
- [Model registry](model-registry.md): register `ModelConfig.cache` capability metadata.
- [Provider layer](provider-layer.md): provider/model registries and provider events.
- [Provider packages](provider-packages.md): package-owned mapping to provider-native cache APIs.
- [Public contracts](public-contracts.md): public type list for cache contracts and helpers.