# @verevoir/llm

Provider-agnostic LLM call surface with token + cost accounting.

## Purpose

Lets a downstream project make LLM calls with retry-friendly errors, token
accounting, per-direction cost tracking, and tool-calling, without coupling to a
specific provider SDK. Pick the provider you use via a subpath import; the unused
SDK does not enter your bundle.

## Subpaths

- `@verevoir/llm` — core types, the `chat()` / tool-calling contracts, cost
  accounting, and the model catalog + provider-routing helpers. No SDK dependency.
- `@verevoir/llm/anthropic` — Anthropic SDK adapter (`chat`, `chatWithTools`,
  `chatWithToolLoop`, live `onProgress`). Requires `@anthropic-ai/sdk`.
- `@verevoir/llm/google` — Google Gemini SDK adapter (`chat`, `chatWithTools`,
  `chatWithToolLoop` as of 0.13.0). Requires `@google/genai`.
- `@verevoir/llm/openai` — OpenAI adapter via the Responses API
  (`chat`; `chatWithTools` + `chatWithToolLoop` via the OpenAI-compatible factory).
  Requires `openai`. The only keyless-capable adapter: a base-URL override with a
  blank key points it at a local server (LM Studio / Ollama / vLLM).
- `@verevoir/llm/deepseek` — DeepSeek adapter, OpenAI-compatible via the Chat
  Completions API at `https://api.deepseek.com`. Reuses the `openai` peer — no
  separate SDK. `chat` + tool-calling.
- `@verevoir/llm/samba` — SambaNova adapter (`SAMBA_NOVA_API_KEY`),
  OpenAI-compatible. `chat` + tool-calling.
- `@verevoir/llm/mistral` — Mistral adapter (`MISTRAL_API_KEY`),
  OpenAI-compatible. `chat` + tool-calling.
- `@verevoir/llm/claude-cli` — shells out to the real Claude Code CLI
  (`claude -p`) as a subscription-only substrate, for a caller that must run on
  a subscription credential rather than a billed API key (imitating Claude Code
  to the raw Messages API is not a sanctioned route). Provider id `'claude-cli'`,
  deliberately distinct from `'anthropic'`; never registered into the model
  catalog or connection registry — `resolveModel`/`resolveModelByTerm` can never
  silently substitute it, a caller wanting this substrate imports and calls it
  directly. `chat()` only. No SDK dependency — it spawns a subprocess, and
  that subprocess's environment is built from an allowlist — only `PATH`,
  `HOME`, `TMPDIR`, and the CLI's own subscription-credential env var
  (`CLAUDE_CODE_OAUTH_TOKEN`) ever reach it; nothing else in the caller's
  environment, billed credential or not, is passed through. Replaced an
  earlier denylist that had already missed one live credential vector
  (`ANTHROPIC_AUTH_TOKEN`) in an earlier version — a denylist can only
  exclude names someone thought of; the allowlist doesn't need to.

Every provider in the matrix (Anthropic, Google, OpenAI, DeepSeek, SambaNova,
Mistral) can now drive the tool loop. `claude-cli` is a separate, deliberately
non-interchangeable substrate and is not part of that matrix.

- `@verevoir/llm/catalog` — registers every catalogued provider's model
  catalogue (anthropic, mistral, samba) AND connection (env var names only) as
  an import side effect, with **no SDK import anywhere in its import graph** —
  unlike the real adapter subpaths above, each of which requires its SDK as a
  peer dependency. For a consumer that only needs `resolveModelByTerm`, a
  label, or a rate (never a real call) — e.g. selecting a reviewer model by
  name — importing this subpath resolves it with zero SDK as a transitive
  dependency. Also re-exports the raw `ANTHROPIC_CATALOG` / `MISTRAL_CATALOG` /
  `SAMBA_CATALOG` arrays for a consumer that wants the data directly. The real
  adapter subpaths are unchanged for an existing consumer — each still
  registers its own catalogue + connection itself, idempotently, so importing
  both this subpath and a real adapter is a no-op the second time, not a
  conflict. `/openai`, `/deepseek`, `/google` are not included — they never
  used `ModelCatalogEntry`/`registerModelCatalog` in the first place. `/claude-cli`
  stays deliberately unregistered, per its own design.

## Install

```bash
npm install @verevoir/llm @anthropic-ai/sdk
```

The package's `peerDependencies` are marked optional — install only the SDK(s)
you import (`@anthropic-ai/sdk`, `@google/genai`, `openai`; the OpenAI-compatible
providers share `openai`).

## Canonical usage

```ts
import { anthropic } from '@verevoir/llm/anthropic';

const reply = await anthropic.chat({
  systemPrompt: '...',
  turns: [{ role: 'user', content: '...' }],
  // Omit apiKey when auth is ambient: it resolves from the env
  // (CLAUDE_CODE_OAUTH_TOKEN preferred, else ANTHROPIC_API_KEY). Pass apiKey
  // ONLY to force a specific key (BYOK) — an explicit key short-circuits OAuth.
  modelClass: 'reasoning', // 'reasoning' | 'drafting' | 'extraction'
});

// reply.content: string
// reply.usage:   { provider, model, direction, inputTokens, outputTokens, ... }
```

## Tool calling

Each adapter exposes, alongside `chat()`:
- `chatWithTools(options)` — single-shot; surfaces the model's `ToolUse[]` for
  the caller to execute.
- `chatWithToolLoop(options)` — runs model → execute tools → feed results back →
  repeat, to a `maxIterations` cap. Takes a `ToolExecutor`
  (`(toolUse) => Promise<string | ContentBlock[]>`) plus per-iteration
  `onUsage` / `onIteration` hooks; usage is summed across the loop.

Provider-agnostic types `ToolDef`, `ToolUse`, `ToolExecutor`, `ContentBlock`
(`text` | `tool_use` | `tool_result` | `image` | `document`), plus
`UnsupportedContentBlockError` and `collapseContentToText(provider, content)`:
a provider that cannot carry a block type (only `@verevoir/llm/anthropic`
carries `image`/`document`) throws this error, naming the provider and
block type. Inside a tool executor's result it becomes a named `is_error`
tool_result; inside a user turn the caller built, it throws. The
`ChatWithTools*` / `ChatWithToolLoop*` option/result shapes live on the
core export.

## Cost accounting

Every call returns a `TokenUsage` typed
`{ provider, model, direction, route, inputTokens, outputTokens, ... }` (with
optional `substrateVersion` — which version of an external substrate, e.g. the
`claude` CLI, served the call, `undefined` for a direct API adapter — and
optional `cacheRead` / `cacheWrite` so prompt-cache savings price at their own
rates). `route` (`CredentialRoute`, a core export: `'api-key'` |
`'subscription-oauth'` | `'mixed'`) reports which credential mechanism actually
served a call, positively, never left to be inferred from what's absent. The
core export ships `sumUsages` / `parseUsage` to roll up across calls,
`totalTokens`, `formatTokensCompact` for compact displays, and
`createPriceBook` / `book.costOf(...)` — the cost API, pricing from
LiteLLM's live price table (a bundled snapshot as fallback) and reporting an
unmatched model as `unpriced`, never `0`. The older `estimateCostUSD`,
`RatesTable` / `RateTuple`, `catalogEntryFor`, and `uncoveredModels` are kept
for back-compat but `@deprecated` — new code should use `createPriceBook`.

The `direction` field carries the model-class semantic (`reasoning` /
`drafting` / `extraction`) so per-conversation / per-project rollups can break
down "spent X on reasoning + Y on extraction" natively.

For auditing, `setModelSpanSink(sink)` (core export) registers an optional
process-wide sink that receives a `ModelSpan` — the call's `TokenUsage` plus the
emitting `scope`, e.g. `samba.chatWithToolLoop` — once per underlying model call,
from **every** adapter and every entry point (`chat`, `chatWithTools`, and
`chatWithToolLoop`). A `chatWithToolLoop` call emits one span per iteration with
that iteration's own usage, so expect multiple spans from a single loop call.
Off by default; fail-soft; pass `null` to detach.

## Advisor pair

`withAdvisor(tools, executor, advisor)` (core export, SDK-free) wraps a tool
set + executor so a cheap executor model can consult a stronger advisor
mid-loop: it appends a `consult_advisor` ToolDef (question required, context
optional; input never mutated, name collision throws at wrap time) and routes
consult calls to the injected `advisor.chat` — any adapter's `chat`, bound by
the caller — with the advisor's `systemPrompt` and `modelClass: 'reasoning'`.
The advisor guides, it never certifies: its reply returns as an ordinary tool
result. A failing advisor yields a legible "advisor unavailable" result rather
than a throw, so a dead advisor never kills the work; a missing or empty
question returns a legible ask-again result without calling the advisor.
Optional `onConsult`
hook (fail-soft, not fired on advisor failure) reports each consult's
question/context/answer/usage for consult-rate metrics; the advisor's own
adapter emits its model span as usual.

## Model catalog + provider routing

Model identity is de-brittled — decisions key on `provider/family`, not the exact
version string, so a version bump doesn't zero a model's cost.

- `registerModelCatalog(entries)` / `ModelCatalogEntry` — an adapter declares each
  family once; `models` / `rates` / labels derive from it. `normalizeModelId(id)`
  resolves a drifted version forward to `{ provider, family }`; `catalogEntryFor`,
  `modelLabel`, `registerModelLabels` round it out.
- Provider connection registry — adapters register their connection on import, so
  routing can answer `isProviderConfigured` / `configuredProviders`,
  `providersForFamily(family)` (the inverse index), and `resolveModel({ family?, modelClass?, configuredOnly?, prefer? })`
  (cheapest configured by default). `resolveModelByTerm(term, opts)` resolves a
  loose human term ("deepseek") to a catalog entry; `modelConnection(term, opts)`
  returns a ready OpenAI-compatible `{ provider, modelId, baseUrl, apiKey }`.
  A connection may also declare `altKeyEnvs` — additional env vars whose presence makes
  the provider usable, because the credential CHECK must recognise every credential the
  CALL path accepts. Anthropic declares `CLAUDE_CODE_OAUTH_TOKEN`, so the subscription
  token alone configures it, with no `ANTHROPIC_API_KEY` needed.
- `<PROVIDER>_BASE_URL` overrides — every adapter honours an env override for its
  endpoint (`OPENAI_BASE_URL`, `ANTHROPIC_BASE_URL`, `GEMINI_BASE_URL` /
  `GOOGLE_BASE_URL`, `MISTRAL_BASE_URL`, `DEEPSEEK_BASE_URL`, `SAMBA_NOVA_BASE_URL`),
  so a provider can be pointed at a gateway / proxy / regional / self-hosted
  endpoint without a code change. The override is keyed by **provider/endpoint,
  not model**: running DeepSeek-V3 *via SambaNova* uses `SAMBA_NOVA_*`, not
  `DEEPSEEK_*`. Setting only `OPENAI_BASE_URL` with no key is treated as a
  **keyless local endpoint** (LM Studio / Ollama / vLLM) — the generic OpenAI
  client builds with a placeholder key and routing counts it usable. Every other
  hosted provider (Anthropic, Gemini, SambaNova, Mistral, DeepSeek) always needs
  its key; a base-URL override alone does not make it configured. Routing sees
  only providers whose subpath has been imported.
- Anthropic subscription auth — the Anthropic adapter prefers
  `CLAUDE_CODE_OAUTH_TOKEN` (a Claude subscription token from `claude setup-token`)
  over the metered `ANTHROPIC_API_KEY`, sending it as a bearer token with the
  required `anthropic-beta` flag + a Claude-Code system identity (both overridable
  via `ANTHROPIC_OAUTH_BETA` / `ANTHROPIC_OAUTH_SYSTEM`). Precedence: an explicit
  per-call `apiKey` → `CLAUDE_CODE_OAUTH_TOKEN` → `ANTHROPIC_API_KEY` (fallback
  only), so a set subscription token means the metered key is never billed.

## Status

Pre-stable: the `0.x` line communicates that the API surface can shift before
`1.0`. Track changes in the repo's CHANGELOG.

## Repository

`git@github.com:verevoir/llm.git`

## License

Apache-2.0.