# USD model gateway for local agents

A plan has a USD allowance, one multiplier, and a live selection of conversation models and model tools. The Hub relays individual requests and meters their cost. The host keeps its original planning, tool selection and AgentLoop. Business authorization, approval, session scope, task controls and conversation archives remain separate.

The current release pairs Core 0.10.0, SDK 0.8.0 and DSH 0.8.0. It has no legacy Token-allowance aliases, silent billing fallback, balance conversion or automatic BYOK substitution. The optional gateway first shipped with Core 0.9.0 and SDK/DSH 0.7.0; its original upgrade from Core 0.8.0 required outstanding migrations 063/064. For the current release, follow the Core upgrade guide and apply only outstanding migrations while retaining existing data. Do not erase business authorizations, history or unrelated sessions; abandoned test configuration is separate maintenance, not a customer upgrade requirement.

## Identity and catalog

The trusted product backend issues a Usage credential for the original Hub/user/account. `model_access:'token_gateway'` remains the credential scope spelling for the whole permitted plan; it does not select a Token billing algorithm. A deliberately single-service credential stays restricted to its original service. Supplier URLs and API keys stay server-side.

```js
import { createUsageClient } from 'bailinghub-mcp-server/usage'
const client = createUsageClient({
  hubUrl: trustedLogin.hubUrl,
  userId: trustedLogin.userId,
  accountId: trustedLogin.accountId,
  serviceId: trustedLogin.serviceId,
  accessTokenProvider: () => secureCredentials.readUsageToken(),
})
await client.capabilities()
const models = await client.modelModels()
const tools = await client.modelTools()
const summary = await client.modelSummary()
```

Capabilities must advertise `model_gateway` with schema `bailing.model-gateway.v1`, `billing_unit:'USD'`, `orchestration:'host'`, `turn_required:false`, streaming, native provider response envelopes and asynchronous settlement. Old Core/SDK returns explicit unsupported, not an alternative billing path.

`modelModels()` reads `/usage/v1/model/models`, schema `bailing.model-models.v1`. Its `items` are only conversation models. Render the operator's `label`, store the immutable `service_id`, never route by label or provider model name. Every read refreshes current plan membership. An empty directory is valid and does not authorize choosing an unrelated model.

`modelTools()` reads `/usage/v1/model/tools`, schema `bailing.model-tools.v1`. Each item has a stable `service_id`, `service_revision`, display label, capability/output kinds and a `tool:{name,description,input_schema}`. Register only `callable:true` items in the host's local tool registry. Keep unavailable items as configuration diagnostics with their explicit `reason`. Catalog reads neither generate media nor start business runs. Namespace registrations by service identity to avoid duplicate operator-defined names or collisions with business tools; tool names alone are never authorization identities.

## Output budgets and provider semantics

`max_output_tokens` is the operator's service ceiling, not the supplier's model capability. Explicit output policies add these three fields together to `bailing.usage-model.v1`:

| Field | Meaning |
| --- | --- |
| `default_output_tokens` | Budget used when the caller does not choose a value; a positive integer no greater than the service ceiling |
| `output_token_parameter` | The exact provider option, `max_tokens` or `max_completion_tokens` |
| `output_budget_scope` | `completion_total` includes reported reasoning and answer output; `final_answer` limits the answer only |
| `model_max_output_tokens` (optional) | Verified supplier maximum for this same parameter and scope; the service ceiling cannot exceed it |

`max_completion_tokens` requires `completion_total`. `max_tokens` can have either scope, depending on the verified supplier route. The SDK does not infer semantics from a model name or assume that an OpenAI-compatible endpoint gives identically named parameters the same meaning. A supplier's thought budget is not an extra allowance to add to a total output ceiling. Image tools use their own parameters and do not adopt this conversation-token policy.

An explicit descriptor's `provider_options` includes only the chosen output key, alongside its other supported options. A partial or contradictory policy is `USAGE_RESPONSE_INVALID`; it does not fall back to legacy. Gateways can advertise the additive `model_gateway.output_budget` capability with schema `bailing.model-output-budget.v1` and `separate_default`, `explicit_parameter`, `explicit_scope` all `true`.

Old descriptors omit all these fields and keep their old behavior: the service ceiling doubles as the default and the request can use either previously supported output key. This is a compatibility fallback, not a newly verified default, scope or supplier maximum. Old consumers will still use the ceiling as their default even if an operator adds the new fields; update the consumer before enabling a policy that requires different semantics.

Use the public helpers for **newly prepared requests**:

```js
import { resolveModelOutputBudget, modelOutputOptions } from 'bailinghub-mcp-server/usage'

const catalog = await client.modelModels()
const descriptor = catalog.items.find(item => item.service_id === selectedServiceId)
if (!descriptor) throw new Error('Select a currently available service')
const budget = resolveModelOutputBudget(descriptor)

// The host can use this as its native defaultMaxTokens. Legacy is explicitly marked.
const defaultMaxTokens = budget.defaultTokens
// Omit tokens to use that default, or supply a deliberate per-request budget.
const outputOptions = modelOutputOptions(descriptor, { tokens: requestedTokens })
const providerOptions = { top_p: 0.9, ...outputOptions }
```

`resolveModelOutputBudget()` returns `mode:'explicit'|'legacy'`, `defaultTokens`, `maxTokens`, `parameter`, `scope` and `modelMaxTokens`. For legacy mode, the last three are `null`; `defaultTokens` is only the old ceiling fallback. `modelOutputOptions()` builds one fresh output option, rejects values above the ceiling, and never clamps them. Its optional `parameter` argument can select a legacy supported key or confirm the explicit key; a conflicting explicit key gives `USAGE_OUTPUT_PARAMETER_MISMATCH` before dispatch. The helpers do no I/O and never mutate a descriptor or an existing request.

The host must build the option at its existing preparation boundary, using the selected descriptor and an intentional budget with the declared scope. Do not merge it over a different output key left by a serializer. Do not silently rename an existing `max_tokens` into `max_completion_tokens`, reinterpret a final-answer amount as a total amount, or patch a saved intent. Persist the selected descriptor and the final wire request with their existing digests. A new descriptor applies to a new request only. An old prepared request that has not yet reached Core may be rejected after a policy change; resolve that rejection explicitly and prepare a new operation rather than modifying the original intent. Already dispatched operations retain read-only recovery by their original ID.

`USAGE_OUTPUT_PARAMETER_MISMATCH` exposes safe `feedback.output_budget:{expected_parameter,scope}`. `USAGE_OUTPUT_LIMIT` can expose `feedback.output_limit:{actual,allowed,parameter,scope?}`. The SDK retains only validated scalar fields, not messages, credentials or provider bodies. These pre-dispatch rejections do not authorize an automatic retry or a model switch.

## Long tasks and output truncation

The host owns the Agent loop. The Hub and SDK do not turn a provider length termination into another POST, a model switch, or a replay of business tools. `result_state:complete` means the provider exchange is durably available; it does not say the answer or Agent task is complete. A usable response with `billing_state:pending` remains valid and does not require waiting for settlement before the next authorized step.

Interpret the original `bailing.provider-response.v1` JSON/SSE through the host's existing provider adapter. Only a positively recognized output-limit termination plus a final original operation receipt can lead to a new continuation step. Persist the partial answer, original model and tool receipts, and remaining task checkpoint first. The host then prepares a fresh request with a new operation ID; it must not change a saved intent or reuse its ID with a different body. Unknown finish reasons, token counts equal to a budget, or a short visible answer do not by themselves authorize continuation.

If the final operation frame is missing, the model POST acknowledgement is lost, or dispatch is unknown, inspect the original ID through `inspectModelRequest()` instead. A partial provider stream that contains `length` still needs the final durable receipt before a continuation decision. Never issue a replacement model POST to escape uncertain state. Do not replay already completed tools: model-operation idempotency does not deduplicate a new tool invocation that represents the same business action.

The host should decide continuation before its final turn-stop/close fence, preserve one real user task, impose bounded automatic continuation and no-progress/task budgets, and honor cancellation, approval and authorization changes. Do not fabricate human continuation messages or silently reopen a closed turn. Truncated tool calls remain non-executable even if a fragment happens to parse as JSON; use the native adapter's complete tool-call semantics and existing governance before any business dispatch.

The separate AgentClient `inspectInvocation()` is a read-only receipt query; `resume()` can perform the first business dispatch of an approved invocation and must not be used for polling. Business action checkpoints and original invocation IDs are host responsibilities, not a global tool-name/argument deduplication feature.

## Shared allowance and display

`modelSummary()` returns `bailing.billing-summary.v1`, with USD fields `availableUsd`, `consumedUsd`, `currentPeriodConsumedUsd`, `overageUsd`, the original grant and current plan. The plan supplies `serviceIds` and one `multiplier`; per-model input/output multipliers and Token balances no longer exist.

Use `presentation` (`bailing.usage-presentation.v1`) for customer balances: allowance packs show credits; periodic plans show remaining percentage and reset time. Never recalculate balances locally from raw tokens, prices or a previous model response. Missing/malformed presentation is `USAGE_RESPONSE_INVALID`, not a guessed balance. Account details may show reported provider Token counts as usage information, separate from allowance and percentage. If unavailable, show not provided, never fabricate zero Tokens.

Reference provider pricing is not an actual supplier invoice or guaranteed margin. Each request snapshots its plan multiplier and reference price; server-side USD accounting is exact to twelve decimal places. Receipt fields `reference_cost_usd`, `billed_usd`, `overage_usd` are amounts; `usage` contains integer Token counts when reported, and `raw_usage` can contain non-Token measurements. Unknown usage keeps billing pending. No client-side multiplication, second debit or extra model request is allowed.

## Request and recovery API

| SDK method | Behavior |
| --- | --- |
| `modelComplete(input)` | One JSON model request, no server agent loop |
| `modelStream(input)` | Provider packets followed by the original durable operation receipt |
| `runModelTool(input)` | One model-tool generation request |
| `inspectModelRequest(operationId, originalCoordinates)` | Read original operation only |
| `cancelModelRequest(operationId, originalCoordinates)` | Cancel original operation, never replay it |

Conversation requests use `/usage/v1/model/requests` and `/usage/v1/model/requests/stream`. Tool requests use `POST /usage/v1/model/tools/requests` with `{operation_id,service_id,arguments,conversation_id?,turn_id?}`. A tool POST can return HTTP 202 with a durably admitted pending operation. Poll the original request until complete or cancelled; do not submit another generation. All use the same original request inspection/cancellation path and `bailing.model-operation.v1` receipt. Streaming frames use `bailing.model-stream.v1`.

A completed model result can have `billing_state:'pending'`; billing settlement must not stop the local loop. Native provider response bytes are preserved in `bailing.provider-response.v1`. The local provider adapter interprets content, finish reason and tool calls. A tool result uses `bailing.model-tool-result.v1` with typed outputs. Downloading or using a result in a business attachment space is an explicit subsequent host action, never an automatic business write by the Hub.

Persist the original operation ID, selected service and local event coordinates before any dispatch. If a POST loses its ACK, inspect that original ID; do not create a new ID, regenerate an image, change service, retry through BYOK or label the operation not dispatched. Read/cancel failures preserve uncertainty. Storage failure and recovery gaps remain primary errors.

## Host integration

The host must actually register executable model tools; putting their description in a prompt alone is insufficient. Present concise guidance alongside current registered tools: “These are the model tools available under this plan. Use the supplied schemas. When an operation is pending or its result is uncertain, recover the original operation ID rather than repeating generation.”

The server still rechecks the live plan and selected service on each new request. A cached descriptor is not a permission grant. Business tool scope and approvals are not bypassed by generating a media file.

### Image provider outcome receipts

`result_state=failed`, `state=failed`, `dispatch=rejected` is a confirmed provider rejection. Stop waiting; show the bounded `error` and `next_action=contact_operator`. Never automatically create a replacement operation. `unknown` is not a running task or proof of zero cost: preserve the original ID for inspection. Only `pending` means keep waiting. A complete result is usable independently of pending billing.

The diagnostic whitelist is `code`, fixed safe `message`, optional `http_status`, `provider_code`, `provider_request_id`, `retryable=false`, and `next_action`. Provider bodies and credentials are excluded. Cancelled/ended turns never regain active tools through late receipts. Existing durable request identities and prior unknown records remain unchanged.