/** * Server-owned provider session state for the auth-gateway. * * `SimpleStreamOptions.providerSessionState` is how a provider keeps what it * learned about an endpoint across turns of one conversation: Anthropic's * sticky `strictToolsDisabled` / `fastModeDisabled` / * `replayUnsignedThinkingDisabled` flags and dropped-thinking-prefix set, * OpenAI's strict-tools and reasoning-effort fallbacks, Codex's WebSocket and * turn-state sessions. An in-process omp session owns that `Map` for its whole * lifetime, so a grammar-too-large 400 or a fast-mode rejection costs one * wasted round-trip per session rather than one per turn. * * The map is deliberately non-serializable — `Set`/`Map` fields, live sockets, * a `close()` method — so `pi-native-client` strips it from the wire and * `pi-native-server` never accepts it. Gateway clients therefore cannot bring * their own, and without a server-side owner every containerized / robomp turn * re-learns every lesson from a fresh upstream rejection. * * A plain `Map` in a long-lived server process is a leak: nothing * ever reclaims an entry, and the entries own timers and sockets. This store is * an LRU with a hard entry ceiling that calls `close()` on everything it drops * and on everything it still holds at shutdown — but it only ever drops an * entry no request is holding, because `close()` on a live entry tears down * state an in-flight stream is still streaming through. */ import type { Api, Context, Model, ProviderSessionState } from "../types.js"; /** * Retained logical sessions. Each entry is a handful of small provider records * plus, for Codex, a WebSocket session — cheap to keep, but not free, so the * ceiling is what turns "one entry per session id forever" into a bounded cost. * Eviction is least-recently-used, so the ceiling only ever drops sessions that * have been quiet longer than the 256 most recent ones. */ export declare const AUTH_GATEWAY_MAX_SESSION_STATES = 256; /** * One request's claim on a retained session. * * `release()` is what makes the entry evictable again, so it MUST run for every * outcome of the request — a `finally` at the call site for the synchronous * paths, stream completion for the streaming ones. It is idempotent, so the * two can overlap. */ export interface AuthGatewaySessionStateLease { /** The map to hand to `streamSimple` as `providerSessionState`. */ readonly states: Map; /** Reset account-scoped records if an in-request auth retry switches accounts. */ updateAccount(account: string): void; /** Give up this request's claim. Idempotent. */ release(): void; } /** Everything the store needs to place one request on a retained session. */ export interface AuthGatewaySessionStateRequest { /** * The client's own session key (`prompt_cache_key` / `sessionId`), or * `undefined` when it sent none — blank counts as none. A supplied key is * authoritative: the client is telling us which conversation this is. */ clientKey: string | undefined; model: Model; /** * System prompt, tools and message history of this request. Used only when * `clientKey` is absent, to place the request on the conversation it * continues. */ context: Context; /** * Stable identity of the account this request's credential resolved to. * A change means the gateway switched the session to a sibling credential, * so the account-dependent lessons in the retained map are re-probed. The * comparison happens on acquire and whenever an in-request auth retry * resolves a sibling credential. */ account: string; } /** * Bounded per-session provider state, owned by one gateway server instance. * * Two gateways in the same process get separate stores, so neither can hand a * request another gateway's learned state or close it out from under one. * * The recency order is this class's own (a `Map` iterates in insertion order, * and every acquire re-inserts) rather than `LRUCache`'s, because the policy * needs two things a general cache cannot express: an entry that a request is * still holding must be skipped when picking a victim, and an entry must be * able to change key — `LRUCache` disposes on every removal, which is precisely * the `close()` we must not run here. */ export declare class AuthGatewaySessionStateStore { #private; constructor(max?: number); /** Retained logical sessions. */ get size(): number; /** * Claim the provider-session map for one request, created on first use and * returned by reference so provider mutations persist into the next request. * * Keyed by provider + model + conversation (see {@link sessionKeys}). A * client is free to reuse one session id across models, and the coarsest * provider entries do not separate models themselves (`openai-responses` * keys its strict-tools / history-replay record by provider alone, * Antigravity by a single constant), so the model belongs in the key here. * Endpoint is deliberately absent: every provider whose learning is * endpoint-specific already sub-keys it internally * (`anthropic-messages:${baseUrl}\0${modelId}`, * `openai-completions:${provider}:${baseUrl}:${modelId}`), and repeating it * would only fragment the map. The credential is absent for the same reason * — most of what is retained is true of the endpoint whoever calls it, and * Codex already sub-keys its transport by account and bearer — so a * credential switch resets the account-dependent subset instead of * splitting the entry (see `resetAccountScopedProviderSessionState`). * * The returned lease MUST be released; until then the entry cannot be * evicted. */ acquire(request: AuthGatewaySessionStateRequest): AuthGatewaySessionStateLease; /** Close and drop every retained state. Called when the gateway shuts down. */ close(): void; }