/** * FoundryLocalProvider — on-device models over Foundry Local's * OpenAI-compatible `/v1/chat/completions`. * * Pattern: Adapter (GoF) + Ports-and-Adapters (Cockburn 2005). * Role: Outer ring — translates `LLMRequest`/`LLMResponse` to/from the * wire Foundry Local serves on localhost. Knows nothing about * agents, recorders, or compositions. * Emits: N/A. * * ─── Why this exists ───────────────────────────────────────────────── * * The adapter ladder is `mock()` → a local model → a paid API, and * `ollama()` is the proof that the middle rung is worth owning: zero * dependencies, honest refusals, real token counts. Foundry Local is * Microsoft's runtime for the same rung — ONNX under the hood, models * pulled with `foundry model run `, no key, no account — and a * Windows or macOS machine that has it installed deserves the same * one-import experience. So this file owns that wire the way * `OllamaProvider.ts` owns Ollama's: * * • ZERO dependencies — one `fetch` POST and SSE. The official * `foundry-local-sdk` is NOT imported; its manager is accepted * duck-typed (see {@link FoundryLocalProviderOptions.manager}) so a * consumer who already uses it can hand over the discovered URL * without this package gaining a dependency. * • HONEST REFUSALS — a typed {@link FoundryLocalUnavailableError} * that names the endpoint it tried and the command to run. The * service's port is DYNAMIC per start, which makes "nothing is * answering" the most likely first failure — so that message * carries the discovery command, not just the start command. A * failure the service reports IN BAND — an `error` frame on an * already-200 stream, the out-of-memory a laptop runtime really does * hit — is RAISED the same way, never handed over as a shorter * answer that reads like a clean stop. * • REAL TOKEN COUNTS while streaming — `stream_options: * { include_usage: true }` is always sent. This is OUR wire, a * documented Foundry Local surface, not an arbitrary * OpenAI-compatible server — so the caution that made * `openai({ baseURL })` withhold the field (and silently zero every * local token count until 9.73.0) does not apply here. * * ─── Wire realities this file owns ─────────────────────────────────── * * • THE PORT IS DYNAMIC. Every `foundry server start` may pick a new * port; the docs' own REST example shows `http://localhost:5272` and * that is the default here, but the truthful discovery is * `foundry server status` (or the SDK manager's `.urls`). Note the * CLI group was RENAMED from `foundry service` to `foundry server` — * every message in this file uses the NEW spelling. * • ALIASES vs VARIANT IDS. The catalog speaks in aliases * (`qwen2.5-0.5b`) that fan out to hardware variants * (`qwen2.5-0.5b-instruct-generic-cpu:1`), but REST chat calls take * the FULL variant id. This adapter resolves an alias through * `GET /foundry/list` — first matching variant wins, because the * list's order IS the service's priority order — and caches the * answer per provider instance, HIT OR MISS: exactly one catalog * attempt per name, so an alias the catalog never answers for cannot * re-ask before every call. A fresh provider is the retry. A name that * already carries a variant's execution-provider suffix * (`-cpu`/`-gpu`/`-npu`, optional `:version`) is used as-is with no * catalog round-trip. * • NO API KEY EXISTS. The docs' own samples pass placeholders. This * adapter sends no `Authorization` header at all — there is nothing * to put in one, and an invented value would only end up in somebody's * proxy log. * * ─── Ceilings (stated, not worked around) ──────────────────────────── * * • NO FORCED TOOL CHOICE. `tool_choice` support is UNDOCUMENTED on * this wire, so `carriesForcedToolChoice` is `false` and an agent * using `.outputSchema(parser, { strategy: 'tool-forced' })` refuses * at run start, naming this provider. Claiming an undocumented field * works would turn a guarantee into a suggestion. * • TOOL CALLING IS MODEL-DEPENDENT. `/foundry/list` reports * `supportsToolCalling` per variant, but this adapter does not * preflight-refuse on it — a wrong refusal is worse than a weak * answer, the same stance `ollama()` takes on `/api/show`. Pick a * tool-capable variant. * • NO MULTI-MODAL. `LLMMessage.content` is a string. Same ceiling as * every other adapter here. * • NO PROMPT CACHING — resolves to the NoOp cache strategy. * • NO STRUCTURED THINKING. The wire has no thinking field; a reasoning * model's `` tags ride the answer text untouched. */ import type { LLMCallHooks, LLMChunk, LLMProvider, LLMRequest, LLMResponse, WireRole } from '../types.js'; /** * The two failures an on-device runtime actually has, told in words that * contain the fix. * * Both are things the person at the keyboard can resolve in one command, * which is exactly why they get a type instead of a wrapped * `ECONNREFUSED` or a bare `404`. `reason` is the discriminator; the * message already reads as instructions — and because Foundry Local's * port changes per start, the unreachable message teaches the discovery * command (`foundry server status`) alongside the start command. */ export declare class FoundryLocalUnavailableError extends Error { readonly name = "FoundryLocalUnavailableError"; /** Which of the two situations this is. */ readonly reason: 'service-unreachable' | 'model-not-available'; /** The endpoint that was tried — the thing to check or change. */ readonly endpoint: string; /** The model asked for. Absent when the service never answered at all. */ readonly model?: string; /** Models this machine DOES have cached, when the service could tell us. */ readonly availableModels?: readonly string[]; constructor(init: { reason: 'service-unreachable' | 'model-not-available'; endpoint: string; model?: string; availableModels?: readonly string[]; /** * The 404 body did not speak this dialect, so "the model is missing" is * a guess and the endpoint deserves naming too. Shapes the MESSAGE only; * `reason` stays the discriminator consumers branch on. */ routeUnconfirmed?: boolean; cause?: unknown; }); } export interface FoundryLocalProviderOptions { /** * Where the Foundry Local service is listening — the ROOT url; this * adapter appends `/v1/chat/completions`, `/foundry/list` and * `/openai/models` itself. A URL ending in `/v1` is accepted and * trimmed (the same courtesy `ollama()` extends to its 8.0.0 configs), * and a bare `host:port` gets `http://`. * * Defaults to `FOUNDRY_LOCAL_ENDPOINT` when set, then * `FOUNDRY_LOCAL_BASE_URL` — the second spelling is honored because * our own demo taught it, and a config written for that demo should * keep working — otherwise `http://localhost:5272`. Know that the * default is only the docs' example port: Foundry Local picks a NEW * port on every `foundry server start` unless one was pinned with * `--port`, and `foundry server status` prints the live URL. * * A BLANK value anywhere in that chain — `ENV FOUNDRY_LOCAL_ENDPOINT=` * in a Dockerfile, an empty compose value, `manager.urls = ['']` — * counts as unset, not as the URL `http:`. The next candidate gets its * turn. */ readonly endpoint?: string; /** * A `foundry-local-sdk` `FoundryLocalManager`, duck-typed — this * package never imports the SDK. When given, `manager.urls[0]` (the * manager's discovered service URL) wins over the env vars and the * default, so a consumer already using the SDK for model management * gets the REAL dynamic port for free. An explicit `endpoint` still * beats it — the most specific word wins. */ readonly manager?: { readonly urls?: readonly string[]; }; /** * Model used when `LLMRequest.model` is the `'foundry-local'` * shorthand. Prefer the positional form: `foundryLocal('qwen2.5-0.5b')`. */ readonly defaultModel?: string; /** Default token cap when the request doesn't set one. Maps to `max_tokens`. */ readonly defaultMaxTokens?: number; /** * How long to wait for the service to ANSWER, in ms. Default 10000. * * This bounds the wait for response headers, NOT generation: a laptop * model may take minutes to finish a long answer and that is fine. * What it prevents is the failure this whole file exists to avoid — a * run that hangs because nothing is listening. Stopping a call that is * already streaming is the caller's `AbortSignal`'s job, not this * one's — and that signal is honored for the WHOLE call, body included. */ readonly timeoutMs?: number; /** @internal Custom fetch implementation for tests. */ readonly _fetch?: typeof fetch; } /** * Build an `LLMProvider` backed by an on-device Foundry Local service. * * Free, offline, no API key. The rung between `mock()` and a paid API on * a machine where Foundry Local is the local runtime — and the agent * code above it does not change between the three. * * @example * import { Agent } from 'agentfootprint'; * import { foundryLocal } from 'agentfootprint/providers'; * * const agent = Agent.create({ * provider: foundryLocal('qwen2.5-0.5b'), * model: 'qwen2.5-0.5b', * }).build(); * * @example // the service on its real (dynamic) port, via the SDK's manager * foundryLocal('phi-3.5-mini', { manager }); * * @example // a pinned port on another machine * foundryLocal('qwen2.5-0.5b', { endpoint: 'http://192.168.1.20:5272' }); */ export declare function foundryLocal(model: string, options?: FoundryLocalProviderOptions): LLMProvider; /** * Object form, matching the shape every sibling factory accepts. * `defaultModel` names the model; everything else keeps its meaning. */ export declare function foundryLocal(options?: FoundryLocalProviderOptions): LLMProvider; /** * Class form for consumers who prefer `new FoundryLocalProvider(...)`. */ export declare class FoundryLocalProvider implements LLMProvider { readonly name = "foundry-local"; readonly carriesInMessages: readonly WireRole[]; readonly carriesForcedToolChoice = false; private readonly inner; constructor(model?: string | FoundryLocalProviderOptions, options?: FoundryLocalProviderOptions); complete(req: LLMRequest, hooks?: LLMCallHooks): Promise; stream(req: LLMRequest, hooks?: LLMCallHooks): AsyncIterable; } //# sourceMappingURL=FoundryLocalProvider.d.ts.map