/** * Vision / media agent-tool family — `vision_analyze` / `image_generate` / * `text_to_speech` (T11951 · M7 · epic T11456 · SG-TOOLS). * * The FIRST occupants of the previously-empty `media` toolset. Every model / * multimodal call routes THROUGH the single E9 chokepoint * ({@link import('../llm/system-resolver.js')}'s `resolveLLMForSystem` → * {@link import('../llm/model-runner.js')}'s `ModelRunner`), EXACTLY like * `browser_vision` ({@link ./web-agent-tools.js}). There is NO raw provider client, * NO new transport, NO API-key read at this call-site — the plaintext credential is * materialized from the sealed handle ONLY at the wire inside `ModelRunner.build` * (Gate-13 stays green). * * - **`vision_analyze`** — read a local image + a prompt, send them as a single * multimodal user turn through the chokepoint, return the model's analysis. * - **`image_generate`** — send an image prompt through the chokepoint; returns * the model's response (text/asset-ref it emits). When the resolved model * cannot generate an image the chokepoint degrades — the tool surfaces an * `unsupported` flag rather than constructing a bespoke image client. * - **`text_to_speech`** — send text through the chokepoint for narration intent; * same degradation contract when the modality is unsupported. * * ## Availability (AC3 — mirrors `browser_vision`) * * Hidden unless outbound egress is permitted AND the host advertises a multimodal * model (`networkEgressAllowed !== false && capabilities.multimodal === true`). * Registered-but-hidden so core runs credential-OFF / egress-OFF and the catalog * stays stable. (The capability flag is the host's "a multimodal model is * resolvable" promise — availability is a SYNC predicate, so the actual resolution * happens at `execute` time with graceful degradation when no credential resolves.) * * @epic T11456 * @task T11951 * @see ./web-agent-tools.js — `browser_vision`, the E9-routed multimodal pattern mirrored here * @see ../llm/system-resolver.js — `resolveLLMForSystem` (the E9 chokepoint) * @see ../llm/model-runner.js — `ModelRunner` (the single SSoT wire builder) */ import type { AgentToolRegistry, AvailabilityCheck } from './agent-registry.js'; /** * Available only when outbound egress is permitted AND the host advertises a * multimodal model. Mirrors `browser_vision`'s availability: registered-but-hidden * (so the `media` catalog is stable) until a context advertises both * `networkEgressAllowed !== false` and `capabilities.multimodal === true`. */ export declare const multimodalAvailable: AvailabilityCheck; /** * Read a local image file as base64 + sniff its MIME type from the extension. * Injectable so the unit test supplies image bytes without a real file on disk. */ export type ImageReader = (path: string) => Promise<{ base64: string; mediaType: string; }>; /** The outcome of a chokepoint-routed media call. */ export interface MediaModelResult { /** Whether a model produced a response. */ readonly ok: boolean; /** The model's textual response (analysis / description / refusal). */ readonly content?: string; /** The model that answered (present once a credential resolves). */ readonly model?: string; /** * `true` when no credential resolved through the E9 chokepoint — the call * degraded gracefully rather than throwing (mirrors `browser_vision`). */ readonly aiUnavailable?: boolean; /** * `true` when the resolved model cannot serve the requested modality * (e.g. true image synthesis / TTS audio). The tool does NOT construct a * bespoke client for an unsupported modality (Gate-13) — it reports this. */ readonly unsupported?: boolean; /** A stable code + message when the call failed for another reason. */ readonly error?: { readonly code: string; readonly message: string; }; } /** Options for {@link registerMediaAgentTools} — all injectable for testing. */ export interface MediaAgentToolOptions { /** The local-image reader. Defaults to a real `fs` read + MIME sniff. */ readonly readImage?: ImageReader; /** Project root threaded into the E9 resolver (defaults to the resolver's own fallback). */ readonly projectRoot?: string; } /** * Register the vision / media agent-tool family into `registry`. Pure * registration — no model is resolved, no credential is read, no file is opened * here; all of that happens later inside each tool's `execute` through the E9 * chokepoint (and the injected image reader). Import-time side-effect-free. * * @param registry - The registry to populate. * @param options - Injectable image reader / project root (for testing). */ export declare function registerMediaAgentTools(registry: AgentToolRegistry, options?: MediaAgentToolOptions): void; /** * Self-registration marker (AC1) — the identifier the * {@link AgentToolRegistry.discover} bounded source scan greps for. Aliases * {@link registerMediaAgentTools} so a future scan-dir discovery (or the built-in * aggregator) can call it uniformly with the other agent-tool modules. * * @param registry - The registry to populate. */ export declare function registerAgentTools(registry: AgentToolRegistry): void; //# sourceMappingURL=media-agent-tools.d.ts.map