# Tool conformance

## What it does

Tool conformance helpers are dependency-free assertions for tool-dispatch configuration tests. They exercise the blocked-reason matrix and the success path of `dispatchToolCall` without network or credentials, and the tool-disclosure contract (progressive tool loading, plan 041) without any provider call.

Exported from `@arnilo/prism/testing/tool-conformance`:

- `assertToolDispatchConforms(registry, options)`
- `assertToolDisclosureConforms(options)`
- `assertToolBlocked(probe, expectedReason)`
- `dispatchAndCollect(probe)`
- `ToolConformanceOptions`, `ToolDispatchProbeOptions`, `ToolDisclosureConformanceOptions`

## When to use it

Use this helper when configuring a `ToolRegistry` with allow/deny filters, permission policies, and validators. It asserts the canonical blocked reasons and that blocked calls never execute:

- unknown tool → `tool_execution_blocked` with reason `unknown_tool`
- denied tool (filter) → `tool_denied`
- non-object arguments → `invalid_arguments`
- permission denial → `permission_denied`
- validator failure → `validation_failed`
- a valid call emits `tool_execution_started` and returns a result with no error

## Inputs / request

```ts
import { assertToolDispatchConforms } from "@arnilo/prism/testing/tool-conformance";
import { createToolRegistry } from "@arnilo/prism";

await assertToolDispatchConforms(createToolRegistry(), {
  tool: { name: "echo", execute: (args, ctx) => ({ toolCallId: ctx.toolCallId, name: "echo", value: args }) },
  validArgs: { msg: "hi" },
  permission: myPermissionPolicy,
});
```

`ToolConformanceOptions`:
- `tool: ToolDefinition` — registered as the success-path target
- `validArgs: JsonObject` — arguments for the success probe
- `permission?: PermissionPolicy` — applied to every probe (default allow-all)
- `validate?: ToolValidator`, `filter?: ToolFilterInput`, `secrets?: readonly (string | undefined)[]`

## Outputs / response / events

`assertToolDispatchConforms` returns `Promise<void>` and throws on the first violation. `dispatchAndCollect` returns `{ result, events }` capturing the emitted `AgentEvent`s for custom assertions.

`assertToolDispatchConforms` returns `Promise<void>` and throws on the first violation. `dispatchAndCollect` returns `{ result, events }` capturing the emitted `AgentEvent`s for custom assertions.

### Disclosure leg (toolsDisclosure "search")

`assertToolDisclosureConforms(options)` asserts the progressive tool-loading contract against the same narrowing the runtime applies (`filterTools` allow/deny bounds, then `selectDisclosedTools`):

- search mode only narrows: the disclosed set is a subset of the allow/deny-filtered input, never wider, never zero, deterministic order for identical turns; a deny-listed tool is never described to the provider
- the generated `search_tools` tool is always kept in the disclosed set
- fail closed: an index over the frozen 1024-tool cap discloses the full eligible list
- `search_tools` output is inert: names plus byte-truncated (512-char) descriptions only — no JSON structure, no tool schemas — oversized hosts descriptions are truncated, never executed, and configured `secrets` never surface even when a description carries one
- activation is bounded to topK and only ever selects from the eligible set; activated tools stay disclosed on the next turn

```ts
import { assertToolDisclosureConforms } from "@arnilo/prism/testing/tool-conformance";

assertToolDisclosureConforms({
  tools: hostTools,                        // incl. schema-bearing and oversized-description tools
  filter: { deny: ["legacy_tool"] },       // host allow/deny bounds
  search: { topK: 16 },
  secrets: [hostSecret],                   // secret-scan of model-facing search output
});
```

## Request/response example

```ts
import { assertToolBlocked } from "@arnilo/prism/testing/tool-conformance";

await assertToolBlocked(
  { call: { type: "tool_call", id: "c", name: "missing", arguments: {} }, registry: createToolRegistry() },
  "unknown_tool",
);
```

## Implementation example

```ts
import { assertToolDispatchConforms } from "@arnilo/prism/testing/tool-conformance";
import { createToolRegistry } from "@arnilo/prism";

await assertToolDispatchConforms(createToolRegistry(), {
  tool: { name: "echo", execute: (args, ctx) => ({ toolCallId: ctx.toolCallId, name: "echo", value: args }) },
  validArgs: {},
});
```

## Extension and configuration notes

- The helper registers `options.tool` into the supplied registry; pass a fresh registry to avoid duplicate-name errors.
- Execution is observed via the `tool_execution_started`/`tool_execution_blocked` events the runtime emits — the helper does not mutate the caller's tool.
- Use `dispatchAndCollect` directly for custom probes (e.g. middleware ordering) beyond the standard matrix.

## Security and performance notes

- No credentials, no network required.
- Supply `validate` to exercise your policy; use `createJsonSchemaToolArgumentValidator()` from `@arnilo/prism-core/validation/json-schema` for standards-based `parameters` validation.
- The helper uses an allow-all permission policy by default; supply `permission` to validate your fail-closed policy.
- Blocked calls are proven not to execute by the absence of `tool_execution_started`.

## Related APIs

- [Tools](tools.md)
- [Tool effects](tool-effects.md)
- [Settings, auth, trust, security](settings-auth-trust-security.md)
- [Provider conformance](provider-conformance.md)