# Limitations

This document states what AgentInspect **does not** provide today. It complements [KNOWN-ISSUES.md](./KNOWN-ISSUES.md).

## Product boundaries

- **No SaaS dashboard** or hosted multi-tenant product in the open-source core workflow.
- **No production APM replacement**: no sampling agents, no fleet-wide aggregation, no uptime SLAs.
- **No vendor upload pipeline**: no built-in Langfuse/Braintrust/New Relic/Datadog direct exporters as live sinks.
- **No automatic universal instrumentation** of every framework: integration is explicit (manual traces, log ingest, optional adapters).
- **No retry execution engine:** TraceContract `retry` rules (including additive `retry.operations[]` recovery oracles in 6.27) evaluate attempt identity and evidence only. AgentInspect does not retry tools, remediates nothing, and does not treat a client `idempotencyKey` as proof of exactly-once writes. Write-tool timeout/unknown/running completion is unevaluable for `retry.operations` write recovery (client keys and producer no-side-effect flags do not clear that finding); write-retry is not safe by default.
- **Transport facts are source-boundary only:** HTTP status, server-advertised retry delay, selected delay, and their sources must be captured at the transport/fetch boundary. An MCP wrapper cannot recover facts already collapsed into an SDK error message.
- **Run comparability is optional guidance:** Ordinary trajectory checks do not fail when cohort/prompt/tool commitments are absent. See [RUN-COMPARABILITY.md](./RUN-COMPARABILITY.md). A machine-readable comparable-run profile is not a default gate in this line.
- **Omitted-payload digests** (`createOmittedPayloadCommitment`) are bounded (1 MiB preflight). Digests prove omitted bytes existed; they are not redaction or authorization.

## Correlation metadata (v1.3.0)

- **`list` / `view` / `export`** do not filter by correlation fields yet — `stats --correlation-id` / `--group-id` and `search` provide targeted read paths; full CLI filtering remains incremental.

## Persisted event model

- **Manual global tracing remains v0.1.** `inspectRun()` / `step()` still write `schemaVersion: "0.1"` JSONL for compatibility.
- **Persisted writer/runtime output targets schema 1.0.** `createInspector()` with built-in writers emits schema 1.0 persisted rows; v0.1 and v0.2 remain readable.
- **Migration is explicit, not automatic.** `agent-inspect migrate <input> --to 1.0 --dry-run` reports what would change, and `--output <file>` writes a separate file. AgentInspect does not rewrite old traces in place.

## Runtime writers and universal readers (v1.6)

- **Subpath APIs:** `agent-inspect/writers`, `agent-inspect/readers`, and advanced helpers are available for local adoption from their owning subpaths. `createInspector()` is part of the small root API.
- **Explicit writer ownership:** `createInspector()` does not print terminal lifecycle output or implicitly choose a disk writer. Use `fileWriter()` / `bufferedFileWriter()` / custom writers when persistence is desired.
- **Diagnostic evidence, not an event-sourced runtime:** AgentInspect traces are local diagnostic evidence of what an agent did. They are **not** a write-ahead log, durable-before-effect journal, or application event-sourcing runtime. Buffered writers may lose a bounded tail of events on abrupt process termination. Call `flush()` / `close()` according to writer docs when you need the queue drained before exit; that lifecycle is deterministic and idempotent, but it is **not** an fsync / crash-durability guarantee.
- **No standards upload:** OpenInference and OTLP JSON support is local read/export compatibility only. There is no OTLP gRPC/HTTP streaming sink, collector client, or hosted ingestion behavior. OTLP GenAI export is metadata/token-oriented (`OTEL_GEN_AI_SEMCONV_PIN`); it does not emit `gen_ai.prompt` / `gen_ai.completion` bodies.
- **Conservative detection:** `agent-inspect open` does not silently accept arbitrary JSON. Unsupported or ambiguous inputs produce errors/warnings rather than guessed traces.
- **Large inputs:** reader inputs are bounded and read into local memory. This is not a database index or production log warehouse.

## Framework adapters (v1.7)

- **AI SDK integration is explicit telemetry wiring.** Use `@agent-inspect/ai-sdk` through AI SDK `experimental_telemetry.integrations`; AgentInspect does not wrap providers, patch fetch, or enable telemetry globally.
- **AI SDK privacy settings are caller-owned.** Examples set `recordInputs: false` and `recordOutputs: false`; leaving those enabled in user code can cause the AI SDK telemetry layer to include richer data before AgentInspect receives events.
- **OpenAI Agents JS support is Supported (metadata-first).** `@agent-inspect/openai-agents` maps runtime spans through the safe `setTraceProcessors()` boundary and does not capture raw payloads by default. Bounded `capture: "preview"` is available through the shared adapter capture contract.
- **LangGraph support is a boundary decision, not a separate package.** Initial support is expected through `@agent-inspect/langchain` callbacks unless no-network fixtures prove a separate package is needed.
- **No root/core adapter dependencies.** AI SDK, OpenAI Agents, LangGraph, OpenTelemetry, and LangChain remain outside the root/core runtime dependency graph.
- **Preview capture is bounded, not sanitized.** `capture: "preview"` persists truncated, key-redacted previews of framework-provided input/output fields. Key-based redaction cannot detect a secret embedded in free text, and there is no full-content capture mode. Fields the framework never exposes are reported as `AI_CAPTURE_FIELD_UNAVAILABLE` rather than reconstructed.

## LangChain streaming (v1.3.0)

- **Metadata-focused only** — `stream: true` records chunk counts, timing, and optional bounded previews; it is **not** a replay/cassette system.
- **No full token stream storage by default** — even with `stream: true`, `capture: "metadata-only"` does not persist raw streamed text.
- **No per-token JSONL events** — streaming does not emit one trace line per token.

## Data fidelity

- **No full prompt/output capture by default** for manual traces (by design: safety and PII risk).
- **Log-derived runs** and **manual JSONL traces** may differ in fidelity (timestamps, nesting, confidence).
- **Confidence labels** qualify inferred relationships; they do not guarantee correctness.

## Trace safety bounds

- **Redaction profiles** (`local`, `share`, `strict`) are key-based presets — not compliance-grade PII detection. Review exports before sharing even with `--redaction-profile strict`.
- **`@agent-inspect/redact` and `agent-inspect redact` create redacted copies.** They do not encrypt source traces, mutate originals, certify compliance, or guarantee every sensitive value is detected.
- **Default metadata redaction** covers common sensitive keys (exact key match, case-insensitive). Custom secret field names are not redacted unless you add rules via `redact: { rules: [...] }`.
- **Error-message free text** also runs high-confidence credential detectors before disk (provider API keys, bearer tokens, JWTs, GitHub tokens, PEM private keys, and high-confidence `key=value` secrets). This is still not compliance-grade detection; `redact: false` opts out.
- **Metadata truncation** applies to string values and nested structures; very large metadata may be replaced with a truncation marker when `maxEventBytes` is exceeded (default 64 KiB per JSONL line).
- **Redaction is not encryption.** Local trace files remain readable on disk; treat `.agent-inspect-runs/` like any developer artifact that may contain operational data.

## Guardrails and circuits (v2.5 planning)

- **Guardrails and circuits are deterministic local utilities, not compliance certification.** `@agent-inspect/guardrails` and `@agent-inspect/circuit` (planned v2.5) evaluate text, JSON, and trace patterns with bounded evidence; they do not prove safety for every sharing context or replace human review.
- **No LLM judge or remote policy engine.** Built-in rules use pattern lists, redact findings, and structural heuristics only.
- **No automatic remediation by default.** Results are advisory unless caller code maps failures to errors, trace events, or enforcement.

## Checks, artifacts, and test reporters

- **Checks are deterministic local rules, not compliance certification.** `check`, `scan`, and `verify-safe` surface bounded findings and diagnostics over supported local inputs; they do not prove a trace is safe for every sharing context.
- **Eval is deterministic local heuristics.** `@agent-inspect/eval` and `agent-inspect eval` do not provide LLM-as-judge scoring, hosted datasets, replay, semantic grading, or production quality analytics.
- **Safe CI artifacts are structural summaries.** They avoid raw prompt/output/request/response/header/tool payload content by default, but teams should still review generated files before sharing.
- **Vitest/Jest reporters are optional package surfaces.** Recipes document config patterns and explicit associations; package publication is controlled by release readiness and maintainer authorization.

## Execution semantics

- **No replay / fork** of past runs from traces alone.
- **No time-travel debugging** across arbitrary runtime state.
- **No multi-run statistical evaluation** built into core. v2.1 eval checks are per selected trace/run unless user code aggregates results.

## Economics

- **No cost engine**: no pricing tables, invoice-grade usage, or provider billing reconciliation. Usage fidelity may preserve cache-read (`cached`), cache-write (`cacheWrite`), and reasoning token counts when providers report them; those fields are observational only.
- **Token usage is supplied metadata only**: AgentInspect may display `input`, `output`, `total`, and `cached` counts when callers/adapters provide them; core does not count tokens or infer provider billing.

## Local observability commands (v1.4.0)

- **`timeline`**, **`stats`**, and **`search`** scan local JSONL files — no database index; large directories may be slow.
- **`search`** is deterministic exact/contains matching only — no semantic or fuzzy search.
- **`stats`** is local file aggregation — not production fleet analytics.

Visual demos: [SCREENSHOTS.md](./SCREENSHOTS.md) · [CLI.md](./CLI.md)

## Scale

- Designed for **developer machines** and **inner-loop debugging**, not petabyte log warehouses.

For roadmap intent, see [ROADMAP.md](../ROADMAP.md) (direction only — not a delivery guarantee).