---
title: "OpenTelemetry"
description: "Configure OpenTelemetry destinations, content capture, and third-party exports."
url: "/observability/otel"
---

`eve dev` records local traces by default. Use `otel()` for process-wide
settings and `otelIntegration()` for third-party destinations.

## Configure OpenTelemetry

`otel()` and `otelIntegration()` are optional:

- Use `otel()` only when you need process-wide settings such as the resource,
  sampler, propagators, or trace capture policy. Declare it at most once.
- Use `otelIntegration()` only when adding a third-party OpenTelemetry
  destination. Declare one file per exporter or processor chain.

Add `agent/instrumentation/otel.ts` when you need process-wide settings or
want to control the content eve writes to OpenTelemetry spans:

```ts title="agent/instrumentation/otel.ts"
import { otel } from "eve/instrumentation/otel";

export default otel({
  resource: { "deployment.environment": process.env.VERCEL_ENV ?? "development" },
  tracePolicy: ({ audience, environment }) => ({
    emit: true,
    recordInputs: audience === "public" || environment === "development",
    recordOutputs: audience === "public" || environment === "development",
  }),
});
```

The OpenTelemetry `tracePolicy` is a capture ceiling shared by every OTel
destination. A destination cannot restore content excluded by this policy. It
does not affect lifecycle instrumentation created with `defineInstrumentation()`.
See [Audience](../channels/overview#audience) for how channels classify the
conversation passed to this policy.

## Instrument Node.js dependencies

Pass OpenTelemetry instrumentation instances to `otel({ instrumentations })`.
For example, install the PostgreSQL instrumentation alongside `pg`:

```bash
pnpm add pg @opentelemetry/instrumentation-pg
```

```ts title="agent/instrumentation/otel.ts"
import { PgInstrumentation } from "@opentelemetry/instrumentation-pg";
import { otel } from "eve/instrumentation/otel";

export default otel({
  instrumentations: [new PgInstrumentation()],
});
```

Keep the instrumented package external so Node.js module hooks can intercept
it. Add `pg` to your agent's [build configuration](/docs/agent-config#other-defineagent-fields):

```ts title="agent/agent.ts"
import { defineAgent } from "eve";

export default defineAgent({
  model: "openai/gpt-5.4-mini",
  build: { externalDependencies: ["pg"] },
});
```

## Add an OpenTelemetry destination

Use this section when sending traces to a third-party destination. Install
`@vercel/otel` before using its exporters:

```bash
pnpm add @vercel/otel
```

```ts title="agent/instrumentation/honeycomb.ts"
import { OTLPHttpProtoTraceExporter } from "@vercel/otel";
import { otelIntegration } from "eve/instrumentation/otel";

export default otelIntegration({
  exportPolicy: {
    span: () => ({ redact: true, inputs: true, outputs: true }),
  },
  traceExporter: new OTLPHttpProtoTraceExporter({
    url: "https://api.honeycomb.io/v1/traces",
    headers: {
      "x-honeycomb-team": process.env.HONEYCOMB_API_KEY!,
    },
  }),
});
```

Pass `spanProcessors` to `otelIntegration()` when a custom destination needs
filtering or transformation. The example's `exportPolicy` preserves span
metadata while removing inputs and outputs from this destination.

## Add runtime context

Use `runtimeContext` to add JSON values to AI SDK spans for each model attempt.
For typed channel metadata, import the channel definition and narrow with
`isChannel`:

```ts title="agent/instrumentation/support.ts"
import { isChannel } from "eve/instrumentation";
import { otelIntegration } from "eve/instrumentation/otel";

import supportChannel from "../channels/support";

export default otelIntegration({
  runtimeContext(input) {
    if (!isChannel(input.channel, supportChannel)) return undefined;

    return {
      "support.channel_id": input.channel.metadata.channelId ?? "",
      "support.user_id": input.channel.metadata.triggeringUserId ?? "",
    };
  },
});
```

The resolver receives the channel, session, final model input, step, and turn.
Keys beginning with `eve.` are reserved. Content excluded by
`otel({ tracePolicy })` is not available to the resolver.

Authored values appear under `ai.settings.context.*`. Framework channel, session,
step, turn, and version identifiers use structural span attributes instead of
runtime-context copies. `ai.settings.context.eve.environment` remains available.

## Trace topology

OpenTelemetry destinations receive the following trace topology for an ordinary
agent turn:

```text
invoke_agent <agent>
  └── agent.step
        ├── chat <model>
        └── execute_tool <tool>
```

Each root agent turn starts a new trace. If eve receives a local caller's trace
context, the first local subagent turn uses that trace. Its span is a
child of the dispatch span and has a different span ID. Without caller trace
context, the first subagent turn starts a new trace. Later turns in the child
session start new traces. They do not use the original dispatch span as their
parent.

Use `gen_ai.tool.call.id` to correlate tool and approval spans. Tool failures use
`error.type`. These attributes replace the duplicate `agent.action.call_id` and
`agent.action.error.code` fields without changing trace schema version 4.

Model and activation spans record token counts under `gen_ai.usage.*`, without
duplicate `agent.usage.*` token fields. Step and tool spans retain `agent.usage.*`.
Cost values remain unchanged.

A remote child starts a separate trace with a root `invoke_agent` span. Its first
turn links to the caller with `eve.link.type=agent.dispatch`. The receiving
deployment applies its sampler. Queries for local dispatches must use
`parentSpanId` or `agent.parent_run.id` and `agent.parent_call.id` instead of a
dispatch link.

Use `gen_ai.conversation.id` to find the root session and its child sessions
across traces. eve adds this ID to spans created during model, tool, and memory
execution. This includes spans from third-party instrumentation. `agent.run.id`
identifies one agent session, not the full conversation.

The shared conversation ID does not give access to message history, agent state,
or credentials. It does not give authorization or require the same worker for
each turn. Remote requests carry the conversation ID in W3C Baggage.
`trustedForwarders` controls whether the receiver accepts parent session data
and the trace-content policy. See
[Remote agents](../guides/remote-agents#preserving-trace-content).

## Local traces

`eve dev` records local traces by default. Omitting
`agent/instrumentation/local.ts` preserves that default. Export
`localTraces(...)` from that file to reconfigure it, or disable it explicitly:

```ts title="agent/instrumentation/local.ts"
import { disableInstrumentation } from "eve/instrumentation";

export default disableInstrumentation();
```

## Enable tracing on Vercel

For Agent Runs sampling defaults and existing-deployment setup, see
[Enable tracing on Vercel](./agent-runs#enable-tracing-on-vercel).

## Filter a destination

`otelIntegration()` and `localTraces()` accept one
`exportPolicy` object or an array applied in order. Each policy filters spans
and attributes before that destination's processors receive them:

```ts title="agent/instrumentation/local.ts"
import { localTraces } from "eve/instrumentation/otel";

export default localTraces({
  exportPolicy: {
    span: ({ name }) => ({ emit: name !== "internal.cache.refresh" }),
    attribute: ({ key }) => (key === "customer.id" ? { emit: false } : { emit: true }),
  },
});
```

Return `{ emit: true }` from `span` to retain a span unchanged, or
`{ emit: false }` to omit it from this destination. To retain the span while
redacting content, return `{ redact: true }` with `inputs: true`,
`outputs: true`, or both. A redaction decision implies emission and requires
at least one direction. For example, return
`{ redact: true, inputs: true, outputs: true }` to redact both directions.
A throwing `span` callback drops the span from this destination. Return
`{ emit: true }`, `{ emit: false }`, or
`{ replace: true, value }` from `attribute` to retain, remove, or change
one attribute.

## What to read next

- [Instrumentation](/docs/observability/instrumentation): handle eve lifecycle events.
- [Agent Runs](./agent-runs): configure Vercel's built-in trace destination.
- [Migrate instrumentation](/docs/observability/instrumentation-migration): replace `agent/instrumentation.ts`.
- [Local development](../guides/dev-tui): inspect local traces in the TUI.