# pi-langfuse

[![npm version](https://img.shields.io/npm/v/pi-langfuse)](https://www.npmjs.com/package/pi-langfuse)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)

[**English**](./README.md) | [**简体中文**](./README_CN.md)

Langfuse observability extension for [Pi Coding Agent](https://github.com/earendil-works/pi-coding-agent). It sends complete Pi runs to [Langfuse](https://langfuse.com) so the prompt, agent workflow, LLM generations, tool calls, final response, usage, cost, and health scores appear in one trace.

## What This Adds to Pi

- One Langfuse trace per user prompt, grouped by Pi session.
- Root `agent`, per-request `generation`, and per-tool `tool` observations.
- Foreground child sessions keep their SDK exporters isolated, so parent telemetry continues after a child exits.
- Final assistant output capture, generation/tool error visibility, and trace-level scores.
- Privacy controls for inputs, outputs, tool I/O, system prompt, and cwd.
- Secret redaction and local path hashing before upload.
- Capability-gated REST fallback for self-hosted Langfuse setups that expose the legacy trace API when OTel spans arrive but traces do not materialize. Langfuse v4 `events_only` deployments use OTel without legacy fallback ingestion.

## Prerequisites

- **Node.js** >= 22
- **Pi Coding Agent** installed and configured
- A **Langfuse** account ([cloud](https://cloud.langfuse.com) or self-hosted)

## Quick Start

1. Install the extension:

   ```bash
   pi install npm:pi-langfuse
   ```

2. Run Pi once. If no credentials are configured yet, Pi prompts for:
   - Langfuse public key, starting with `pk-lf-...`
   - Langfuse secret key, starting with `sk-lf-...`
   - Langfuse host, defaulting to `https://cloud.langfuse.com`
   - Optional Langfuse user ID for per-user usage and cost aggregation

3. Run Pi normally:

   ```bash
   pi "Explain the architecture of Redis"
   ```

4. Open Langfuse and inspect the new trace.

## Configuration

Langfuse API keys are available in **Langfuse Cloud** -> **Settings** -> **API Keys**.

### Method 1: Interactive setup

Run any `pi` command with the extension loaded. On first run without configuration, Pi prompts in the CLI or TUI and saves the result to `~/.pi/agent/pi-langfuse/config.json`.

To run setup again:

```text
/langfuse-setup
```

To inspect the active configuration without exposing secrets:

```text
/langfuse-status
```

The status command reports the config source, host, masked public key, whether a user ID is configured, capture policy, active-run state, config path, and last runtime error. It never prints the user ID itself.

### Method 2: Environment variables

Set these before starting Pi:

```bash
export LANGFUSE_PUBLIC_KEY="pk-lf-xxxx"
export LANGFUSE_SECRET_KEY="sk-lf-xxxx"
export LANGFUSE_BASE_URL="https://cloud.langfuse.com"  # optional; LANGFUSE_HOST is also supported
export LANGFUSE_USER_ID="user-123"                     # optional; max 200 characters
```

Saved config takes precedence. Environment variables are only used when `~/.pi/agent/pi-langfuse/config.json` is missing or incomplete.
If saved credentials exist but `userId` is absent, `LANGFUSE_USER_ID` is still used. An explicitly saved `userId` takes precedence.

For short-lived SDK hosts, set the bounded final score-delivery attempt during shutdown:

```bash
export PI_LANGFUSE_SCORE_SHUTDOWN_TIMEOUT=2  # seconds; defaults to 2 seconds
```

The extension attempts queued trace-level scores before other shutdown telemetry work. This value cannot extend the overall shutdown deadline.

Privacy controls can also be set through environment variables:

```bash
export LANGFUSE_PRIVACY_PRESET="full-debug"
```

Available presets:

| Preset | Captures |
|--------|----------|
| `metadata-only` | Metadata only; omits inputs, outputs, tool I/O, system prompt, and cwd |
| `prompts-only` | Prompt/provider inputs plus metadata |
| `conversations` | Inputs and assistant outputs, but omits tool I/O, system prompt, and cwd |
| `full-debug` | Full trace detail; this is the default |

Fine-grained flags override presets:

```bash
export LANGFUSE_CAPTURE_INPUTS=true
export LANGFUSE_CAPTURE_OUTPUTS=true
export LANGFUSE_CAPTURE_TOOL_IO=false
export LANGFUSE_CAPTURE_SYSTEM_PROMPT=false
export LANGFUSE_CAPTURE_CWD=false
export LANGFUSE_CAPTURE_SOURCE_METADATA=false
export LANGFUSE_CAPTURE_PATHS=false
```

Source metadata remains off in every preset unless `LANGFUSE_CAPTURE_SOURCE_METADATA=true` is set explicitly.
The same holds for absolute paths and `LANGFUSE_CAPTURE_PATHS`.

All captured payloads are redacted before upload. The extension masks common API keys, bearer tokens, passwords, cookies, private keys, Langfuse keys, GitHub/npm/AWS-style tokens, and local absolute paths.

### Absolute paths

By default, local absolute paths (`/Users/...`, `/home/...`, `/tmp/...`, `C:\Users\...`) are
replaced everywhere with a stable `[PATH_HASH:<12 hex chars>]` digest, so usernames and repository
names never reach Langfuse. This applies to inputs, outputs, tool I/O, tool error messages, and the
`cwd` metadata field. Opt in to see real paths in traces:

```bash
export LANGFUSE_CAPTURE_PATHS=true
```

Or persist it in `config.json`:

```json
{ "capture": { "LANGFUSE_CAPTURE_PATHS": "true" } }
```

Like `LANGFUSE_CAPTURE_SOURCE_METADATA`, this stays off in every privacy preset until it is set
explicitly, and it only affects paths — secret masking (tokens, keys, cookies, passwords) is always
on regardless. Note that `LANGFUSE_CAPTURE_CWD=false` is a different control: it drops the `cwd`
metadata field entirely rather than changing how paths are rendered.

`/langfuse-status` reports the current setting under `Capture: absolute paths`.

### Payload limits

Before upload, payloads are shaped: strings are truncated and deeply nested or
very wide structures are trimmed. These caps keep traces small and protect the
Langfuse ingestion pipeline. Override any of them (no rebuild needed):

```bash
export PI_LANGFUSE_MAX_STRING_LENGTH=12000       # per-string chars (system prompt, inputs)
export PI_LANGFUSE_MAX_TOOL_PAYLOAD_LENGTH=24000 # per tool input/output chars
export PI_LANGFUSE_MAX_DEPTH=6                    # max nesting depth
export PI_LANGFUSE_MAX_ARRAY_ITEMS=50            # max array elements kept
export PI_LANGFUSE_MAX_OBJECT_KEYS=80            # max object keys kept
export PI_LANGFUSE_MAX_PAYLOAD_NODES=2000        # max total nodes per payload
```

Set any limit to `0`, `off`, `none`, or `unlimited` to disable that cap
entirely (captures the full value). Unset or invalid values fall back to the
defaults shown above. To capture a very large system prompt or big tool
payloads in full, raise or disable the relevant limit (e.g.
`PI_LANGFUSE_MAX_STRING_LENGTH=off`).

JSON strings above the string limit are replaced with `[payload too large]` rather
than cut into invalid JSON that could bypass sensitive-field redaction. Error
status text uses the same shaping and redaction limits, even when IO capture is disabled.
Status formatting also masks JSON credentials after Pi/provider error prefixes.
If surrounding diagnostics make structured error text unparseable, status and
captured tool-error output use `[unparseable error payload]` instead of the raw text.
The same protection applies to unparseable JSON fragments with sensitive field
names replayed inside later provider requests; ordinary surrounding text is retained
when it contains no sensitive JSON fields.
String-only capture paths (including assistant replies, compaction summaries, and
custom IDs) replace text containing unmasked sensitive JSON fields with
`[REDACTED_SECRET]`. Already-masked fields and ordinary text stay intact; captured
output remains a string.
This check also decodes nested JSON strings within the existing depth/node limits.
Ambiguous escaped text or exhausted inspection limits hide the string rather than expose unchecked content.
Credential assignments are masked with or without quotes, including escaped or
unterminated values. Escaped quotes in serialized text protect the entire value,
including spaces.
Ambiguous single-quote escaping can also mask the remaining diagnostic text.
Known tokens are masked before string truncation so a token
crossing a capture limit cannot escape detection as a shorter fragment.

The REST fallback ingestion is chunked so each request body stays well below
the Langfuse gateway's payload limit (~4.5MB). These knobs control the chunk
budget and a hard ceiling for the whole fallback payload:

```bash
export PI_LANGFUSE_MAX_INGESTION_BATCH_BYTES=4194304  # per-request body budget, default 4MB

export PI_LANGFUSE_MAX_FALLBACK_TOTAL_BYTES=33554432  # whole-payload ceiling, default 32MB
```

When the accumulated fallback payload exceeds the 32MB ceiling, ingestion is
skipped with a warning instead of attempting an unrecoverably large upload.
An individual event above the per-request budget is skipped with a size-only
warning; the remaining events are still sent. Raise capture limits with this
per-event ceiling in mind.
Fallback checks each trace separately within one visibility window and restores
only missing traces, preserving each trace's session, user, name, and metadata.
Visibility polling uses at most three quarters of the remaining shutdown time,
leaving the rest for ingestion even when SDK cleanup was slow.

### Reasoning tokens

Pi reports reasoning (thinking) tokens for Anthropic, OpenAI Codex, OpenRouter,
opencode-go and Qwen. Providers count them inside `output`, and by default the
extension reports `output` whole, so the reasoning share is not visible in
Langfuse. Opt in to report it as its own usage bucket:

```bash
export PI_LANGFUSE_SPLIT_REASONING_TOKENS=true
```

Or persist it in `config.json`:

```json
{ "capture": { "PI_LANGFUSE_SPLIT_REASONING_TOKENS": "true" } }
```

With the split on, a generation that used 37 output tokens of which 10 were
reasoning is reported as `output: 27` plus `output_reasoning_tokens: 10`. Both
keys contain `output`, so the Output row in Langfuse still shows 37; only the
breakdown beneath it gains the reasoning share. Reasoning is clamped to the
reported `output`, so the buckets always add up to the total.

> **Before enabling — check your model prices.** Langfuse matches prices to
> usage by exact key, and model definitions you created in your project take
> precedence over Langfuse's maintained defaults. A custom model priced on
> `input` and `output` only would cost `output_reasoning_tokens` at zero, so
> reasoning-heavy generations would look cheaper than they are. Add a price
> for `output_reasoning_tokens` to every custom reasoning model in
> **Settings → Models** first, then turn the split on. Langfuse's built-in
> prices for reasoning models already include it. Providers that report their
> own cost are unaffected: Langfuse uses the reported cost as-is and does not
> recompute it from usage.

The split is off by default, so upgrading changes nothing until you enable it.
Unset `PI_LANGFUSE_SPLIT_REASONING_TOKENS` (or set it to `false`) to go back;
traces already ingested keep their buckets.

### Method 3: Persistent `config.json`

Create or update `~/.pi/agent/pi-langfuse/config.json`:

```json
{
  "publicKey": "pk-lf-xxxx",
  "secretKey": "sk-lf-xxxx",
  "host": "https://cloud.langfuse.com",
  "userId": "user-123",
  "privacyPreset": "conversations"
}
```

`userId` is optional, explicit-only, and limited to 200 characters. The extension does not infer it from the local operating-system account. When configured, it is propagated to every observation in the trace so Langfuse can provide per-user filtering and cost aggregation.

Fine-grained capture flags can also be persisted:

```json
{
  "publicKey": "pk-lf-xxxx",
  "secretKey": "sk-lf-xxxx",
  "host": "https://cloud.langfuse.com",
  "capture": {
    "LANGFUSE_PRIVACY_PRESET": "metadata-only",
    "LANGFUSE_CAPTURE_INPUTS": "true"
  }
}
```

> **Security**: Keep `~/.pi/agent/pi-langfuse/config.json` private. Never commit API keys to version control.
> When the extension writes this file itself, it creates the config directory with `0700` permissions and the file with `0600` permissions where the host filesystem supports POSIX modes.

## Verify the Extension

Check that Pi has loaded the package:

```bash
pi list
```

`pi-langfuse` should appear in the installed package list.

To verify the Langfuse host and API keys from inside Pi, run:

```text
/langfuse-test
```

This command makes a timeout-bounded authenticated request to Langfuse and, if it succeeds, sends a small test trace.
The result also probes the observations read API. It tries `v2/observations` first and falls back to the legacy
observations API only when v2 returns 404 or 405, so older servers remain supported. Read failures produce a
warning and do not block the test trace. This checks endpoint availability, not whether the new trace is queryable.
Langfuse v4 in `events_only` mode removes legacy reads; the compaction and error-text fixes do not require v4.

## What Appears in Langfuse

- Each Pi session gets its own Langfuse session ID.
- Each user prompt within that session becomes a separate trace.
- The trace contains the final assistant output shown in Pi.
- Tool runs appear as tool observations with arguments, results, and error state.
  Pi 1.0 nested calls (including codemode tools) are parented by `parentToolCallId`.
  A tool's own model usage/cost appears in a child `tool-usage` generation; transcript totals that
  include descendants are not ingested again. Pi's tool usage does not identify each nested model,
  so these aggregate generations have no inferred model name or price.
- LLM requests appear as generation observations, including usage and cost when the provider exposes them.
  Failed assistant messages are marked `ERROR` with a redacted `statusMessage`, including streaming
  failures after HTTP 200 and failures without provider hooks. User-aborted generations are `WARNING`.
  A successful final assistant response clears an earlier transient HTTP error from an internal retry.
  Reasoning tokens are reported as their own usage bucket when `PI_LANGFUSE_SPLIT_REASONING_TOKENS` is enabled.
  Virtual routing preserves `selectedModel`/`selectedProvider` alongside `actualModel`/`actualProvider`;
  the generation model and `provider` metadata use the actual identity when available.
- Pi 0.86 transcript-aware prompt/tool state appears as `system-state` events. Generations reference the
  effective prompt/tool state hashes, active tool count, update transport, and cache hit ratio.
  State is refreshed before every provider request, including after lazy tool loading, and unchanged
  state is deduplicated within an attempt.
- Automatic retries are grouped under `agent-attempt` spans and the trace remains open until `agent_settled`.
- Cache-warming decisions and persisted `cache_warm` usage are reported, including standalone session-grouped
  traces when warming happens while no user-request trace is active.
- Compaction is recorded as `session-compaction`, with a nested `compaction-summary` generation when usage is available.
  Both compaction events are recorded even when no agent run is active — recovery compaction often lands after the
  run settled — and then form their own trace named `pi-session` inside the same session.
- On Pi versions that emit `session_compact_failed`, failed compactions appear as
  `session-compaction-failed` events: `ERROR` for failures and `WARNING` for cancellations.
  Pi reports a cancelling extension as an abort and may omit its original error text.
- Trace-level scores include tool counts, tool success rate, and whether the run had tool errors.

The package also includes a Langfuse CLI skill, so Langfuse data can be queried directly from Pi:

```text
/pi-langfuse-langfuse <your-query>
```

## Source Metadata

Repository source capture is independent of the privacy presets and disabled by default. Enable it only after deciding that commit identity is appropriate for the Langfuse project:

```bash
export LANGFUSE_CAPTURE_SOURCE_METADATA=true
```

For a Git worktree, the extension records only revision state:

```json
{
  "source_type": "git-repo",
  "vcs.ref.head.revision": "0123456789abcdef...",
  "git_detached": "false",
  "git_dirty": "false",
  "metadata_source": "git-detection"
}
```

The revision is the full `HEAD` commit. Dirty state includes tracked changes and untracked files, but never their paths or contents. Detached state is reported without a branch or tag name. Git remotes, URLs, credentials, usernames, branches, absolute paths, and repository names are never inspected or uploaded by this collector.

When capture is off, the collector does not invoke Git and reports `source_type: "disabled"`. Non-Git directories report `non-git`; a missing or unusable Git executable and incomplete Git state report `unavailable`.

Use native Langfuse and OpenTelemetry settings for deployment identity and explicit operator-owned overrides instead of repository files:

```bash
export LANGFUSE_RELEASE="1.2.3"
export LANGFUSE_TRACING_ENVIRONMENT="production"
export OTEL_SERVICE_NAME="pi-agent"
export OTEL_RESOURCE_ATTRIBUTES="service.version=1.2.3,vcs.repository.name=public-repo"
```

`LANGFUSE_RELEASE` and `LANGFUSE_TRACING_ENVIRONMENT` keep their Langfuse semantics. `OTEL_SERVICE_NAME` and `OTEL_RESOURCE_ATTRIBUTES` are loaded as process-scoped OpenTelemetry resource attributes; values such as `vcs.repository.name` apply to every session sharing the runtime, may identify private source, and require a runtime restart to change.

### Compatibility and rollback

Earlier versions emitted `git_commit`, branch, remote, owner, repository, and repo-local `.pi-langfuse.metadata.json` values by default. New traces use `vcs.ref.head.revision` and stop reading that file. Update dashboards before enabling source capture; historical traces are unchanged.

Unset `LANGFUSE_CAPTURE_SOURCE_METADATA` to stop collection immediately. Pin `pi-langfuse@1.5.12` only if the old schema is required during migration; doing so also restores its broader default source disclosure.

## Session Lineage

Pi writes the session file a session was forked or spawned from into its header, and the extension reads it through `sessionManager.getHeader()`. Every observation of such a derived session then carries:

```json
{
  "parentSessionId": "01a1639b-58c5-729a-8315-f17b045ccd28",
  "sessionKind": "fork"
}
```

`sessionKind` is `subagent` when `pi-subagents` sets `PI_SUBAGENT_CHILD=1` and `fork` otherwise. That environment flag is a `pi-subagents` implementation detail, not a contract this package owns. Foreground children may omit it and receive the `fork` label; the parent link still comes from the session header. Normal parent UUIDs remain exact, while custom IDs pass through secret and path redaction before upload, including standalone traces.

Sessions remain separate in Langfuse, because Langfuse has no session-to-session relation. Correlate through metadata instead, for example after resolving a child session id:

```bash
curl -s -u "$LANGFUSE_PUBLIC_KEY:$LANGFUSE_SECRET_KEY" \
  "$LANGFUSE_HOST/api/public/v2/observations?sessionId=<child-session-id>&fields=core,metadata"
```

Root sessions carry neither key. Only the parent session id is read; the parent transcript is never opened beyond the header line that names it.
Standalone compaction and idle cache-warming traces retain these links, including
cache-warming usage and costs, through both OTel export and REST fallback.

## Troubleshooting

### No traces appearing?

- Verify the API keys and run `/langfuse-setup` again if needed.
- Run `/langfuse-status` to confirm the loaded host, config source, privacy mode, and last runtime error.
- Confirm the Langfuse project is active and accepts writes.
- Confirm the keys have write permission.
- Look for `📊 Langfuse:` log messages in Pi output.

### Extension not loading?

```bash
pi list
pi install npm:pi-langfuse
```

### "Missing config" on startup?

- Run `/langfuse-setup`.
- Or set `LANGFUSE_PUBLIC_KEY` and `LANGFUSE_SECRET_KEY` before starting Pi.

### Model or cost not showing?

- Some providers do not expose cost information.
- Inspect the raw observation data in Langfuse traces.
- The `model` field can come from provider events, finalized assistant messages, `model_select`, or `ctx.model`.

### API key errors?

- Public keys start with `pk-lf-`.
- Secret keys start with `sk-lf-`.
- For self-hosted deployments, verify the host URL.

## Development Docs

Development setup, source installation, runtime architecture, trace model, tracked fields, and validation steps are documented in [DEVELOPMENT.md](./DEVELOPMENT.md) and [DEVELOPMENT_CN.md](./DEVELOPMENT_CN.md).

## License

MIT