πŸ” AgentLens

Open-source observability for AI agents β€” with a tamper-evident audit trail
Every event SHA-256 hash-chained & cryptographically verifiable β€” built for EU AI Act Article 12 record-keeping

PyPI npm server npm mcp License: MIT Build Status Container: ghcr.io/agentkitai/agentlens

πŸ“– Documentation Β· Quick Start Β· Dashboard Β· ☁️ Cloud

--- ## πŸ“‘ Table of Contents - [Tamper-Evident by Design](#-tamper-evident-by-design) - [Quick Start](#-quick-start) - [Architecture](#-architecture) - [Integration Guides](#-integration-guides) - [Key Features](#-key-features) - [Dashboard](#-dashboard) - [AgentLens Cloud](#-agentlens-cloud) - [Packages](#-packages) - [API Overview](#-api-overview) - [CLI](#-cli) - [Development](#-development) - [Contributing](#-contributing) - [AgentKit Ecosystem](#-agentkit-ecosystem) - [License](#-license) --- AgentLens is a **flight recorder for AI agents**. It captures every LLM call, tool invocation, approval decision, and error β€” then presents it through a queryable API and real-time web dashboard. ## πŸ”’ Tamper-evident by design What sets AgentLens apart from other observability tools: every event is **SHA-256 hash-chained** to the one before it, the same way git commits and blockchains are linked. The audit log is **append-only and cryptographically verifiable** β€” alter, delete, or reorder a single record after the fact and verification fails, pointing at the exact event that broke. Purpose-built for the record-keeping obligations of **EU AI Act Article 12** and the emerging **IETF Agent Audit Trail** work. **See it for yourself in 30 seconds** (needs Docker): ```bash git clone https://github.com/agentkitai/agentlens && cd agentlens ./demo/aha.sh ``` ```text 1/5 Starting AgentLens (SQLite, zero-config)… βœ“ up at http://localhost:3400 2/5 Ingesting a 5-event agent trace… βœ“ 5 events ingested 3/5 Verifying the hash chain… βœ“ CHAIN VALID β€” no tampering detected 4/5 Tampering with one event in the database… βœ“ altered llm_call (changed the logged model) 5/5 Re-verifying the hash chain… βœ— CHAIN BROKEN β€” tampering detected βœ… ``` The demo ingests a real trace, verifies the chain (passes), edits one record directly in the database behind the audit log's back, then re-verifies (fails). Auditors get a signed, verifiable JSON snapshot from `GET /api/audit/verify/export`. **Five ways to integrate β€” pick what fits your stack:** | Integration | Language | Effort | Capture | |---|---|---|---| | πŸ”­ **[OpenTelemetry](#-opentelemetry-any-genai-agent--no-sdk)** | Any | **Point your OTLP exporter** | Any `gen_ai.*`-instrumented agent β€” **no AgentLens SDK** | | πŸ€– **[OpenClaw Plugin](#-openclaw-plugin)** | [OpenClaw](https://github.com/openclaw/openclaw) | **Copy & enable** | Every Anthropic call β€” prompts, tokens, cost, tools β€” zero code | | 🐍 **[Python Auto-Instrumentation](#-python-auto-instrumentation)** | Python | **1 line** | Every OpenAI / Anthropic / LangChain call β€” deterministic | | πŸ”Œ **[MCP Server](#-mcp-integration)** | Any (MCP) | Config block | Tool calls, sessions, events from Claude Desktop / Cursor | | πŸ“¦ **[SDK](#-programmatic-sdk)** | Python, TypeScript | Code | Full control β€” log events, query analytics, build integrations | ## πŸš€ Quick Start **One command** β€” server + dashboard on SQLite, zero config: ```bash docker run -p 3400:3400 -e AUTH_DISABLED=true -e JWT_SECRET=dev-secret ghcr.io/agentkitai/agentlens # Open http://localhost:3400 ``` Or without Docker: ```bash npx @agentkitai/agentlens-server # http://localhost:3400 with SQLite β€” zero config ``` > `AUTH_DISABLED=true` is for a quick local trial (`JWT_SECRET` is still required by the hardened image). For anything shared, drop `AUTH_DISABLED`, set a real `JWT_SECRET`, and create an API key (below). **Full stack** (Postgres + Redis, auth, TLS) β€” runs from source: ```bash git clone https://github.com/agentkitai/agentlens && cd agentlens cp .env.example .env docker compose up # production overlay (auth, restart policies): docker compose -f docker-compose.yml -f docker-compose.prod.yml up ``` ### Create an API Key ```bash curl -X POST http://localhost:3400/api/keys \ -H "Content-Type: application/json" \ -d '{"name": "my-agent"}' ``` Save the `als_...` key from the response β€” it's shown only once. Then head to the [Integration Guides](#-integration-guides) to instrument your agent. πŸ“– [Full setup guide β†’](./docs/guide/) ## πŸ—οΈ Architecture ```mermaid graph TB subgraph Agents["Your AI Agents"] PY["Python App
(OpenAI, Anthropic, LangChain)"] MCP_C["MCP Client
(Claude Desktop, Cursor)"] TS["TypeScript App"] OC["OpenClaw Plugin"] end PY -->|"agentlensai.init()
auto-instrumentation"| SERVER MCP_C -->|MCP Protocol| MCP_S["@agentkitai/agentlens-mcp"] MCP_S -->|HTTP| SERVER TS -->|"@agentkitai/agentlens-sdk"| SERVER OC -->|HTTP| SERVER subgraph Server["@agentkitai/agentlens-server"] direction TB INGEST[Ingest Engine] QUERY[Query Engine] ALERT[Alert Engine] LLM_A[LLM Analytics] HEALTH[Health Scoring] COST[Cost Optimizer] REPLAY[Session Replay] BENCH[Benchmark Engine] GUARD[Guardrails] end SERVER --> DB[(SQLite / Postgres)] SERVER --> DASH["Dashboard
(React SPA)"] EXT["AgentGate / FormBridge"] -->|Webhook| SERVER ``` ## πŸ”§ Integration Guides ### πŸ”­ OpenTelemetry (any GenAI agent β€” no SDK) If your agent is already instrumented with the **[OpenTelemetry GenAI semantic conventions](https://opentelemetry.io/docs/specs/semconv/gen-ai/)** β€” via OpenLLMetry, OpenInference, or the official OTel instrumentations β€” just point its OTLP exporter at AgentLens. **No AgentLens SDK required.** ```bash # Send standard OTLP/HTTP to AgentLens (JSON or protobuf, /v1/traces) export OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:3400 export OTEL_EXPORTER_OTLP_TRACES_ENDPOINT=http://localhost:3400/v1/traces ``` AgentLens maps `gen_ai.*` spans into its model and into the tamper-evident audit log: | OTel GenAI span (`gen_ai.operation.name`) | Becomes | |---|---| | `chat` / `text_completion` / `generate_content` | a paired `llm_call` + `llm_response` (model, provider, messages, `usage.input_tokens`/`output_tokens`, finish reason, latency, **cost**) | | `execute_tool` | `tool_call` (`gen_ai.tool.name`, `gen_ai.tool.call.id`, arguments) | | `embeddings` | embedding event with token usage | | `invoke_agent` / `create_agent` | agent-invocation event | Each OTel **trace** maps to a session (or `gen_ai.conversation.id` if present), and every event is hash-chained like any other β€” so traces from any GenAI framework get the same verifiable audit trail. Set `OTLP_AUTH_TOKEN` to require a bearer token on the OTLP endpoints in production. > **Cost with no SDK:** OTel GenAI instrumentation reports tokens but rarely cost. AgentLens reconstructs `costUsd` from the model's per-1M-token pricing (fuzzy-matched on the model id), so OTel-only agents get the same cost analytics as SDK-instrumented ones β€” no per-call cost attribute required. ### πŸ€– OpenClaw Plugin If you're running [OpenClaw](https://github.com/openclaw/openclaw), the AgentLens plugin captures every Anthropic API call automatically β€” prompts, completions, token usage, costs, latency, and tool calls. ```bash cp -r packages/relay-plugin /usr/lib/node_modules/openclaw/extensions/agentlens-relay openclaw config patch '{"plugins":{"entries":{"agentlens-relay":{"enabled":true}}}}' openclaw gateway restart ``` Set `AGENTLENS_URL` if your AgentLens instance isn't on `localhost:3400`. See the [plugin README](./packages/relay-plugin/README.md) for details. ### 🐍 Python Auto-Instrumentation One line β€” every LLM call captured automatically across **9 providers** (OpenAI, Anthropic, LiteLLM, AWS Bedrock, Google Vertex AI, Google Gemini, Mistral AI, Cohere, Ollama): ```bash pip install agentlensai[all-providers] ``` ```python import agentlensai agentlensai.init( url="http://localhost:3400", api_key="als_your_key", agent_id="my-agent", ) # Every LLM call is now captured automatically ``` **Key guarantees:** βœ… Deterministic Β· βœ… Fail-safe Β· βœ… Non-blocking Β· βœ… Privacy (`init(redact=True)`) πŸ“– [Python SDK full docs β†’](./docs/guide/) ### πŸ”Œ MCP Integration For Claude Desktop, Cursor, or any MCP client β€” add to your config: ```json { "mcpServers": { "agentlens": { "command": "npx", "args": ["@agentkitai/agentlens-mcp"], "env": { "AGENTLENS_API_URL": "http://localhost:3400", "AGENTLENS_API_KEY": "als_your_key_here" } } } } ``` AgentLens ships **22 MCP tools** β€” covering core observability, intelligence & analytics, and operations. [Full MCP tool reference β†’](./docs/reference/api.md) πŸ“– [MCP setup guide β†’](./docs/guide/) ### πŸ“¦ Programmatic SDK **Python:** ```bash pip install agentlensai ``` ```python from agentlensai import AgentLensClient client = AgentLensClient("http://localhost:3400", api_key="als_your_key") sessions = client.get_sessions() analytics = client.get_llm_analytics() ``` **TypeScript:** ```bash npm install @agentkitai/agentlens-sdk ``` ```typescript import { AgentLensClient } from '@agentkitai/agentlens-sdk'; const client = new AgentLensClient({ baseUrl: 'http://localhost:3400', apiKey: 'als_your_key' }); const sessions = await client.getSessions(); ``` πŸ“– [SDK reference β†’](./docs/reference/api.md) ## ✨ Key Features - **🐍 Python Auto-Instrumentation** β€” `agentlensai.init()` captures every LLM call across 9 providers automatically. Deterministic β€” no reliance on LLM behavior. - **πŸ”Œ MCP-Native** β€” Ships as an MCP server. Works with Claude Desktop, Cursor, and any MCP client. - **πŸ”­ OpenTelemetry GenAI** β€” Ingests `gen_ai.*` OTLP traces from any OTel-instrumented agent (OpenLLMetry, OpenInference, official OTel) β€” no AgentLens SDK required. - **🧠 LLM Call Tracking** β€” Full prompt/completion visibility, token usage, cost aggregation, latency measurement, and privacy redaction. - **πŸ“Š Real-Time Dashboard** β€” Session timelines, event explorer, LLM analytics, cost tracking, and alerting. - **πŸ”’ Tamper-Evident Audit Trail** β€” Append-only event storage with SHA-256 hash chains per session. - **πŸ’° Cost Tracking** β€” Track token usage and estimated costs per session, per agent, per model. Alert on cost spikes. - **🚨 Alerting** β€” Configurable rules for error rate, cost threshold, latency anomalies, and inactivity. - **β€οΈβ€πŸ©Ή Health Scores** β€” 5-dimension health scoring with trend tracking. - **πŸ’‘ Cost Optimization** β€” Complexity-aware model recommendation engine with projected savings. - **πŸ“Ό Session Replay** β€” Step-through any past session with full context reconstruction. - **βš–οΈ A/B Benchmarking** β€” Statistical comparison of agent variants using Welch's t-test and chi-squared analysis. - **πŸ›‘οΈ Guardrails** β€” Automated safety rules with dry-run mode for safe testing. - **πŸ”Œ Framework Plugins** β€” LangChain, CrewAI, AutoGen, Semantic Kernel β€” auto-detection, fail-safe, non-blocking. - **πŸ”— AgentKit Ecosystem** β€” Integrations with [AgentGate](https://github.com/agentkitai/agentgate), [FormBridge](https://github.com/agentkitai/formbridge), [Lore](https://github.com/agentkitai/lore), and [AgentEval](https://github.com/agentkitai/agenteval). - **πŸ”’ Tenant Isolation** β€” Multi-tenant support with per-tenant data scoping and API key binding. - **🏠 Self-Hosted** β€” SQLite by default, no external dependencies. MIT licensed. ## πŸ“Έ Dashboard AgentLens ships with a real-time web dashboard for monitoring your agents.
πŸ“Έ Dashboard Screenshots (click to expand) ### Overview β€” At-a-Glance Metrics ![Dashboard Overview](demo/dashboard-overview.jpg) The overview page shows **live metrics** β€” sessions, events, errors, and active agents β€” with a 24-hour event timeline chart, recent sessions with status badges, and a recent errors feed. ### Sessions β€” Track Every Agent Run ![Sessions List](demo/dashboard-sessions.jpg) Every agent session with sortable columns: agent name, status, start time, duration, event count, error count, and total cost. ### Session Detail β€” Timeline & Hash Chain ![Session Detail](demo/dashboard-session-detail.jpg) Full event timeline with tamper-evident hash chain verification. Filter by event type, view cost breakdown. ### Events Explorer β€” Search & Filter Everything ![Events Explorer](demo/dashboard-events.jpg) Searchable, filterable view of every event across all sessions. ### 🧠 LLM Analytics β€” Prompt & Cost Tracking ![LLM Analytics](demo/dashboard-llm-analytics.jpg) Total LLM calls, cost, latency, and token usage across all agents with model comparison. ### 🧠 Session Timeline β€” LLM Call Pairing ![LLM Timeline](demo/dashboard-llm-timeline.jpg) LLM calls in session timeline with model, tokens, cost, and latency. ### πŸ’¬ Prompt Detail β€” Chat Bubble Viewer ![LLM Call Detail](demo/dashboard-llm-detail.jpg) Full prompt and completion in a chat-bubble style viewer with metadata panel. ### β€οΈβ€πŸ©Ή Health Overview β€” Agent Reliability ![Health Overview](demo/dashboard-health.jpg) 5-dimension health score for every agent with trend tracking. ### πŸ’‘ Cost Optimization β€” Model Recommendations ![Cost Optimization](demo/dashboard-cost-optimization.jpg) Analyzes LLM call patterns and recommends cheaper model alternatives with confidence levels. ### πŸ“Ό Session Replay β€” Step-Through Debugger ![Session Replay](demo/dashboard-session-replay.jpg) Step through any past session event by event with full context reconstruction. ### βš–οΈ Benchmarks β€” A/B Testing for Agents ![Benchmarks](demo/dashboard-benchmarks.jpg) Create and manage A/B experiments with statistical significance testing. ### πŸ›‘οΈ Guardrails β€” Automated Safety Rules ![Guardrails](demo/dashboard-guardrails.jpg) Create and manage automated safety rules with trigger history and activity feed.
## ☁️ AgentLens Cloud Don't want to self-host? **AgentLens Cloud** is a fully managed SaaS β€” same SDK, zero infrastructure: ```python import agentlensai agentlensai.init(cloud=True, api_key="als_cloud_your_key_here", agent_id="my-agent") ``` - **Same SDK, one parameter change** β€” switch `url=` to `cloud=True` - **Managed Postgres** β€” multi-tenant with row-level security - **Team features** β€” organizations, RBAC, audit logs - **No server to run** β€” dashboard at [app.agentlens.ai](https://app.agentlens.ai) πŸ“– [Cloud Setup Guide](./docs/guide/cloud-setup.md) Β· [Migration Guide](./docs/guide/cloud-migration.md) Β· [Troubleshooting](./docs/guide/troubleshooting.md) ## πŸ“¦ Packages ### Python (PyPI) | Package | Description | PyPI | |---|---|---| | [`agentlensai`](./packages/python-sdk) | Python SDK + auto-instrumentation for 9 LLM providers | [![PyPI](https://img.shields.io/pypi/v/agentlensai)](https://pypi.org/project/agentlensai/) | ### TypeScript / Node.js (npm) | Package | Description | npm | |---|---|---| | [`@agentkitai/agentlens-server`](./packages/server) | Hono API server + dashboard serving | [![npm](https://img.shields.io/npm/v/@agentkitai/agentlens-server)](https://npmjs.com/package/@agentkitai/agentlens-server) | | [`@agentkitai/agentlens-mcp`](./packages/mcp) | MCP server for agent instrumentation | [![npm](https://img.shields.io/npm/v/@agentkitai/agentlens-mcp)](https://npmjs.com/package/@agentkitai/agentlens-mcp) | | [`@agentkitai/agentlens-sdk`](./packages/sdk) | Programmatic TypeScript client | [![npm](https://img.shields.io/npm/v/@agentkitai/agentlens-sdk)](https://npmjs.com/package/@agentkitai/agentlens-sdk) | | [`@agentkitai/agentlens-core`](./packages/core) | Shared types, schemas, hash chain utilities | [![npm](https://img.shields.io/npm/v/@agentkitai/agentlens-core)](https://npmjs.com/package/@agentkitai/agentlens-core) | | [`@agentkitai/agentlens-cli`](./packages/cli) | Command-line interface | [![npm](https://img.shields.io/npm/v/@agentkitai/agentlens-cli)](https://npmjs.com/package/@agentkitai/agentlens-cli) | | [`@agentkitai/agentlens-dashboard`](./packages/dashboard) | React web dashboard (bundled with server) | private | ## πŸ”Œ API Overview | Endpoint | Description | |---|---| | `POST /api/events` | Ingest events (batch) | | `GET /api/events` | Query events with filters | | `GET /api/sessions` | List sessions | | `GET /api/sessions/:id/timeline` | Session timeline with hash chain verification | | `GET /api/analytics` | Bucketed metrics over time | [Full API Reference β†’](./docs/reference/api.md) ## ⌨️ CLI ```bash npx @agentkitai/agentlens-cli health # Overview of all agents npx @agentkitai/agentlens-cli health --agent my-agent # Detailed health with dimensions npx @agentkitai/agentlens-cli optimize # Cost optimization recommendations ``` Both commands support `--format json` for machine-readable output. See `agentlens health --help` for all options. ## πŸ› οΈ Development ```bash git clone https://github.com/agentkitai/agentlens.git cd agentlens pnpm install pnpm typecheck && pnpm test && pnpm lint # Run all checks pnpm dev # Start dev server ``` **Requirements:** Node.js β‰₯ 20.0.0 Β· pnpm β‰₯ 10.0.0 ## 🀝 Contributing We welcome contributions! See **[CONTRIBUTING.md](CONTRIBUTING.md)** for setup instructions, coding standards, and the PR process. ## 🧰 AgentKit Ecosystem | Project | Description | | |---------|-------------|-| | **AgentLens** | Observability & tamper-evident audit trail for AI agents | ⬅️ you are here | | [AgentGate](https://github.com/agentkitai/agentgate) | Human-in-the-loop approval gateway + reactive guardrails | | | [Lore](https://github.com/agentkitai/lore) | Cross-agent memory and lesson sharing | | | [AgentEval](https://github.com/agentkitai/agenteval) | Testing & evaluation framework | | | [FormBridge](https://github.com/agentkitai/formbridge) | Agent-human mixed-mode forms | | ## πŸ“„ License [MIT](LICENSE) Β© [Amit Paz](https://github.com/amitpaz)