# πŸ“ˆ pi-analytics β€” Understand Pi Activity Without Sending Data Away [![npm](https://img.shields.io/npm/v/@narumitw/pi-analytics)](https://www.npmjs.com/package/@narumitw/pi-analytics) [![Pi extension](https://img.shields.io/badge/Pi-extension-blue)](https://pi.dev) [![License: MIT](https://img.shields.io/badge/license-MIT-green.svg)](./LICENSE) Measure local model, skill, tool, and provider reliability activity without storing conversation or tool content or sending analytics elsewhere. ## ✨ Features - Collects content-free metrics automatically after installation. - Counts settled response cycles, logical LLM calls, skill activations, tool calls, and observed provider errors. - Reports tool failures and duration, model attribution, and per-response call distributions. - Separates recovered provider errors from terminal failures. - Provides Today, rolling 7-day, rolling 30-day, and all-time views through `/analytics`. - Stores private, versioned JSONL locally, isolates concurrent writers, and does not start an analytics server. ## πŸ“¦ Install Install persistently: ```bash pi install npm:@narumitw/pi-analytics ``` Try the published package without installing: ```bash pi -e npm:@narumitw/pi-analytics ``` Build and try a local checkout from the repository root: ```bash npm --workspace @narumitw/pi-analytics run build pi -e ./packages/pi-analytics ``` The package declares `dist/index.ts`, so an unbuilt local checkout must run the build before Pi loads the package directory. The extension uses Node's built-in filesystem APIs and has no native database dependency. Pi extensions run with the Pi process's user permissions, so install only trusted packages. ## πŸš€ Quick start Complete at least one Pi response, then run: ```text /analytics ``` The default overview covers the last seven rolling days: ```text Analytics Β· Last 7 days Response cycles 83 LLM calls 192 Calls per response 2.31 Β· P95 6 Tool calls 414 Tool errors 7 Skill activations 31 Provider errors 4 Recovered errors 3 ``` Use the menu to change the range or inspect Skills, Tools, Provider reliability, Response cycles, and Data & privacy. The dashboard includes finalized cycles and omits active work. ## πŸ“ Metric definitions ### Response cycles and LLM calls A **response cycle** starts when Pi begins agent work and normally ends at `agent_settled`. Retries, overflow-compaction recovery, tool follow-ups, and queued continuations before settlement remain in that cycle. An **LLM call** is one logical provider generation confirmed by an assistant-message lifecycle. A provider can make several HTTP attempts within it, so `429 β†’ 429 β†’ 200` counts as one LLM call, three observed HTTP responses, two provider errors, and one recovered generation. Pi cache-warming requests emit provider hooks without an assistant lifecycle; they are excluded from ordinary LLM-call and reliability counts rather than being reported as interrupted generations. ### Skills An activation is **User initiated** when an observed interactive or RPC `/skill:` input belongs to an active or subsequently started response cycle. This includes skill commands queued while Pi is streaming. It is **Model initiated** when Pi's built-in `read` tool successfully loads the exact canonical `SKILL.md` path Pi discovered. Each skill counts at most once per response cycle, with explicit user use taking precedence. Pi does not expose a first-class skill-invocation event or post-chain acceptance event to input observers. The extension does not count unsuccessful reads, provider behavior hidden from Pi, or non-standard loading such as `bash` plus `cat SKILL.md`. ### Tools A tool call starts at Pi's `tool_execution_start` event and finishes at `tool_execution_end`. The extension stores the tool name, model attribution, timing, completion state, and final error flag. Pi does not expose enough information to distinguish a call blocked by another extension from other tool errors, so both count as errors. ### Provider reliability Pi exposes HTTP responses and final assistant failures, but not every provider-SDK transport retry. The dashboard therefore calls these values **observed provider errors**. It reports HTTP 429 and 5xx counts, conservative error categories, recovered errors, and terminal failures. Raw error messages are classified in memory and discarded. ## πŸ’¬ Commands Run `/analytics` to inspect local usage, skills, tools, and provider reliability over a chosen time range. It accepts no arguments and supports TUI and RPC; print and JSON modes reject it before reading analytics data. Deleting analytics data requires confirmation, and cancellation leaves data unchanged. ## πŸ”’ Security and privacy Current analytics live under: ```text /pi-analytics/ β”œβ”€β”€ current └── generations/ └── / └── .jsonl ``` The opaque IDs are storage coordination identifiers generated by the extension; they are not Pi session IDs. On Unix, directories are restricted to mode `0700` and files to `0600`. Linked storage roots, markers, and writer files are rejected. Stored fields are limited to timestamps and durations; extension-generated record IDs; provider/model IDs and thinking level; tool and skill names; user/model skill source; counts, outcomes, and completion states; HTTP status codes; and classified provider-error categories. Provider-supplied tool-call IDs are replaced with local ordinals before publication. The extension does **not** store prompts, responses, thinking content, tool arguments or results, raw error messages, HTTP headers, cwd/project/file paths, session names or IDs, or credentials. Each finalized response cycle is one versioned, newline-terminated frame. Frames larger than 1 MiB are dropped. Local writes receive a 5-second cancellation deadline. Node filesystem cancellation is best-effort, so an operating-system request that has begun may still finish. The extension reports the first failed or timed-out write and a later recovery without exposing filesystem errors. `/analytics` streams and validates the active generation, checks cancellation between files and records, and periodically yields to the event loop. It ignores a crash-truncated final frame. Completed malformed frames and unsupported format versions fail closed without replacing existing files. ### Clear analytics data Choose **Data & privacy β†’ Clear analytics data…** to publish a fresh active generation atomically. Other Pi processes observe it before their next write. A record racing with Clear can land immediately before or after the switch. The extension then removes obsolete generations. If another process still uses an obsolete file, Clear reports incomplete physical cleanup but the logical clear remains complete. Stop other Pi processes and clear again to retry cleanup. File deletion does not guarantee secure erasure from the storage medium. ## 🧭 Legacy SQLite data Versions that used Turso/SQLite stored data in: ```text /pi-analytics.db /pi-analytics.db-wal ``` The JSONL version does not open, import, migrate, delete, or rewrite these files. New analytics start empty. If legacy history matters, stop every old Pi process first and preserve both files together. If it does not matter, stop every old Pi process before deleting both files manually. Never copy or remove only the main DB while an old process may still own its WAL. ## 🚧 Limitations - There are no retention settings; records remain until explicitly cleared. - Analytics are best-effort derived metadata. A failed or interrupted local write may be omitted. - Large all-time histories require scanning the active JSONL generation when the dashboard opens. - Prometheus, JSON/CSV export, cloud sync, browser dashboards, token/cost reporting, and project attribution are not included. - Statistics cover only events visible through Pi's public extension API. - Dedicated cache-warming observations require an upstream request-kind and completion signal; persisted warming token and cost totals are not imported into analytics. ## πŸ—‚οΈ Package layout ```text packages/pi-analytics/ β”œβ”€β”€ src/ # Authoritative implementation and helpers β”‚ β”œβ”€β”€ index.ts # Thin Pi entrypoint β”‚ └── analytics.ts # Collection lifecycle and dashboard command β”œβ”€β”€ dist/ # Generated Jiti runtime β”œβ”€β”€ scripts/build-runtime.mjs # Runtime builder └── test/ # Behavior and lifecycle coverage ``` The generated runtime is built from `src/index.ts` and does not import back into `src`. ## πŸ”Ž Keywords Pi extension, Pi coding agent, local analytics, agent skills, tool usage, model calls, provider reliability, JSON Lines, content-free metrics. ## πŸ“„ License MIT. See [`LICENSE`](./LICENSE).