# @erichll/pi-auto-review

A fail-closed, model-backed boundary reviewer for the Pi coding agent.

The extension participates in `@gotgenes/pi-permission-system` as the
`pi-auto-review` authorizer and exposes a process-local broker for OS sandbox
adapters. It is an authorizer *inside* the permission system, so installing
that dependency is a hard prerequisite (see [Install and enable](#install-and-enable)). The npm package and reviewer model have separate names:

- package: `@erichll/pi-auto-review`
- authorizer: `pi-auto-review`
- default reviewer model: `codex-auto-review`

## Contents

- [Install and enable](#install-and-enable)
- [Security model](#security-model)
- [Configuration](#configuration)
  - [Deadlines and retries](#deadlines-and-retries)
- [Operator feedback and exact retry](#operator-feedback-and-exact-retry)
- [Boundary broker API](#boundary-broker-api)
- [Reviewer context and token budgets](#reviewer-context-and-token-budgets)
- [Sandbox integration](#sandbox-integration)
- [Trust boundary](#trust-boundary)
- [Permission policy audit](#permission-policy-audit)
- [Telemetry](#telemetry)
- [Real-model smoke test](#real-model-smoke-test)

## Install and enable

> **Prerequisite:** pi-auto-review is an authorizer inside
> `@gotgenes/pi-permission-system`. Pi does not auto-install peer packages, so
> install the permission system separately (once per machine) before this
> extension. This release line supports permission-system 30.0.0 and later,
> validated through 33.0.1:

Node.js 22.13.0 or newer is required. Permission auditing uses Node's built-in
`node:sqlite`; it does not require a SQLite CLI, system SQLite library, or npm
SQLite package.

```bash
pi install npm:@gotgenes/pi-permission-system
pi install npm:@erichll/pi-auto-review
```

Install the package outside the agent-writable workspace (see
[Trust boundary](#trust-boundary)):

Add it to the permission-system authorizer chain:

```json
{
  "authorizerChain": ["pi-auto-review"]
}
```

The configured reviewer provider and model must also be registered in Pi.

## Security model

Requests pass through these controls in order:

1. deterministic hard denies;
2. bounded model review;
3. the local permission terminal when required; and
4. an exact, expiring, one-use grant for an external sandbox adapter.

The model cannot override hard denies or grant authority directly. External
adapters must consume the exact grant before retrying an operation. Changing
the command, path, resolved path, destination, cwd, agent, or tool input
invalidates that grant.

Permission-system downgrades authorizer allows on `path` and
`external_directory` to `defer`. In an interactive TUI,
`autoConfirmBoundedAllows` can bind the exact model allow to the immediately
following recognized permission dialog. The bridge is request-ID-bound,
expires after ten seconds, and is consumed once. Mode, component, request, or
event-order mismatches leave the original human dialog in place.

Permission-system 31.x also projects paths named directly by `for`/`select`
word lists and `case` subjects onto the existing `path` and
`external_directory` surfaces. Those additional requests follow the same
bounded review and one-shot auto-confirm rules; no new surface is implicitly
trusted.

Automatic review stops for the current turn after three consecutive denials or
ten denials in the last fifty reviews. An explicit denial tells the agent that
automatic policy denied the request (not a human click), not to rephrase or
circumvent the same action, and points to `/auto-review-approve` for an exact
non-critical reviewer retry or `/auto-review-break-glass` for a critical model
denial. Local deterministic hard denies never offer an override command.

Deterministic hard denies cover recursive forced wipes of `/`, `~`, and
`$HOME`. A named path under `/home/...` is reviewed by the model as high-risk,
not treated as a home-directory wipe.

## Configuration

Configuration is resolved in this order:

1. package defaults in `src/config.json`;
2. optional trusted user overlay at
   `~/.pi/agent/extensions/pi-auto-review/config.json`; and
3. optional project tighten-only settings at `.pi/pi-auto-review.json`.

Use the user-global file for normal customization. It may set any legal key:

```json
{
  "model": "provider/reviewer-model",
  "autoConfirmBoundedAllows": ["external_directory", "path"]
}
```

For a complete `@gotgenes/pi-permission-system` config that wires
`pi-auto-review` into the authorizer chain — a copyable baseline covering
read/write/edit, a read-only bash allowlist, an MCP discovery policy, and a
`path` deny block for secret and credential files — see
[`examples/pi-permission-system.config.example.json`](examples/pi-permission-system.config.example.json).

MCP rules are evaluated last-match-wins across the candidates derived for one
call: the tool name, its bare server, and `mcp_call`. Keep `"*"` first in the
`mcp` block and narrower server or tool entries after it, as the example does,
so a specific rule cannot be shadowed by the catch-all. A bare server entry
such as `"atlassian": "allow"` also matches the `mcp connect <server>`
candidate list, so it auto-allows starting that server — prefer the `server_*`
and `server:*` forms for tool calls. Any entry meant to narrow a broad allow
(such as a destructive-tool pattern) must be written after it, or
last-match-wins shadows it.

Package defaults:

```json
{
  "model": "codex-auto-review",
  "reasoning": "low",
  "timeoutMs": 90000,
  "maxTokens": 256,
  "retries": 2,
  "maxUserTranscriptTokens": 1200,
  "maxToolTranscriptTokens": 1200,
  "maxRelevantResultTokens": 800,
  "maxReviewerInputTokens": 8192,
  "breakGlassEnabled": true,
  "failureMode": "deny",
  "grantTtlMs": 60000,
  "autoConfirmBoundedAllows": ["external_directory", "path"],
  "policyAudit": {
    "enabled": true,
    "retentionDays": 180
  }
}
```

`failureMode: "deny"` is the default. `"defer"` falls through to the human
terminal. Set `autoConfirmBoundedAllows` to `[]` to keep every bounded allow
manual.

Project configuration may only lower timeouts, token/evidence limits, retries,
and grant TTL, set `failureMode` to `"deny"`, set `breakGlassEnabled` to
`false`, or remove auto-confirmed surfaces. It cannot re-enable break glass,
select a model, raise a trusted limit, or weaken fail-closed behavior. Invalid
configuration disables the reviewer for that session.

The trusted user config may set `policyAudit.enabled` and a retention of
1–3,650 days. Project config may only set `enabled: false` or shorten the
inherited retention; it cannot re-enable globally disabled collection or
extend retention.

### Deadlines and retries

`timeoutMs` is one deadline shared by model resolution, authentication, model
attempts, and retry delays. Each attempt receives only the remaining time.
Provider-internal retries are disabled, and a review makes at most two actual
model calls even when the public `retries` value is higher.

Valid decisions, output-length stops, timeouts, aborts, authentication/model/
request errors, and unknown failures do not retry. Empty, non-JSON, or
schema-invalid output and recognized connection, temporary 5xx, or 429
failures may retry once when the retry budget and deadline allow it. A
`Retry-After` above five seconds or beyond the remaining deadline fails closed.
Format retries preserve the canonical request and selected evidence and append
only a fixed, budget-checked schema correction.

## Operator feedback and exact retry

Interactive sessions show the current permission check in a single widget
above the editor. Each check first shows its surface, compact target, and the
dynamically configured reviewer model, then replaces that content in place
with the outcome, target and rationale, model, token usage, duration, and any
extra call count. A new check replaces the previous result. Allowed and
auto-confirmed results dismiss after eight seconds so they do not occupy the
editor; denials, deferrals, and local-confirmation waits stay until the next
check, a session change, or shutdown. Concurrent older checks cannot overwrite
the most recently started one.

Every request still has its own model call, verdict, grant, local confirmation,
and audit record. No new review-result transcript entries are written. Existing
`pi-auto-review` entries in historical sessions remain renderable. Non-TUI
modes retain best-effort notifications, and a failed TUI widget update falls
back to the same notification path. UI delivery never changes the authorization
result.

In an interactive TUI, `/auto-review-approve` lists up to ten recent
non-critical model denials from the current session. Selecting one asks the
agent to retry exactly that request. The old `/approve` command is not
registered.
The host-generated override:

- binds the complete request hash;
- expires after 60 seconds and is consumed once;
- remains separate from untrusted user/tool evidence;
- still goes through deterministic hard denies and model review; and
- cannot be reissued while a previous authorization for it is pending, or
  again after consumption until the same action is denied afresh (one approval
  per denial).

It is authorization evidence, not a direct allow.

`/auto-review-break-glass` is a separate, high-friction path for model denials
whose risk level is `critical`. It lists only critical model denials from the
same session made in the last five minutes. After showing the rationale,
surface, cwd, command or target summary, and request-hash fingerprint, it
requires an explicit confirmation and a random `BREAK-GLASS <CODE>` phrase
within 60 seconds. Successful confirmation creates a 60-second, one-use
authorization bound to the complete request hash, session, and original
request ID. The request hash deliberately excludes the per-attempt `id` and
`toolCallId` fields: the exact retry is a brand-new model tool call issued in
a later turn, so retry-minted identifiers cannot participate in the match.
The exact retry reruns local hard-deny checks, then allows directly without
calling the reviewer and, for sandbox adapters, still issues the normal
one-shot grant. A break-glass allow resets the turn circuit breaker (a human
re-authorized the scope), and break glass can be disabled in trusted or
project configuration with `breakGlassEnabled: false`.

## Boundary broker API

The extension publishes a process-local service at:

```ts
Symbol.for("pi-auto-review:boundary-approval-broker")
```

Adapters should use the exported helper:

```ts
import {
  getBoundaryBroker,
  type BoundaryRequest,
} from "@erichll/pi-auto-review";

const request: BoundaryRequest = {
  id: "sandbox-runtime-query-id",
  source: "sandbox-runtime",
  surface: "network",
  operation: "connect",
  cwd: "/workspace/project",
  command: "npm install",
  destination: "registry.npmjs.org:443",
};

const broker = getBoundaryBroker();
const decision = await broker?.review(request, {
  sessionId: "pi-session-id",
  scopeKey: "pi-session-id:turn-id",
  issueGrant: true,
});

// A break-glass allow includes structured provenance:
// decision.authorization = {
//   kind: "break-glass",
//   originalRequestId: "...",
//   confirmedAt: 0,
// };

if (
  decision?.kind === "allow" &&
  decision.grant &&
  broker?.consumeGrant(request, "pi-session-id", decision.grant.token)
) {
  // Retry this exact operation once inside the OS sandbox.
}
```

Grants expire after `grantTtlMs` and cannot be reused.

## Reviewer context and token budgets

The reviewer receives one compact canonical request plus bounded, explicitly
untrusted evidence. Selection is deterministic rather than semantic:

- the latest raw user message is the authorization anchor;
- older user messages require an exact request/tool/requester association or an
  exact trusted retry;
- tool calls require an exact tool-call ID, exact structured request fields, a
  surface profile, or a security-combination classification; and
- selected results stay paired with their producer and are limited to exact
  results, deletion prechecks, Git push context, matching branch protection,
  and Sandbox Runtime process evidence.

Unrelated reads, directory listings, builds, tests, assistant prose, and old
task history are excluded. Compaction and branch summaries are labeled as
non-authorization context and are never injected as user intent. The host does
not rewrite a model allow from regex matches on user text, vague continuations,
or a computed authorization ceiling; those judgments stay with the reviewer
model. Hard denies still terminate before the model.

The current operation appears once as stable, key-sorted JSON. Duplicate exact
tool-call arguments collapse to an ID/name/reason linkage shell, while fields
not represented by the request remain available as evidence. This compact
reviewer representation does not affect request hashes, grants, overrides, or
audit evidence.

`maxReviewerInputTokens` covers the fixed policy, canonical request, override,
evidence, omissions, JSON framing, and a 64-token provider-framing reserve. Its
legal range is 2,048–32,768. Because no matching tokenizer is bundled, the
`conservative:utf8` estimator counts every UTF-8 byte as one token.

When over budget, the host removes secondary reasons, older structured tool
matches, then optional producer/result units, re-estimating after each step. It
never silently removes the canonical request, exact override, latest user
evidence, security-combination evidence, exact tool linkage, or a required
surface profile. If mandatory evidence does not fit, review fails closed before
calling the model. More than four security-combination candidates also fails
closed locally.

## Sandbox integration

This package exposes the broker contract but does not intercept OS sandbox
events itself. Adapters translate a concrete boundary into a `BoundaryRequest`
and consume the exact grant before allowing it.

This monorepo's `pi-sandbox` adapter uses Anthropic Sandbox Runtime. Filesystem
policy is static and fail-closed; unmatched public network destinations use the
broker for one connection. Each Bash command or built-in subagent session owns
an independent Sandbox Runtime broker process. Adapter implementations should
use the package's `./sandbox` export and must not create broad permanent rules
in `.pi/sandbox.json`.

## Trust boundary

Production copies must live outside the agent-writable workspace. Pi installs
user npm and Git packages under `~/.pi/agent/npm/` and `~/.pi/agent/git/`.
Workspace-loaded copies are rejected unless local development explicitly opts
in:

```bash
PI_AUTO_REVIEW_ALLOW_UNTRUSTED_DEV=1 pi --approve
```

Writes to the installed reviewer package, its user-global configuration,
project and global security configuration, and the global audit directory are
deterministically denied.

## Permission policy audit

The extension observes terminal `permissions:decision` broadcasts and stores
only daily, redacted aggregates. Collection is enabled by default and retains
180 days. It starts when this version first initializes successfully; no
permission-system JSONL or RTK history is read or imported.

Before storage, Bash values become observed command templates such as
`git status --short`, `cat *`, or `npm test * --token *`. URLs, paths,
assignments, quoted values, and option values are replaced with `*`; safe bare
command words remain in plaintext so the report can emit a matching rule.
Paths used for path-surface statistics become only
`workspace`, `temp`, `home`, `external`, `sensitive`, or `unknown`. Request IDs,
project locations, and matched-rule patterns are HMACed. Syntactically valid
custom tool names are also retained in plaintext; invalid names stay anonymous
as `<custom-tool>`. The database never stores raw commands, raw paths,
URLs, credentials, or non-Bash tool arguments and inputs. Because safe bare Bash
words are retained, command templates can contain non-secret filenames or
project-specific labels; inspect suggestions before copying them.

Data lives at
`~/.pi/agent/extensions/pi-auto-review/policy-audit.sqlite`, beside an
owner-only HMAC key. The directory is mode `0700`; the key, database, WAL, and
SHM are mode `0600`. Initialization, lock, write, or corruption failures disable
auditing and warn once without changing any permission result. A corrupt
database is not deleted or rebuilt automatically. Disable new collection with:

```json
{
  "policyAudit": { "enabled": false }
}
```

Run `/auto-review-policy-audit` for a durable TUI report that is not sent to the
LLM. Options are `--days 1..retention`, `--top 1..50`, `--min-count >=1`, and
`--scope current|all`; defaults are `30`, `20`, `5`, and `current`.

In addition to the redacted statistics, the report proposes copyable
permission-system fragments. `--scope current` targets
`.pi/extensions/pi-permission-system/config.json`; `--scope all` targets
`~/.pi/agent/extensions/pi-permission-system/config.json`. The extension never
reads, edits, or rewrites either file. Merge suggested Bash patterns into the
existing `permission.bash` map and place the narrow allow entries after broader
matching ask entries, because permission-system uses last-match-wins.

Suggestions are discovered entirely from observed audit data; there is no
built-in tool, executable, Git action, package-manager action, or read-only
command catalog. Every valid observed permission surface and reliably templated
Bash command can qualify, including previously unknown tools and write
operations. Environment prefixes, compound syntax, pipelines, redirection,
path executables, and unreliable parsing block that Bash template. Forwarded
requests are statistics-only because the requester's cwd is not available. A
candidate needs at least `--min-count` successful ask-path approvals and no
denial, gate error, unavailable confirmation, blocked structural variant, or
forwarded-only evidence in the selected window. Existing policy/infrastructure
allows do not count as evidence of user friction. Repeated approval is evidence
of user preference, not independent proof that a capability is safe.

Schema v2 migrates v1 aggregates transactionally. Old totals remain visible,
`collecting_since` is preserved, and `recommendations_since` records when safe
recommendation evidence began; pre-migration data cannot produce a suggestion.

The report is an extension-owned custom entry. It is deliberately not exposed
as an Agent tool or packaged skill, so neither a tool schema nor skill metadata
is added to the model context. Permission changes remain a separate, explicit
user operation.

`PI_AUTO_REVIEW_AUDIT_FILE` remains a test/release observation sink. It is not
an input to this audit and supplies no RTK token or parsing metrics.

## Telemetry

Every actual model call emits an internal `review_attempt`; each approval emits
one `review_complete`. Events contain stable status/error classes, timings,
usage counters, evidence metadata, and prompt-part counts. They do not contain
prompt or response text, provider errors, credentials, headers, or URL query
values. Usage is marked `unknown_provenance` when pi-ai cannot distinguish
provider counters from initialized values, and `unavailable` when absent.

## Real-model smoke test

For a controlled real-model smoke test, load only the provider, reviewer,
sandbox, and audit listener:

```bash
PI_AUTO_REVIEW_ALLOW_UNTRUSTED_DEV=1 \
PI_AUTO_REVIEW_SMOKE_AUDIT_PATH=/tmp/pi-auto-review-smoke-audit.jsonl \
PI_AUTO_REVIEW_BASELINE_ID=reviewer-check \
PI_AUTO_REVIEW_BASELINE_CACHE_STATE=cold \
PI_AUTO_REVIEW_BASELINE_RUN_ORDER=1 \
PI_AUTO_REVIEW_BASELINE_SAMPLE_SET=reviewer-v1 \
PI_AUTO_REVIEW_SMOKE_TRIGGER=baseline-v1 \
pi --no-extensions --no-skills --no-prompt-templates --no-context-files \
  --no-builtin-tools --no-session --print \
  --extension /trusted/path/to/provider/extensions/index.ts \
  --extension ./packages/pi-auto-review/src/index.ts \
  --extension ./packages/pi-sandbox/src/index.ts \
  --extension ./scripts/real-model-smoke-audit.ts \
  --model provider/reviewer-model \
  "Run the configured synthetic reviewer baseline sample, then reply done."
```

The listener submits filesystem-write, network, delete, Git-push, and forwarded
subagent boundaries. No represented operation is executed; the main-agent
request is aborted after the samples finish. `--no-builtin-tools` ensures Bash
comes from `pi-sandbox` rather than Pi's built-in implementation.