# Going live with pi-helm

This is the production runbook: how to run the pi-helm cockpit against a real pi-web, a real GitHub
webhook, and (optionally) a real model judge. The three scripts in `scripts/` make it turnkey; this
doc explains what each does, the expected output, and how to operate it.

---

## What you're running

pi-web is a single Node process (`node --import tsx server.ts`). It loads pi-helm as an in-process
pi extension (from `$PI_WEB_CWD/.pi/web/extensions/`). When a session is created, the extension's
`session_start` handler fires and boots the whole cockpit in that process:

```
                 ┌──────────────────────────────────────────────────────────────┐
   GitHub ───> tunnel (public) ──> 127.0.0.1:$COCKPIT_PORT  (loopback listener)
                                          │
                                          ▼
   pi-web process (127.0.0.1:$PORT) ── pi-helm extension (in-process)
      │   ├── webhook  → shape Trigger → engine.receive
      │   ├── Engine   → policy gate → Assessor (judge) → decide
      │   ├── Executor → launches a pi-web worker session (one per ticket, own workspace)
      │   ├── Poller   → watches live workers; on settle harvests the final message → engine.report
      │   └── Outbox   → relays answers / reports / "needs a human" to the MAIN session
      └── main pi session (the human's session)  ◄───────────────────────────────┘

   durable state ──> $WS/.pi-helm/cockpit.db  (single SQLite writer, WAL)
   worker workdirs ─> $WS/.pi-helm/work/<ticketId>/
```

Key facts:
- The **webhook listener binds loopback only** (`127.0.0.1:$COCKPIT_PORT`, default `7890`). GitHub can't
  reach it directly, so a **public tunnel** fronts that one port. The tunnel never exposes the pi-web
  cockpit itself.
- The **executor talks back to the same pi-web** it runs in (`PI_WEB_BASE_URL` = that pi-web's address,
  authenticated with `PI_WEB_TOKEN`).
- Workers are real pi-web **sessions**, each in its own workspace under `COCKPIT_WORKDIR`.
- Results reach **you** as follow-up messages in the main pi session (`COCKPIT_MAIN_SESSION`).

---

## Interactive first (no tunnel, no GitHub needed)

You do **not** need a tunnel or a GitHub webhook to use pi-helm. Once the cockpit is running, ask your
agent in plain language and it calls the interactive tools:

| You say | Tool |
|---|---|
| "what's the helm status?" / "any open tickets?" | `helm_status` |
| "run a task: fix the flaky test in my repo" | `helm_run` { task, repo?, model? } |
| "what's the result of ticket <id>?" | `helm_result` { ticket } |
| "end ticket <id>" | `helm_end` { ticket, reason? } |

`helm_run` is always honored (you're `COCKPIT_OPERATOR`, an allowed sender, so it's an **explicit**
mandate) — even under the dry-run judge. Use this to try pi-helm immediately; the GitHub-webhook flow
below is for **unattended** operation (a repo event starts work on its own). Verify the interactive
path with no external dependencies:

```
cd <pi-helm>   # after launch-helm.sh has pi-web + the extension running
node --import tsx scripts/demo-console.mts "say HELMOK"   # console task → real worker session → report
```

---

## Prerequisites

| Thing | Notes |
|-------|-------|
| Node ≥ 22.5 | uses the built-in `node:sqlite`. |
| A pi-web checkout (`PIWEB`) | the dir with `server.ts` + a `node_modules` that has `tsx`. This is the pi-web you run the cockpit from. |
| A working agent dir | `models.json` + `auth.json` with a real model API key, for **real** mode. (Skip for a `PI_WEB_MOCK=1` pipeline dry-run.) |
| A tunnel | `cloudflared` (preferred, no account) or `ngrok`. |
| `gh` + `git` authenticated | to create the GitHub webhook (and to publish). |
| A GitHub repo | to attach the webhook to. |
| A model endpoint *(optional)* | any OpenAI-compatible chat endpoint, for the real model judge. Without it, the deterministic dry-run judge is used. |

---

## The turnkey flow (3 commands)

0. **Get the package** — clone the repo (or `npm pack` / extract the tarball) so `scripts/` is available.

1. **Start the cockpit + extension:**
   ```bash
   PIWEB=/path/to/piweb/src \
   BASE_AGENT=/path/to/a/working/agent/dir \
   COCKPIT_SECRET=<pick a long random string> \
   COCKPIT_WORKER_MODEL=unistra/t3/unistra-coder \
   COCKPIT_WORKER_MODELS=unistra/t3/unistra-coder \
   WS=/home/you/helm-live-ws \
   scripts/launch-helm.sh
   ```
   `COCKPIT_WORKER_MODEL` is the model workers run on by default; `COCKPIT_WORKER_MODELS` is the
   allowlist (a task asking for a model outside it is refused loudly). Leave both unset to inherit
   the pi-web host's default model. See the **Worker model** note in the README for the per-task `model`
   override and the capability caveat.
   This provisions a clean workspace, installs the extension, starts pi-web with every `COCKPIT_*` var
   set, creates a session (→ `session_start` → pi-helm boots + binds the webhook port), and prints the
   next steps. **Remember the `COCKPIT_SECRET` it shows and the webhook port.**

2. **Tunnel the loopback webhook port** (run where it can reach `127.0.0.1`, e.g. the same ai-jail):
   ```bash
   COCKPIT_PORT=7890 scripts/tunnel.sh        # prints the public https://… URL
   ```

3. **Point a GitHub webhook at it** (same secret as step 1):
   ```bash
   REPO=<you>/<repo> WEBHOOK_URL=<public url from step 2> \
   COCKPIT_SECRET=<same secret> scripts/github-webhook.sh
   ```

4. **First test:** open an issue or comment in that repo mentioning the agent (default `@agent`), then:
   ```bash
   COCKPIT_DB=/home/you/helm-live-ws/.pi-helm/cockpit.db npm run panel
   ```
   and watch your **main pi session** for the relayed answer/report.

---

## Full walkthrough

### Step 1 — `scripts/launch-helm.sh` (in detail)

What it does, in order:
1. **Clean slate.** Creates `$WS` (default `./helm-live-ws`) with a `.helm-live` marker. It only wipes a
   folder it created itself; otherwise it reuses and warns. It never touches an unrelated directory.
2. **Agent dir.** If `PI_CODING_AGENT_DIR` is set, uses it. Otherwise uses `$WS/agent`, copying
   `models.json`/`settings.json`/`auth.json`/`models-store.json` from `BASE_AGENT` if provided. (Real
   mode needs a configured agent dir; for a dry-run set `PI_WEB_MOCK=1`.)
3. **Port + secrets.** Fails if `$PORT` (default `4321`) is taken. Generates `PI_WEB_TOKEN` (pi-web
   bearer) and, if you didn't set `COCKPIT_SECRET`, generates one and prints it — **reuse that exact
   value in GitHub** (steps 2–3 must all agree on it).
4. **Installs the extension** by calling `scripts/install-ext.sh` with `PI_WEB_CWD=$WS`: it symlinks the
   source dirs into `$WS/.pi/web/extensions/` and drops a thin `pi-helm.ts` wrapper re-exporting
   `./extension/index.ts` (the entry imports its siblings by relative path, so the symlinks make them
   resolve).
5. **Starts pi-web** detached (`setsid`, survives the shell), with both the pi-web env and the pi-helm
   env on the process: `HOST=127.0.0.1 PORT=$PORT PI_WEB_CWD=$WS PI_WEB_TOKEN=… PI_CODING_AGENT_DIR=…`
   **and** `PI_WEB_BASE_URL=http://127.0.0.1:$PORT COCKPIT_SECRET=… COCKPIT_PORT=7890
   COCKPIT_DB=$WS/.pi-helm/cockpit.db COCKPIT_WORKDIR=$WS/.pi-helm/work` (+ optional `COCKPIT_MODEL_*`).
6. **Waits** for `GET /api/sessions` to return `200`, then **creates a session**
   (`POST /api/sessions/new`) — that fires `session_start`, so pi-helm launches and binds the webhook port.
7. **Prints** the live summary: cockpit URL + token, pi session id, the state DB path, which judge is
   active, and the exact commands for steps 2–4. State is also written to `$WS/helm-launch.env`.

Watch for the line `[pi-helm] started (webhook :7890, main session "…", judge …)` in `$WS/piweb.log`.

**Logging into the cockpit:** the printed URL is `http://127.0.0.1:<port>/?token=<token>`. The token
link normally auto-fills pi-web's token gate; if you still see pi-web's **login screen, paste that same
token into the field** — the gate takes the token (the value after `?token=`), not a password. Always
use the token from the *current* launch output: a relaunch generates a new one, so an older `?token=`
link (or a token you re-typed from a previous run) will be rejected with “Invalid credentials”.
`OPEN_AUTH=1` runs pi-web with the open policy (no API auth — handy for curl / programmatic / ai-jail
loopback use); the pi-web UI still needs a session, so the token gate remains for the browser.

**Dry run first (recommended):** add `PI_WEB_MOCK=1` to step 1. This boots the full pipeline against a
mock pi-web (no real model, no real sessions) so you can verify the extension loads, the webhook binds,
and the state DB is created before wiring a real tunnel + GitHub.

### Step 2 — `scripts/tunnel.sh` (in detail)

- Fronts `127.0.0.1:$COCKPIT_PORT` with a public URL. Prefers `cloudflared` quick tunnels
  (`https://….trycloudflare.com`, no account); falls back to `ngrok` (reads the URL from its local API
  at `127.0.0.1:4040`).
- Runs in the background, writes the URL + pid to `tunnel.env` (override with `TUNNEL_STATE=`), logs to
  `tunnel.env.log`.
- Prints the public URL and the exact GitHub webhook fields to paste (Payload URL, Content type
  `application/json`, the **Secret** = your `COCKPIT_SECRET`, the events).
- `scripts/tunnel.sh kill` stops it.

> **ai-jail note:** if pi-web runs inside ai-jail, run the tunnel **inside the same jail** so it can
> reach the loopback port. A tunnel on the host can't see a loopback port that only exists in the jail.

### Step 3 — `scripts/github-webhook.sh` (in detail)

- Creates (or updates) a webhook on `REPO` via `gh api`, with `config.url` = the tunnel URL,
  `content_type` = `json`, `secret` = your `COCKPIT_SECRET`, `active` = `true`, events =
  `issues`, `issue_comment`, `pull_request` (override with `EVENTS=`).
- Matching by URL/name means re-running it updates the existing hook instead of creating duplicates.
- Prints a **test ping** command (`gh api -X post repos/$REPO/hooks/$ID/pings`) and a
  **watch deliveries** command.

### Step 4 — first end-to-end test

1. `gh api -X post repos/$REPO/hooks/$ID/pings` → should 200.
2. Open an issue or a comment in `$REPO` that mentions the agent: e.g. body `@agent please summarize this`.
3. `gh api repos/$REPO/deliveries --jq '.[0].status'` → should be `Delivered` (not `Ignored`/error).
4. `COCKPIT_DB=… npm run panel` → the new ticket should appear.
5. Your **main pi session** should receive the relayed answer/report (the outbox). With the dry-run
   judge, a candidate whose trigger addressed the agent is accepted; with a real model, the judge decides.

---

## Model judge

- **Dry-run (default):** no `COCKPIT_MODEL_URL` → the deterministic judge accepts a candidate whose
  triggers addressed the agent and ignores otherwise. Great for testing the pipeline end to end.
- **Real model:** set `COCKPIT_MODEL_URL` (OpenAI-compatible chat endpoint), `COCKPIT_MODEL_TOKEN`,
  `COCKPIT_MODEL_NAME`. The assessor then sends a prompt and requires a strict `VERDICT: accept|ignore`
  line; anything unparseable is treated as `unavailable` and the eval retries rather than guessing.

Set these on `launch-helm.sh` (they're passed through to the pi-web process) — e.g.
`COCKPIT_MODEL_URL=https://… COCKPIT_MODEL_TOKEN=… COCKPIT_MODEL_NAME=gpt-4o scripts/launch-helm.sh`.

---

## Configuration reference

All read by the extension entry at `session_start` (set them on the pi-web process — `launch-helm.sh`
does this for you):

| Var | Meaning | Default |
|-----|---------|---------|
| `PI_WEB_BASE_URL` | Base URL of the running pi-web (**required**). | — |
| `PI_WEB_TOKEN` | pi-web bearer token (used to call the session API). | — |
| `COCKPIT_AGENT` | Agent name GitHub mentions route to. | `agent` |
| `COCKPIT_SECRET` | GitHub webhook HMAC secret (**required**). | — |
| `COCKPIT_PORT` | Loopback webhook port (`0` = free port; negative = disable). | `7890` |
| `COCKPIT_WORKDIR` | Root dir for per-ticket worker workspaces. | `.pi-helm/work` |
| `COCKPIT_DB` | SQLite path. | `.pi-helm/cockpit.db` |
| `COCKPIT_POLICY` | JSON `PolicyConfig` (the admission-table gate). | deny-inferred default |
| `COCKPIT_MAIN_SESSION` | pi session that relays outbox replies. | the session that started it |
| `COCKPIT_MODEL_URL` / `TOKEN` / `NAME` | Real model judge endpoint / bearer / model id. | dry-run judge / — / `gpt-4o` |
| `COCKPIT_WORKER_MODEL` | Default model workers run on (the per-task `helm_run { model }` overrides it). | pi-web default |
| `COCKPIT_WORKER_MODELS` | Comma-separated worker-model allowlist; a disallowed model is refused before any worker starts. | none (inherit default) |
| `COCKPIT_REPOS` | JSON `repos` map routing a repo task to a git worktree (see below). | none (scratch workspaces) |

pi-web's own vars (set by `launch-helm.sh`): `HOST=127.0.0.1`, `PORT`, `PI_WEB_CWD=$WS`,
`PI_CODING_AGENT_DIR`, `PI_WEB_MOCK`.

## Repo tasks, multi-project, and the PR lifecycle

By default a task runs in a **scratch** workspace under `COCKPIT_WORKDIR`. To run work against a
**real repo**, set `COCKPIT_REPOS` — one entry per project:

```sh
COCKPIT_REPOS='{"repos":{"acme/api":{"url":"https://github.com/acme/api.git","defaultBranch":"main"}}}'
```

A trigger names its repo via `meta.repo`, matched **exactly on the key**:
- a **GitHub** trigger sets `meta.repo` to the repo's `owner/name` — so key the entry by its
  **full_name** (`acme/api`), not a short alias;
- the **console** `helm_run { repo }` uses whatever value you pass.

No repo named (or a key that matches nothing) → the task runs in a scratch workspace.

When a repo task runs, pi-helm:
1. keeps a cached local clone under `COCKPIT_WORKDIR/.clones/`, and
2. gives the ticket its **own git worktree** on a fresh branch `helm-<ticket>`.

**The PR is the done-criterion.** The worker is told to keep its work on that branch, commit, and
`git push`. pi-helm **pre-binds** the branch-keyed PR follow *before* the worker reports, so the
moment the worker says “ready” the ticket **holds** on the open follow rather than completing.
When a pull request from `helm-<ticket>` is opened it is confirmed idempotently, and when it is
**merged/closed** the matching webhook ends the ticket `merged`/`closed`.

**What the worker needs to open a real PR (deployment requirement):** in the sandbox the worker has
only `git`, so it pushes the branch but cannot open a PR — you (or a later step) open it, then the
merged webhook closes the loop. For the unattended flow, the worker's worktree must have `gh`
authenticated (or the GitHub API + a token with write access to the repo) so `gh pr create` works;
`launch-helm.sh` passes through any `GH_TOKEN`/`GITHUB_TOKEN` you set on the environment.
The pre-bound follow means the ticket **never auto-completes** until the PR is actually merged, so a
worker that can't open the PR just leaves it held for you to open by hand.

### GitLab repos (optional)

A repo can live on **GitLab** instead of GitHub: set `"source": "gitlab"` on its `COCKPIT_REPOS` entry, e.g.

```sh
COCKPIT_REPOS='{"repos":{"you/repo":{"url":"https://gitlab.com/you/repo.git","source":"gitlab"}}}'
```

GitLab webhooks authenticate with a **static token** (not an HMAC), so the two halves need the same one:

```sh
# launch pi-helm with the static hook token (the same listener also accepts GitLab deliveries)
COCKPIT_GITLAB_TOKEN=<token> ...
# point a GitLab project hook at the listener (needs a GitLab PAT with api scope)
PROJECT=you/repo WEBHOOK_URL=<tunnel-url> GITLAB_TOKEN=<pat> COCKPIT_GITLAB_TOKEN=<token> scripts/gitlab-webhook.sh
```

The worker then needs `glab` + a GitLab token in its env (`GITLAB_TOKEN`/`GITLAB_PAT`/`GITLAB_HOST` —
`launch-helm.sh` forwards them) so it can `glab mr create`; the ticket's done-criterion becomes the
**merge request** merge. GitHub and GitLab repos can run on the **same** cockpit — the one listener
routes by the auth header (`X-Gitlab-Token` vs `X-Hub-Signature-256`) and webhook offsets are namespaced.

### Verifying the whole chain for real

`npm run verify:real` boots a real (no-mock) pi-web + extension, fires a signed GitHub delivery,
waits for the **real** worker to commit + push `helm-<ticket>` to a local seed repo, then closes the
ticket with a signed PR-merged delivery — proving env → extension → gate → worktree → real model →
`git push` → PR-follow → merge end-to-end. It needs `PIWEB` + `BASE_AGENT` (+ a real model key in
`BASE_AGENT/auth.json`) and a tool-capable `WORKER` (default `unistra/t3/unistra-coder`).

```sh
PIWEB=<.../piweb/src> BASE_AGENT=<.../agent> npm run verify:real
```

### Domain rules: why a mention may not become a ticket

The portable Engine only **accepts** work for an **explicit** mandate: a sender listed in
`COCKPIT_POLICY.allowedSenders` directly mentioning `@<agent>` (i.e. `addressed && listed`). Any other
sender's mention is **inferred**; the Policy admits it only into an allowlisted admission cell, and the
Engine then accepts it **only if** the assessment matched enabled rules (`COCKPIT_POLICY.enabledRules`).

Consequences for a live run:
- With the **dry-run (no-model) judge**, `enabledRules` is empty, so inferred candidates are *refused*
  (`notExplicit`). Only an allowed sender's direct mention produces a ticket. Set
  `COCKPIT_POLICY='{"allowedSenders":["<you>"]}'` for the quick test.
- With a **real model judge**, the judge can accept an inferred candidate when its assessment carries rules
  that match `enabledRules`, so non-allowlisted senders can drive work too (configure `admitCells` +
  `enabledRules` to match).

If a delivery is received but no ticket appears, the decision record's `reason` in `npm run panel` will say
why (e.g. `notExplicit`, or the event didn't shape into a `Trigger` at all).

---

## Operations

- **Inspect** any time (read-only, safe even while live):
  `COCKPIT_DB=<db> npm run panel`.
- **Logs / state:** `$WS/piweb.log` (pi-web + `[pi-helm] …` lines), `$WS/tunnel.env.log` (tunnel),
  `$WS/helm-launch.env` (port/token/db/session/pid), `tunnel.env` (public URL/pid).
- **Stop:** `scripts/launch-helm.sh kill` (pi-web) and `scripts/tunnel.sh kill` (tunnel). The GitHub
  webhook can stay; it just stops receiving once the listener is gone.
- **Crash recovery / restart:** everything is checkpointed in the single-writer SQLite DB. Kill and
  re-run `launch-helm.sh` (same `WS` + `COCKPIT_DB`); the Engine reopens the DB and **reconciles open
  tickets on startup** (re-launches dead workers, resumes where it left off). Don't delete the DB to
  "reset" — that's the checkpoint.
- **State locations:** DB at `COCKPIT_DB`; each ticket's worker workspace under `COCKPIT_WORKDIR/<ticketId>`.

---

## Security

- The webhook listener binds **loopback only**; the tunnel forwards **only the webhook port**, never the
  pi-web cockpit. Don't publish `$PORT` (pi-web) to the internet.
- Every GitHub delivery is **HMAC-verified** against `COCKPIT_SECRET` (mismatches are rejected); the
  pi-web session API is protected by `PI_WEB_TOKEN`.
- **Per-ticket isolation**: each ticket runs in its own workspace; the DB has exactly **one writer**.
- The tunnel (cloudflared quick / ngrok free) is **ephemeral** and unauthenticated in transit beyond
  GitHub's HMAC — fine for your own repo; for anything public, prefer a named Cloudflare Tunnel and keep
  the repo's webhook secret strong.

---

## Troubleshooting

| Symptom | Likely cause / fix |
|---------|--------------------|
| No `[pi-helm] started` in `piweb.log` | Extension didn't load. Confirm `install-ext.sh` symlinked into `$WS/.pi/web/extensions/`, that `PI_WEB_BASE_URL` + `COCKPIT_SECRET` were set on the process, and that the session was actually created. Re-check `$WS/piweb.log`. |
| Webhook `401` / signature rejected | `COCKPIT_SECRET` differs between GitHub and the running process; or the tunnel isn't forwarding the port; or the source IP is unexpected. Make all three use the **same** secret. |
| Delivery `Ignored` / no ticket | The event didn't shape into a `Trigger` that passes the `Policy` gate (mention routing + admission table). Check `COCKPIT_AGENT`, that the event mentions the agent, and the policy. |
| Tunnel URL disappears | Quick tunnels are ephemeral. For stability use a **named** Cloudflare Tunnel or an ngrok reserved domain, and re-run `github-webhook.sh` if the URL changes. |
| A Ticket never starts a worker | Check `GET /api/sessions` on pi-web lists the worker session, and that its workspace dir appeared under `COCKPIT_WORKDIR`. The executor maps `worker-<ticketId>` to pi-web's server-assigned session id. |
| Model judge stays `unavailable` | Endpoint/token/model wrong, or the model isn't emitting a strict `VERDICT: accept|ignore` line. It retries rather than guessing. |
| Port in use | Pick another `PORT=` (pi-web) and/or `COCKPIT_PORT=` (webhook). |
| Running under ai-jail | Run the **tunnel in the same jail** so it can reach the loopback webhook port. |

---

## End-to-end test checklist

```bash
# 0) mock dry-run to verify the pipeline (no tunnel/github needed)
PIWEB=/path/to/piweb/src WS=./helm-mock PI_WEB_MOCK=1 COCKPIT_SECRET=x scripts/launch-helm.sh
#    → expect "[pi-helm] started (webhook :7890, …, judge dry-run)" in $WS/piweb.log
COCKPIT_DB=./helm-mock/.pi-helm/cockpit.db npm run panel          # → "# pi-helm … _No open tickets._"
scripts/launch-helm.sh kill

# 1) real run
PIWEB=/path/to/piweb/src BASE_AGENT=/path/to/agent WS=/home/you/helm-live-ws \
  COCKPIT_SECRET=<secret> scripts/launch-helm.sh                  # note webhook port + secret
COCKPIT_PORT=7890 scripts/tunnel.sh                                # note public URL
REPO=you/repo WEBHOOK_URL=<public-url> COCKPIT_SECRET=<secret> scripts/github-webhook.sh
gh api -X post repos/you/repo/hooks/<id>/pings                     # → 200
#    … post an issue/comment mentioning @agent in you/repo …
gh api repos/you/repo/deliveries --jq '.[0].status'                # → "Delivered"
COCKPIT_DB=/home/you/helm-live-ws/.pi-helm/cockpit.db npm run panel   # → your ticket
#    … and the main pi session should have received the relayed answer/report …
```