# pi-nightshift

[![npm version](https://img.shields.io/npm/v/pi-nightshift?color=cb3837&logo=npm)](https://www.npmjs.com/package/pi-nightshift)
[![npm downloads](https://img.shields.io/npm/dm/pi-nightshift?color=cb3837&logo=npm)](https://www.npmjs.com/package/pi-nightshift)
[![license](https://img.shields.io/npm/l/pi-nightshift)](LICENSE)

**Background jobs for [pi](https://github.com/earendil-works/pi/).** 

English | [中文](README.zh.md)

Your agent stops blocking on long commands: `job_start` returns immediately, pi keeps working, and when the job finishes the agent is **woken up on its own** to collect the result — no polling, no sleep, no human.

Found these handy built-in tools while using [Deepseek Harness](https://github.com/deepseek-ai/deepseek-harness), and recreated them for [pi](https://github.com/earendil-works/pi/).

## What it feels like

```text
# Session 1
you ▸ pnpm test takes 3 minutes. Run it in the background and keep going.
pi  ▸ Started background job bash-1 (bash: pnpm test).

      (pi keeps working on your other task…)

      ⏰ background job bash-1 (bash: pnpm test) finished [status: completed, exit code: 0].
pi  ▸ Tests passed — 42 passed, 0 failed. (woke itself up, read the output, reported back)
```

The completion notice carries only the job id and status; the output stays in the registry and the woken agent decides whether to read it with `job_output`. No turn is spent on output nobody asked for.

## Installation

```bash
# Install from npm (recommended)
pi install npm:pi-nightshift

# …or from a local checkout
pi install ./pi-nightshift
```

That's it — the four tools (`job_start`, `job_output`, `job_list`, `job_kill`) work immediately. Optional config lives in `~/.pi/agent/nightshift.json` (see [Configuration](#configuration)).

## Tools

| Tool | Description |
|------|-------------|
| `job_start(command, label?)` | Run a shell command in the background; returns a `bash-N` job id immediately |
| `job_output(job_id, wait?, timeout_ms?)` | Read output **since the previous read** (no repeats); every response ends with `[status: ...]`. `wait: true` blocks until the job finishes or the timeout expires (a timed-out wait returns `[status: running]` and leaves the job alive) |
| `job_list()` | `<id> [<kind>] <status> - <label>`, or `(no background jobs)` |
| `job_kill(job_id, reason?)` | Request cancellation immediately (kills the whole process tree); a finished job returns its state without consuming unread output |

## File layout

```
~/.pi/agent/
  nightshift.json                  # optional config
  pi-nightshift/
    <sessionId>/
      bash-1.json                  # undelivered completion notices (durable)
```

Notice files are written **before** delivery and deleted only after the send is confirmed — a crash between the two leaves them for replay when the session reopens.

## How it works

### Completion detection

`job_start` spawns the command through the same shell pi's builtin bash uses (Git Bash on Windows, `/bin/bash` elsewhere). stdout/stderr accumulate into the job; a cursor tracks what has been read. The job **settles** on the child's `close` event (process exited *and* stdio closed — so the output is complete), not on `exit`.

### Two delivery lanes

When a job settles, where the notice goes is decided automatically:

- **Idle agent → wake** — `followUp + triggerTurn`: a brand-new turn starts, exactly like a user prompt. The model sees the notice and can call `job_output` right away.
- **Busy agent → steer** — the notice joins the current turn's next step.

### Wake budget (anti self-excitation)

A woken turn may start another job, whose completion wakes it again — an unbounded chain burns tokens. `maxConsecutiveWakes` (default 3) caps the turns one session may open this way; further notices degrade to steer. **Only real user input refills the budget** — a notice never refills what it spent.

### Batching

Jobs settling within 150ms of each other share one notice (`triggerTurn` takes the OR of the group), so a burst of parallel jobs wakes the agent once, not N times. A hard 1s cap from the first settle keeps a lone job's notice from waiting forever. Over 4KB the notice degrades to job ids, then to "N jobs finished, use job_list".

### Durable delivery

The notice is written to disk at settle (atomic tmp+rename), then sent. Accepted → file deleted. Crashed mid-delivery → file survives → replayed as steer (no triggerTurn) when the same session reopens. Jobs themselves die with pi — only *undelivered notices* are guaranteed to survive.

### Deduplication

The model already knows a job's outcome — through `job_kill`, a terminal read, or a `wait: true` that returned terminal state — so no notice is sent.

| Path | Why the model already knows |
|------|----------------------------|
| `job_kill` | The model killed the job itself, so it obviously knows it ended — a notice would be noise |
| Terminal read | `job_output` on a finished job returns `[status: completed]`, so the model learns it from the read |
| `wait: true` returning terminal state | `job_output(job_id, wait: true)` blocks until the job finishes — the model waited for it |

All three paths mark the job as `reported`, so the notice is dropped at settle (`decideLane` returns `drop`).

## Configuration

File: `~/.pi/agent/nightshift.json` (missing = all defaults). Invalid values fail at startup.

| Key | Default | Description |
|-----|---------|-------------|
| `waitTimeoutMs` | `30000` | Wait used when `wait: true` omits `timeout_ms` |
| `maxWaitTimeoutMs` | `600000` | Cap for model-supplied waits; larger values are clamped |
| `maxConsecutiveWakes` | `3` | Turns one session may open by wake before notices degrade to steer |
| `completionDelivery` | `"wakeup"` | `"wakeup"` opens a turn on an idle agent; `"quiet"` only steers, notices wait for the next real turn |

A `waitTimeoutMs` above `maxWaitTimeoutMs` fails at load.

## Troubleshooting

| Symptom | Cause | Fix |
|---------|-------|-----|
| Job never settles, no notice | A descendant process keeps stdout open (daemon-style) | The job is still producing output — `job_kill` it, or let it finish |
| Agent doesn't wake up | Wake budget spent (`maxConsecutiveWakes`), or `completionDelivery: "quiet"`, or the job settled mid-turn (steer lane) | Type anything — the notice rides the next real turn |
| Notices reappear after restart | Durable replay — a delivery was interrupted | By design; the replay is steer-only and never triggers a turn on its own |
| Same notice twice | Crash between "send accepted" and "file deleted" | Rare crash window, accepted tradeoff |
| `job_start` fails | No bash found (Windows without Git Bash) | Install Git for Windows or set `shellPath` in pi settings |
| Config changes ignored | Wrong file path, or invalid JSON (fails loudly at load) | Check `~/.pi/agent/nightshift.json` |
| `job_output` truncates output | Output exceeds pi's 50KB tool limit | Truncation is by design; read incrementally with repeated calls |

## Running tests

```bash
# Unit tests — no LLM, deterministic, fast
npm test

# End-to-end tests — require pi + API key
node test/e2e-wake.mjs     # idle agent auto-woken by a real job completion
node test/e2e-batch.mjs    # 3 jobs finishing together wake exactly once
node test/e2e-durable.mjs  # a crash-survived notice replays on session resume
```

| Level | Command | Requirements | What it tests |
|-------|---------|-------------|---------------|
| Unit | `npm test` | none | Registry status flow, incremental reads, lane decisions, batch window, config validation, durable store |
| E2E | `node test/e2e-*.mjs` | pi + API key | The full loop: real job → settle → wake → model reads output |

## Development

No build step — pi loads TypeScript directly.

```bash
pi -e ./src/index.ts   # interactive session with the extension
npm test               # unit checks
```

Structure: `src/index.ts` wires pi APIs only; the logic lives in pure modules (`jobs.ts` registry, `notify.ts` lane decisions, `batch.ts` window, `config.ts` validation, `durable.ts` store) so it runs under plain `node --test` without pi.

## Publishing (maintainers)

```bash
npm version patch   # or minor / major
npm publish         # requires an npm account; the pi-package keyword lists it in the pi package directory
pi install npm:pi-nightshift   # verify the published install
```

## Known limitations

- **Agent state-switch window may swallow a notice** — between the turn loop's last inbox check and the agent committing idle (`agent_settled`), the agent still reads as busy, so a job settling right then is steered and nothing wakes it. Closing this window requires changes in pi's agent loop; the extension has no hook for it.
- **A queued steer notice is lost if the session ends before delivery** — the durable file is deleted as soon as `sendCompletion` reports accepted, but in the steer lane accepted means queued, not delivered. If the session ends (quit, `/new`, resume away) with the notice still queued, the inbox is cleared and the file is already gone. The wake lane opens its run immediately and is unaffected; the replay path only covers files that were never sent.
- **A spent wake budget is not restored by time** — the wake budget refills only when a real user message arrives (`before_agent_start`); time passing never restores it. So an unattended agent that has spent its `maxConsecutiveWakes` (default 3) wakes will never wake itself again: further notices degrade to steer and queue until the user returns or something else opens a turn. This is the deliberate anti self-excitation bound; the tradeoff is that a fully unattended batch wakes at most 3 times.
- **Stream reads are single-consumer** — each job has one read cursor, and `job_output` advances it with every read, so consumed deltas don't come back. With the model as the only consumer this is fine; but two parallel `job_output` calls in the same turn, or a future second observer (another extension, another session), means the later reader gets `(no new output)`. Multiple consumers need a new API (one cursor per observer).
- **Output grows unbounded** — a long-running stream job's unread output accumulates in memory; terminal-style jobs (builds, tests) are unaffected.