# mu — Usage Guide A practical, copy-pasteable tour of mu (current main; v0.4.1). Everything below works against the built CLI. Terms are canonical — see [VOCABULARY.md](VOCABULARY.md) for definitions; the complete current verb list is in `## CLI — complete verb list` of [skills/mu/SKILL.md](../skills/mu/SKILL.md). > **Status:** v0.4.1 (pre-1.0). ~65 typed verbs across 9 > namespaces (`workstream`, `agent`, `task`, `workspace`, `log`, > `snapshot`, `archive`, `db`, `me`) plus bare top-level verbs > (`state`, `doctor`, `sql`, `undo`). Every verb accepts `--json` > (one allow-listed exception, `mu agent attach`), per-agent VCS > workspaces (jj/sl/git/none), activity log with `--tail` > subscription, bare `mu` TTY dashboard, canonical static state card > (`mu state` default / `--tui` render modes), whole-DB snapshots > auto-captured before destructive verbs + `mu undo` / > `mu snapshot {list,show}`, evidence on lifecycle verbs, schema v8 > (v5 surrogate INTEGER PKs + per-workstream UNIQUE on > operator-facing names; v6 added the `archive_*` family additively; > v7 dropped the dead `approvals` table; v8 adds `machine_identity` > and `workstream_sync` for db sync). > See [CHANGELOG.md](../CHANGELOG.md) for the release entry, > and [§ Not in 0.4.1](#whats-not-in-041-and-how-to-work-around-it) > at the bottom for the gaps that still need workarounds. *If anything below disagrees with `mu --help`, trust `mu --help`.* --- ## Table of contents 1. [Setup](#1-setup) 2. [Get oriented (`mu doctor`)](#2-get-oriented) 3. [Create a workstream (`mu workstream init`)](#3-create-a-workstream) 4. [Plan some work as a DAG (`mu task add`)](#4-plan-some-work-as-a-dag) 5. [See the graph (dashboard + state API)](#5-see-the-graph-dashboard--state-api) 5b. [The TUI dashboard (interactive)](#5b-the-tui-dashboard-interactive) 6. [Spawn a crew (`mu agent spawn`)](#6-spawn-a-crew) 7. [Watch the crew live (`tmux attach`)](#7-watch-the-crew-live) 8. [Send work to an agent (`mu agent send`)](#8-send-work-to-an-agent) 9. [Read what an agent did (`mu agent read`)](#9-read-what-an-agent-did) 10. [The claim protocol from inside a pane (`mu task claim`)](#10-the-claim-protocol--from-inside-an-agents-pane) 11. [Drop notes (durable context) (`mu task note`)](#11-drop-notes-durable-context) 12. [Close out a task](#12-close-out-a-task) 13. [The SQL escape hatch (`mu sql`)](#13-the-sql-escape-hatch-is-your-friend) 14. [Recovery scenarios](#14-recovery-scenarios) 15. [Cleanup](#15-cleanup) 15.5. [Archives — cross-workstream preservation](#155-archives--cross-workstream-preservation-of-task-graphs) 15.6. [Multi-machine sync](#156-multi-machine-sync) 16. [One-shot demo script](#16-one-shot-demo-script) 17. [Mental model in three sentences](#mental-model-in-three-sentences) 18. [What's NOT in 0.4.1](#whats-not-in-041-and-how-to-work-around-it) 19. [Where to go from here](#where-to-go-from-here) --- ## 1. Setup From npm (the common path): ```bash npm install -g @martintrojer/mu mu --version # → the current version ``` Update later via `npm install -g @martintrojer/mu@latest`. From a local checkout (when hacking on mu itself): ```bash npm install -g . # `prepare` script auto-builds; `mu` lands on $PATH mu --version # → the current version (see package.json) ``` To update the source-installed copy: pull from upstream, then `npm install -g .` from inside the checkout. The `prepare` script rebuilds before linking the new dist/. ### Install the bundled skill Mu ships a skill at `skills/mu/SKILL.md` that teaches the LLM running inside an agent pane how to use mu (the in-pane working loop, the subscribe-vs-poll pattern, the verb-list reference). The canonical install path is the [skills CLI](https://github.com/vercel-labs/skills) — it auto- detects every supported agent on your system (pi, claude-code, codex, opencode, cursor, ...) and installs into the right per-agent location: ```bash npx skills add martintrojer/mu # interactive: pick scope + agents npx skills add martintrojer/mu -g -y # global, no prompts (pi: ~/.pi/agent/skills/mu/) npx skills update mu # later, to refresh ``` If you installed mu from a local checkout (hacking on the skill itself), point the skills CLI at the checkout instead so edits flow straight through: ```bash npx skills add ./skills/mu # local-path source format (symlinks) ``` If you'd rather not use the skills CLI, mu's skill is just a directory with a `SKILL.md` — symlink or copy it into the agent's skills dir directly. For pi, that's `~/.pi/agent/skills/mu/` (per- user global) or `.pi/skills/mu/` (per-project). The convention `~/.agents/skills/mu/` (cross-tool location) is also picked up by pi and several other agents: ```bash # From an npm-global install mkdir -p ~/.agents/skills ln -sf "$(npm root -g)/@martintrojer/mu/skills/mu" ~/.agents/skills/mu # Or from a checkout ln -sf "$PWD/skills/mu" ~/.agents/skills/mu ``` ### For mu hackers: alias to the build output If you're hacking on mu itself and want fastest iteration, alias directly to the build output instead: ```bash npm install # deps only npm run build # produces dist/ alias mu="node $PWD/dist/cli.js" ``` See [README.md § Install](../README.md#install) for the full set of install patterns. mu requires tmux ≥ 3.0. Make sure you're inside a tmux session before proceeding: ```bash tmux # if you're not already in one ``` --- ## 2. Get oriented For a human at an interactive terminal, bare `mu` is the home base: it launches the read-only TUI with every workstream on the machine loaded as tabs. Initial tab focus uses this ladder: `$MU_SESSION` when it names a loaded workstream; then the current tmux session name when it is `mu-`; then best-effort cwd detection against registered workspace paths; then cwd equal to the VCS-derived project root of any loaded workstream's workspaces (ties broken by most-recent workstream activity); then tab 0. If no workstream exists yet, it prints help plus the one-paste start command: ```bash mu # Get started: mu workstream init ``` For scripts, agents, CI, and pipes, bare `mu` deliberately does NOT enter the TUI: when stdout is not a TTY it prints `mu --help`. Use explicit typed verbs and `--json` for the API surface: ```bash mu state -w --json MU_NO_TUI=1 mu # force the non-TTY/help path even in a terminal ``` Run the diagnostic once to check tmux + DB health: ```bash mu doctor ``` Expected output: ``` mu doctor tmux: ok (tmux 3.6a) $TMUX: set db: ok (/Users/you/.mu/mu.db) workstream: none (pass --workstream or run inside an mu- tmux session) ``` The "workstream: none" line is expected — we haven't joined one yet. Get the full command list: ```bash mu --help ``` Every verb's `--help` is exhaustive (flags, defaults, interactions). Every successful invocation also prints a dim `Next:` block of suggested follow-up commands at the bottom — you never have to leave the terminal to learn what to do next. Every verb accepts `--json` for machine-readable output. Errors in `--json` mode emit a `{ error, message, nextSteps, exitCode }` record to stderr; the `nextSteps` array carries actionable resolutions you can `eval` directly. (One verb opts out: `mu agent attach`, which prints a `tmux attach` command for a human to copy.) ### CLI conventions: validation errors Every operator-error path — missing required option, unknown option, unknown subcommand, missing positional, type-coercion failure, mutex flags, range checks — produces a uniform surface: - **Human path**: red `error: ` on stderr, then the failing subcommand's `--help` block (same text as `mu --help`), then exit **2**. - **`--json` path**: a structured envelope on stderr: ```json { "error": "UsageError", "message": "--self and --for are mutually exclusive", "nextSteps": [], "exitCode": 2, "usage": { "command": "mu task claim", "synopsis": "mu task claim [options] ", "description": "...", "args": [{"name": "id", "required": true, "variadic": false, "description": ""}], "options": [{"flags": "--self", "description": "...", "mandatory": false, "valueRequired": false}, ...] } } ``` `usage.options[].mandatory` is `true` when the operator MUST pass the option (`.requiredOption()` in commander terms). `valueRequired` is `true` when the option's argument can't be omitted if the flag IS passed (`` form vs bare flag). The two are independent. Exit 2 is the consistent code for the whole operator-error class — commander mistakes and handler-thrown `UsageError`s alike. Other classes keep their own codes (3 = not found, 4 = conflict, 5 = substrate, 6 = reaper, 7 = stall). ### CLI conventions: `--json` collection envelope Collection-read verbs emit a canonical `{items: T[], count: number}` shape on stdout: ```bash $ mu task list -w foo --json {"items":[{"name":"a",...},{"name":"b",...}],"count":2} ``` `count` is `items.length` pre-computed so `jq '.count'` is one less hop than `jq '.items | length'`. Future siblings layer on without breaking the existing two fields. Today `mu workspace commits --json` also includes `vcs`, `baseRef`, and `workspacePath` siblings because that verb already computes the workspace's fork metadata. Applies to: `mu task list / next / owned-by / notes`, `mu workstream list`, `mu workstream destroy --empty` (dry-run), `mu archive list / search`, `mu workspace list / orphans / commits`, `mu snapshot list`, `mu log -n N` (read). Two deliberate carve-outs: - **`mu sql --json`** keeps bare-array rows. The verb is the typed- escape hatch; row shape is per-query, not part of the typed contract. - **`mu log --tail --json`** emits NDJSON (one JSON object per line) because it's a stream, not a collection. Stream consumers want one envelope per row, not a single envelope that grows forever. Singleton verbs (`mu task show`, `mu agent show`, `mu workstream init`, `mu task close`, ...) keep their existing object envelopes with named top-level fields (`{task, blockers, dependents, notes}`, `{taskName, ..., nextSteps}`, etc.). The `items + count` envelope is for collection reads only. ### CLI conventions: multi-value flags Multi-value flags accept either repeated invocations (`--blocked-by a --blocked-by b`) or a comma-separated value (`--blocked-by a,b`) or any mix (`--blocked-by a,b --blocked-by c`). All three forms collapse to the same list internally. The syntactic signal is `` in the help-text metavar (the triple-dot); the parenthetical "repeat or comma-separate; or both" reinforces it. Variadic positionals (e.g. `mu task wait a b c`) keep their Unix-style space-separated shape — operands are not commas. Single-valued flags (`-w`, `--by`, `--title`, ...) stay single. The `--status` filter on `mu task list` and `mu task next` accepts the same multi-value shape (`--status OPEN,IN_PROGRESS`, `--status OPEN --status CLOSED`, or any mix) and returns the union. Missing `--status` keeps today's no-filter shape (no auto-default). `mu task wait --status` stays single — the verb means "wait until reaches THIS status". --- ## 3. Create a workstream A **workstream** is mu's unit of organization. One workstream = one tmux session = one logical project. Multiple workstreams on the same machine are isolated (partitioned in the SQLite registry by `session_id`); they never see each other's agents. ```bash mu workstream init auth-refactor ``` ``` Created workstream auth-refactor (tmux session mu-auth-refactor) Attach with: tmux a -t mu-auth-refactor Spawn agents with: mu agent spawn -w auth-refactor ``` Behind the scenes: `tmux new-session -d -s mu-auth-refactor` plus a placeholder window so the session is non-empty. The session sits there detached, waiting for agents. To see what's already on the machine before picking a name: ```bash mu workstream list ``` ``` ┌──────┬───────┬────────┬───────┬───────┬───────┐ │ name │ tmux │ agents │ tasks │ edges │ notes │ ├──────┼───────┼────────┼───────┼───────┼───────┤ │ r6a │ alive │ 0 │ 2 │ 1 │ 1 │ │ r6b │ — │ 0 │ 0 │ 0 │ 0 │ └──────┴───────┴────────┴───────┴───────┴───────┘ ``` The list is the **union** of three sources: distinct `agents.workstream`, distinct `tasks.workstream`, and tmux sessions matching `mu-*`. So a freshly-`init`'d workstream with no tasks/agents still shows up (via its tmux session), and a workstream whose tmux session was killed externally still shows up (via its surviving DB rows) so you can `mu workstream destroy` to clean up properly. ### How mu finds your active workstream Every command after `init` needs to know which workstream you're in. Resolution order, first match wins: 1. **`--workstream ` flag** explicitly 2. **`MU_SESSION` env var** (`export MU_SESSION=auth-refactor`) 3. **Current tmux session name** (mu reads `tmux display-message -p '#S'` and strips the `mu-` prefix) 4. Error if none of the above The third option is the most ergonomic. Once you `tmux a -t mu-auth-refactor`, every command "just works" without flags. ### Off-the-cuff agents: the `scratch` workstream When you just want one helper agent you can keep talking to — not a crew, not a task DAG, not per-agent workspaces — skip the `init` step entirely and spawn into the reserved `scratch` workstream: ```bash mu agent spawn helper -w scratch # auto-creates mu-scratch mu agent send helper 'Investigate the failing test in foo.spec.ts' mu agent read helper -n 50 # check on it anytime mu agent close helper -w scratch # done ``` `scratch` is a shared, ephemeral bucket. It auto-creates on first spawn (no `mu workstream init` needed), tasks are optional (a task-less helper is fine), and idle scratch agents get a gentle staleness nudge in `mu state` and the TUI so they don't pile up forgotten. Because it's deliberately low-ceremony, the name is **reserved**: `mu workstream init scratch` is rejected (it only ever auto-creates on spawn). `mu workstream destroy scratch` works normally when you want to tear the whole bucket down. ``` $ mu workstream init scratch error: workstream name "scratch" is reserved: it is the off-the-cuff bucket and auto-creates on first spawn. Don't 'init' it. ``` **When to use which:** reach for a real `mu workstream init ` the moment coordination *is* the work — multiple agents, dependencies between tasks, gated review. Reach for `scratch` when you just want a single driveable, observable helper. And reach for [`pi-subagents`](https://github.com/nicobailon/pi-subagents) when you only need a one-shot result back rather than a channel you keep open. --- ## 4. Plan some work as a DAG Tasks have **mandatory** `impact` (1–100) and `effort-days` (>0). Edges are blocks-relationships, modelled as **`--blocked-by`** on `mu task add` (and `mu task reparent`): `--blocked-by design` means "this task is blocked by `design`; it can't start until `design` closes." Tasks are **scoped to a workstream** — the dashboard and state views only show tasks for the workstream you're viewing. ```bash # --workstream can be omitted if you're inside the workstream's tmux # session or have $MU_SESSION exported. mu task add design \ --workstream auth-refactor \ --title "Design auth module" \ --impact 80 --effort-days 2 mu task add build \ --workstream auth-refactor \ --title "Build auth module" \ --impact 80 --effort-days 5 \ --blocked-by design mu task add review \ --workstream auth-refactor \ --title "Review auth module" \ --impact 60 --effort-days 1 \ --blocked-by build ``` Add initial context at creation time with `--note` (shell escapes like `\n` are translated the same way as `mu task note`). Use `--note-author ` when the default actor label is not what you want: ```bash mu task add bugfix \ --workstream auth-refactor \ --title "Fix token refresh race" \ --impact 70 --effort-days 1 \ --note 'REPRO: login, wait 24h, refresh\nSCOPE: auth refresh only' \ --note-author orchestrator ``` Task creation + the initial note are one transaction: if either part fails, neither lands. Each task validates its id (`/^[a-z][a-z0-9_-]{0,63}$/`) and rejects duplicates. If you tried `mu task add x --blocked-by y` while `y` already transitively depended on `x`, mu would refuse with a `CycleError`. **Task ids are per-workstream unique.** The same local id can exist in multiple workstreams, so cross-workstream references use the qualified form `/` when a global scope is needed. Blocks-edges are always same-workstream — if a blocker resolves outside the target workstream, mu refuses with a `CrossWorkstreamEdgeError`. ### Modeling external dependencies When a task is waiting on something outside this repo (an upstream PR shipping, a vendor releasing v3 of an API, a coworker's review), don't reach for a new status — add a **placeholder task** for the external thing and `--blocked-by` it. The DAG already encodes "blocked": ```bash mu task add upstream_react_19_lands -w gchatui-node \ --title "Wait for React 19 release (vendor)" \ --impact 30 --effort-days 0.1 mu task note upstream_react_19_lands -w gchatui-node \ "Tracking https://github.com/facebook/react/issues/... Last checked: 2026-05-15. When this lands: bump react in package.json + re-run upgrade tasks." mu task add migrate_to_use_action -w gchatui-node \ --title "Migrate ChatInput to React 19 useActionState" \ --impact 60 --effort-days 1 \ --blocked-by upstream_react_19_lands ``` When the upstream lands, `mu task close upstream_react_19_lands --evidence "shipped 2026-08-12"` — the dependent flips from blocked to ready in the same render. If the upstream gets cancelled, `mu task reject upstream_react_19_lands --cascade --yes` propagates REJECTED through every dependent so you re-think the cascade explicitly. Benefits over a hypothetical `BLOCKED` status: - The placeholder's notes are the audit trail ("who I nudged, when"). - One placeholder cleanly blocks N downstream tasks. - Reject cascade just works. - No new vocabulary; anyone who knows `--blocked-by` already knows this. - The placeholder's own status carries nuance (`OPEN` = someone external is working it, `DEFERRED` = parked indefinitely, `IN_PROGRESS` = you're actively chasing it). --- ## 5. See the graph (dashboard + state API) `mu` exposes one logical "what's going on?" view with two renderers: | Surface | Use it for | | -------------------- | --------------------------------------------------------------- | | **bare `mu`** | A human at a terminal — launches the interactive TUI dashboard. | | **`mu state --tui`** | Same TUI, explicitly opt-in. Useful in scripts / aliases. | | **`mu state`** | Static text card. JSON-friendly; pipeable; `watch`-able. | | **`mu state --json`** | The canonical full snapshot. Agents and scripts read this. | The interactive surface is large enough to deserve its own section — see [§ 5b. The TUI dashboard](#5b-the-tui-dashboard-interactive) immediately below. The rest of this section is the static / JSON contract. ### Static state card (`mu state`) For an agent/script or a static capture, use explicit `mu state`: ```bash mu state -w auth-refactor # human-readable card mu state -w auth-refactor --json # full snapshot mu state --all --json # every workstream on this machine ``` The static card includes every section the TUI cards summarize: agents + orphans + tracks + ready / in-progress / blocked / recent-closed tasks + workspaces + recent events. `--json` emits the same full snapshot shape regardless of `--tui`. **JSON shapes** - `mu state --json` (single-ws): flat `{ workstreamName, agents, orphans, tracks, ready, blocked, inProgress, recentClosed, workspaces, recent }`. - `mu state --json` (multi-ws): wrapped `{ workstreams: [{...}, ...] }`. - bare `mu --json`: prints `--help` rather than entering the TUI; use `mu state --json` for the full snapshot. - `--tui` is render-only and incompatible with `--json` (the TUI has no JSON shape). **Multi-workstream**: pass `-w` multiple values, or `--all`. See [CLI conventions](#cli-conventions-multi-value-flags). In static mode N≥2 stacks one per-workstream card after another. > **Migrating from old state surfaces**: `mu state --hud` and > `mu state --mission` were removed in v0.4; use `mu state --tui` > for the interactive surface and `mu state --json` for the full > snapshot. `tmux display-popup -E 'mu state -w X'` keeps working > unchanged for popup-card use. --- ## 5b. The TUI dashboard (interactive) The interactive TUI is mu's flagship human surface. It is **read-only by design** — every act-intent `y`anks the canonical `mu` command to your clipboard so you run mutations from your shell, with one documented escape (`t` in git-show drills runs `tuicr` in the project root, see below). ### Launch ```bash mu # bare; opens the TUI when stdout is a TTY mu state --tui # explicit; same surface mu state --tui -w a,b # restrict to specific workstreams mu state --tui --all # all workstreams (default for bare `mu`) MU_NO_TUI=1 mu # force the help path even in a TTY mu --json # also forces help; pipe `mu state --json` ``` Quit with `q` or `Ctrl-C`. The dashboard restores your scrollback on exit (alt-screen). **Initial active tab** is picked from this ladder: `$MU_SESSION` → current tmux session name (`mu-`) → cwd inside a registered workspace → cwd at the VCS-derived project root of any workstream (newest activity wins ties) → tab 0. ### Layout: 10 cards, responsive columns The dashboard renders 10 toggleable cards with rounded borders and section headers inset into the top border line (lazygit / btop / k9s convention): | Slot | Card | Toggle | Popup | Content | | ---- | ------------- | ------ | --------- | ---------------------------------------------------- | | 0 | Commits | `0` | `Shift+0` | Recent project-root commits (git / jj / sl) | | 1 | Agents | `1` | `Shift+1` | Active agents + status + cli + role | | 2 | Tracks | `2` | `Shift+2` | Parallel tracks (union-find clusters) | | 3 | Ready (Tasks) | `3` | `Shift+3` | Ready-to-claim tasks (no open blockers) | | 4 | Activity log | `4` | `Shift+4` | Recent `agent_logs` events | | 5 | Workspaces | `5` | `Shift+5` | Per-agent VCS workspaces + behind/dirty status | | 6 | In-progress | `6` | `Shift+6` | IN_PROGRESS tasks owned by agents | | 7 | Blocked | `7` | `Shift+7` | Tasks with at least one open blocker | | 8 | Recent | `8` | `Shift+8` | Recently CLOSED tasks | | 9 | Doctor | `9` | `Shift+9` | Cheap health checks (schema, ghosts, orphans, …) | | — | DAG | — | `g` | Full task DAG forest (keybind-only) | | — | All tasks | — | `t` | Sortable / filterable list of every task | Digit toggles HIDE / SHOW the card on the dashboard; `Shift+digit` opens the matching fullscreen popup. **Single-popup invariant**: only one popup is visible at a time; `Esc` / `q` returns to the dashboard with all toggles + tick rate preserved. **Responsive layout**: cards stack below 120 cols, then reflow into pair-aware 2 / 3 / 4-column layouts at 120 / 180 / 240 cols. Each visible card gets a dynamic row budget so a noisy list cannot crowd out its siblings; overflow shows as `+N more · Shift+N` inset into the bottom border. On very short panes, the dashboard culls low-priority cards (Doctor → Recent → Workspaces → …) and shows `+N cards hidden · resize taller` until the surviving cards fit. Dashboard ordering is slot-stable: within each rendered column, non-stream cards are ordered by toggle digit ascending; stream cards (Commits, Activity log) sit as natural trailers, with slot 0 trailing last. ### Multi-workstream tabs When the TUI is launched with N≥2 workstreams (e.g. bare `mu` on a machine with multiple workstreams, or `mu state --tui -w a,b,c`), a compact tab strip renders above the cards: ``` workstreams: ▸ auth-refactor · ui-rewrite · demo (Tab / Shift-Tab) ``` - `Tab` cycles forward, `Shift-Tab` backward (suppressed inside popups so the same key still navigates inside popups that bind it locally). - The active tab name appears in the status bar's right zone next to the tick rate. - Cards / popups always operate on the active tab — there's no per-row workstream column. - For N=1 the strip renders nothing (frame is byte-identical to single-ws TUI). - When the workstream set is wider than the terminal, the strip windows around the active tab and shows `‹N` / `›N` counters for hidden workstreams. - A `*` prefix (e.g. `*scratch`) marks the reserved off-the-cuff `scratch` workstream as ephemeral, so it's not mistaken for a durable crew. A `~` prefix marks a workstream presumed parked on another machine; `~` wins when both apply. ### Popup drills `Enter` in any list popup drills into the focused row. Where the row is itself an entity (a task), a further `Enter` chains into the shared read-only task-detail leaf (notes timeline): - **Tracks popup (`Shift+2`)**: list of tracks → `Enter` opens the track's task list → `Enter` opens that task's notes timeline. - **Ready / In-progress / Blocked / Recent / All-tasks**: list of tasks → `Enter` opens notes; `y` yanks `mu task show ` (or `mu task claim` / `mu task close` / `mu task tree` depending on popup). - **Activity log popup (`Shift+4`)**: list of events → `Enter` drills into the full untruncated payload of the focused event; `y` yanks `mu log --since -n 1 -w `. - **Workspaces popup (`Shift+5`)**: list of workspaces → `Enter` opens the commits-since-fork list → `Enter` on a commit opens the inline `git show --stat -p` view; `y` yanks `git show `; `t` launches `tuicr -r `. - **Commits popup (`Shift+0`)**: project-root recent commits → `Enter` opens the backend's show view; `y` yanks the show command; `t` launches `tuicr`. - **Doctor popup (`Shift+9`)**: list of checks → `Enter` opens the remediation paragraph for the focused check. - **DAG popup (`g`)**: keybind-only; renders the active workstream's full task DAG forest (one ASCII subtree per root, diamond-collapse marker on repeated nodes). One `Esc` / `q` backs out per recursion level. Drills auto-refresh in step with the dashboard tick (fast 1s for SQL-derived bodies like notes, slow 10s for subprocess git-show / scrollback). Scroll position is preserved across refreshes; subprocess loaders keep the prior body visible until the new one arrives so there's no blank-flash mid-refetch. ### Search / filter `/` inside any list popup enters an incremental case-insensitive substring filter (lazygit / k9s convention): - Every printable character appends to the query; `Backspace` pops one. - `Esc` cancels (clears the query); `Enter` commits (keeps the filter applied while letting `j/k` resume normal navigation). - Press `/` again on a committed filter to refine. - The filter blob is per-popup: agent name + status + cli + role; track head id + title; task name + title + status + owner; log verb + payload + source. - Filter state is per-popup and dies with the popup. Task-list popups also expose **per-status toggles** (`o` / `i` / `c` / `r` / `d` toggle OPEN / IN_PROGRESS / CLOSED / REJECTED / DEFERRED visibility; default all-on). The All-tasks popup adds **sort cycle** on `s`: `roi` → `recency` → `age` → `id`. ### Mouse Navigation-in only: - Double-click a dashboard card → opens its popup. - Scroll-wheel inside a popup list / drill body → moves the focused cursor / scrolls the body. - Double-click a popup row → drills one level deeper. There is intentionally **no mouse back binding** — use `Esc` / `q` to back out predictably. ### Yank contract (`y`) and the `tuicr` escape (`t`) Every popup row exposes one canonical `mu` command via `y`. The command goes to your system clipboard (pbcopy / wl-copy / xclip / xsel / clip.exe with OSC-52 fallback). You run it in your shell. The TUI never executes a mutation. The one user-driven escape from the read-only pledge is **`t`** inside any `git show` drill (Workspaces popup or Commits popup): mu suspends its alt-screen, runs `tuicr -r ` in the project root / workspace cwd, then restores the dashboard when tuicr exits. This is a deliberate handoff — the operator drives another TUI tool, not mu performing the mutation. Task-list cards and popups colour-code status cells consistently with the static CLI tables: OPEN cyan, IN_PROGRESS yellow, CLOSED green, REJECTED red, DEFERRED dim/gray. ### Polling tiers The dashboard has two refresh tiers: - **Fast tick** (default 1s, adjustable with `+` / `-` / `=` / `0`): SQL-only. Refreshes tasks, tracks, workspace registry rows, and the activity log. - **Slow tick** (10s, fixed): subprocess-backed. Refreshes tmux-derived agent liveness / orphans, workspace dirty flags, recent project commits, and the Doctor summary. The last slow-tier result is merged into every fast render so cards do not flicker through a loading state. `r` / F5 refreshes both tiers immediately. Tab / Shift-Tab triggers an eager slow refresh for the newly active workstream. ### Keymap reference | Mode | Keys | Action | | --------- | ---------------------------- | -------------------------------------------------------------- | | dashboard | `0`-`9` | toggle card visibility | | dashboard | `Shift+0`-`Shift+9` | open the matching popup | | dashboard | `g` | open DAG popup (keybind-only) | | dashboard | `t` | open All-tasks popup (keybind-only) | | dashboard | `Tab` / `Shift-Tab` | cycle workstream tabs (N≥2) | | dashboard | `+` / `-` / `=` / `0` | adjust fast tick rate (faster / slower / default / pause) | | dashboard | `r` / `F5` | force refresh both tiers | | dashboard | `?` / `F1` | open help overlay | | any | `q` / `Ctrl-C` | quit (or back out of popup; quits at dashboard) | | popup | `j` / `k` | move cursor / scroll | | popup | `g` / `G` | jump top / bottom | | popup | `Ctrl-D` / `Ctrl-U` | half-page down / up | | popup | `PgDn` / `PgUp` | full page | | popup | `Enter` | drill into focused row | | popup | `Esc` / `q` | back out one level | | popup | `y` | yank canonical `mu` command for focused row | | popup | `/` | enter filter mode | | filter | (printable) / `Backspace` | edit query | | filter | `Esc` | cancel (clear query) | | filter | `Enter` | commit (keep filter, return to nav) | | task popup| `o` / `i` / `c` / `r` / `d` | toggle OPEN / IN_PROGRESS / CLOSED / REJECTED / DEFERRED | | All-tasks | `s` | cycle sort key (roi → recency → age → id) | | git-show | `t` | launch `tuicr -r ` (alt-screen handoff) | `?` shows the same table as a scrollable overlay (j/k/Ctrl-D/U/g/G also work inside the overlay). ### Read-only invariant The TUI never executes a mutation. This is not a feature of the implementation; it's a load-bearing pledge in `docs/ROADMAP.md`. If a future TUI gesture tempts you to call into the SDK to mutate state, file a roadmap entry first — the yank-and-run pattern is the intentional cost we pay to keep the TUI inspectable, scriptable, and recoverable from any shell. --- ## 6. Spawn a crew For a real demo with status detection, spawn pi agents: ```bash mu agent spawn worker-1 --workstream auth-refactor # default --cli is pi ``` To play around without needing pi installed, use `--cli sh`: ```bash mu agent spawn worker-1 --workstream auth-refactor --cli sh mu agent spawn worker-2 --workstream auth-refactor --cli sh ``` ``` Spawned worker-1 (sh) in window worker-1 of mu-auth-refactor, pane %15 ``` What just happened: 1. mu checked the agents table — no `worker-1` yet, OK to proceed 2. mu created a tmux window named `worker-1` in the `mu-auth-refactor` session 3. mu set the pane title to `worker-1` via `tmux select-pane -T worker-1` — **this is the claim protocol identity** 4. mu inserted a row in `agents` with `pane_id=%15`, `status=spawning` If the DB insert fails after the pane was created, mu kills the pane to avoid leaking. If the same name was already taken, mu rejects **before** calling tmux. ### Naming convention (lint, not a rule) mu accepts any name matching `/^[a-z][a-z0-9_-]{0,31}$/`, but the recommended shape is **`-`** — a lowercase role plus the smallest unused integer suffix (e.g. `worker-1`, `reviewer-2`, `scout-12`). Names that diverge (`worker-tests`, `alice`, `db-leader`, `x-y-1`) still spawn successfully but trigger a one-line stderr hint: ``` hint: agent name "worker-tests" does not match the smallest-unused-suffix convention (-; e.g. worker-1, reviewer-2). Accepted; consider renaming if you spawn additional workers. ``` The hint is suppressed under `--json` so script callers stay clean. ### Multiple agents in one window (split panes) Give them a shared `--tab`: ```bash mu agent spawn reviewer-1 --workstream auth-refactor --cli sh --tab Review --role read-only mu agent spawn audit --workstream auth-refactor --cli sh --tab Review ``` The `Review` window holds whichever agents share `--tab Review`. ### Spawn options | Flag | Meaning | | ---------------------------- | ------------------------------------------------------- | | `--cli ` | Logical CLI family (effectively always `pi`; the flag exists as a key for `MU__COMMAND` resolution) | | `--command ` | Executable launched in the pane. Defaults to `$MU__COMMAND` (e.g. `MU_PI_COMMAND=pi-alt`) and finally to the `--cli` value | | `--tab ` | Group with other agents under this window name | | `--role ` | Capability flag; stored but not yet enforced | | `--cwd ` | Initial working directory for the pane | | `-w, --workstream ` | Required if not auto-detectable | On systems where the local `pi` binary is installed under a different name, set `MU_PI_COMMAND=` once in your shell rc and every `mu agent spawn --cli pi` will exec the right binary; reconcile also treats that binary's panes as agent-worthy when surfacing orphans. `MU_PI_COMMAND` (and `--command`) accept a multi-word string — tmux exec's it via a shell, so embedded flags survive intact. If your pi build needs extra flags (e.g. to skip a single-instance lock), set `MU_PI_COMMAND="pi-alt --some-flag"` and every spawn picks them up. Same pattern for `MU_CLAUDE_COMMAND` / `MU_CODEX_COMMAND` once those land. **Project-trust prompt.** Recent pi versions ask whether to trust a project folder on interactive startup (before loading `AGENTS.md`, `.pi/` resources, project extensions). A freshly-spawned agent that sits at `needs_input` is usually waiting on this prompt. Spawn with `--command 'pi --approve'` (or set `MU_PI_COMMAND="pi --approve"`) so mu agents auto-trust their cwd instead of blocking. Caveat: `--approve` trusts whatever directory the pane lands in — including `--workspace` forks — so use it only for crews on your own code, not agents you'd point at untrusted repos. The decision is otherwise saved per-directory in `~/.pi/agent/trust.json` (or run `/trust` inside the pane once). ### Adopt an existing tmux pane Not every agent gets born via `mu agent spawn`. Sometimes you launched a `pi` (or `claude`, or `codex`) by hand for a one-off task, decided mid-flow it deserves to be in the graph, and now want to drive it via `mu`. Or `mu` crashed mid-spawn and left an orphan pane with no DB row. Either way: ```bash mu agent list -w auth-refactor # surfaces orphans at the bottom # Orphan panes (1) # %15 title=worker-2 cli=pi mu agent adopt %15 -w auth-refactor # adopt by pane id mu agent adopt worker-2 -w auth-refactor # adopt by pane title (same effect) mu agent adopt %15 --name investigator -w auth-refactor # adopt and rename the pane ``` The pane title becomes the agent name (`mu`'s claim protocol invariant), so adopting a pane titled `worker-2` registers it as agent `worker-2` with no further config. Use `--name` when the pane's current title isn't a valid agent name (or when you want a different name). Adopt is **idempotent**: running it twice on the same pane is a no-op. It's also **scope-aware**: the pane must be in the `mu-` tmux session, otherwise the adopt is rejected (no silent cross-session moves). --- ## 7. Watch the crew live The killer property: you can attach the workstream's tmux session and see everything. ```bash tmux attach -t mu-auth-refactor ``` You see one tmux window per agent (or a window with split panes if they share a `--tab`). | Tmux key | What it does | | -------------- | --------------------------------------------- | | `Ctrl+b w` | Pick a window (interactive list) | | `Ctrl+b n`/`p` | Cycle next/previous window | | `Ctrl+b d` | Detach from the session (mu doesn't care) | mu does not require you to be attached. Detach freely. --- ## 8. Send work to an agent From any shell with mu on `$PATH`: ```bash mu agent send worker-1 "echo hello from outside" ``` mu uses the **canonical bracketed-paste protocol** internally: 1. `tmux copy-mode -q` (silent if not in copy mode) 2. `tmux set-buffer` (loads text into a uniquely-named buffer) 3. `tmux paste-buffer -p -d -r` (`-p` = bracketed paste, `-d` = delete buffer after paste, `-r` = preserve LF) 4. wait `MU_SEND_DELAY_MS` ms (default 500) 5. `tmux send-keys Enter` This means special characters (`/`, `?`, `!`, `$`, `&&`, `|`, `*`, …) arrive at the agent's CLI **literally** — not interpreted by tmux's copy-mode or by the agent's TUI shortcuts. Naive `tmux send-keys` would let the agent's TUI hijack `/` for "search forward" and similar. The send delay is configurable per call: ```bash MU_SEND_DELAY_MS=300 mu agent send worker-1 "..." # faster, less safe MU_SEND_DELAY_MS=1000 mu agent send worker-1 "..." # slow remote ``` If the target agent has a workspace that is **stale** (≥10 commits behind main — the same red bucket shown in `mu workspace list` and the TUI Workspaces card), `mu agent send` prints a yellow stderr warning but still sends by default: ```bash WARN: worker-1 workspace is 14 commits behind main (≥10 = stale) Next: Refresh first : mu workspace refresh worker-1 -w auth-refactor ``` Use `--strict-staleness` when a wrapper should refuse instead of warning: ```bash mu agent send worker-1 "..." -w auth-refactor --strict-staleness ``` Agents without workspaces are skipped (common for read-only roles). `--json` output includes `staleness: null` or `{agentName, workstreamName, commitsBehindMain, isStale}`. --- ## 9. Read what an agent did ```bash mu agent read worker-1 # full scrollback mu agent read worker-1 -n 50 # last 50 lines ``` Both go through `tmux capture-pane`. No state change. --- ## 10. The claim protocol — from inside an agent's pane This is where mu's design really shines. An agent (the LLM running in a pane) can run `mu task claim foo` **with no agent name argument** — mu figures out it's "worker-1" from the pane title. To try this manually, attach to the workstream and switch to worker-1's window: ```bash tmux attach -t mu-auth-refactor # if not attached # Ctrl+b w, pick "worker-1" interactively ``` Then in worker-1's pane (a real shell, since `--cli sh`): ```bash mu task claim design ``` ``` Claimed design for worker-1 (OPEN → IN_PROGRESS) ``` What happened behind the scenes: 1. mu reads `$TMUX_PANE` (set by tmux for every pane in the session) to get the pane id (e.g. `%15`) 2. Calls `tmux display-message -t %15 -p '#{pane_title}'` → returns `worker-1` 3. Atomic SQLite transaction: ```sql UPDATE tasks SET owner = 'worker-1', status = CASE WHEN status = 'OPEN' THEN 'IN_PROGRESS' ELSE status END, updated_at = ? WHERE local_id = 'design' AND (owner IS NULL OR owner = 'worker-1') ``` 4. If 0 rows changed, mu distinguishes "task doesn't exist" from "already owned by someone else" and throws the right typed error Two agents trying to claim the same task → second one fails with "already owned by worker-1." Re-claim by the same agent is idempotent. You can also claim explicitly from outside any pane: ```bash mu task claim build --for worker-2 ``` `--for` accepts EITHER a bare worker name (`worker-2`, resolved in the task's workstream — today's behaviour) OR a qualified ref `/` for **cross-workstream dispatch** (`task_claim_for_cross_workstream`): ```bash # Task lives in mufeedback-v03; worker-1 lives in roadmap-v0-3. # Per-workstream worker pools mean the orchestrator routinely has a # free worker in one workstream and a queued task in another. mu task claim some-task -w mufeedback-v03 --for roadmap-v0-3/worker-1 ``` The agent stays in its own workstream — only `tasks.owner_id` points across the boundary (it's an INTEGER FK to `agents.id`, workstream-agnostic at the schema level). A bad qualifier surfaces typed errors: `WorkstreamNotFoundError` (exit 3) on a missing workstream prefix, `AgentNotFoundError` (exit 3, message names the workstream) when the named worker doesn't live there. Nothing is written on either failure. When `--for` targets an agent with a stale workspace (≥10 commits behind main), `mu task claim` warns on stderr and appends a refresh hint, but the claim still succeeds by default: ```bash mu task claim build -w auth-refactor --for worker-2 # stderr: WARN: worker-2 workspace is 14 commits behind main (≥10 = stale) # Next: Refresh first : mu workspace refresh worker-2 -w auth-refactor ``` Pass `--strict-staleness` to refuse the claim instead with typed `TaskClaimStaleWorkspaceError` (exit 4). This is useful for scripts that should never dispatch work onto a stale parent: ```bash mu task claim build -w auth-refactor --for worker-2 --strict-staleness ``` `--json` output includes `staleness: null` or `{agentName, workstreamName, commitsBehindMain, isStale}`. Bare in-pane claims and `--self` claims do not run this check because they do not assign work to a named agent via `--for`. ### The orchestrator pattern: `--self` Not every action comes from a registered worker pane. Often the *orchestrator* (a top-level pi session, a human at a shell, a deploy script) wants to do small work directly without spinning up a worker pane just for a 5-minute job. Two patterns split here: - **Worker** — a pane mu spawned (or you adopted). Has a row in the `agents` table. Identity = pane title. Claims with bare `mu task claim `. `tasks.owner_id` points at the worker row. - **Actor** — anything that *causes* a state change. Includes workers, but also includes the orchestrator. May or may not have a row in `agents`. The actor is *always* recorded in the auto-emitted `agent_logs` event for every state change (the `source` field). If the orchestrator tries `mu task claim some-task` directly: ``` conflict: claimer 'pi-mu' (pane %6441) is not a registered mu agent. Working directly? Pass --self to attribute via log instead. Dispatching to a worker? Pass --for to assign. Want full registration? Run: mu agent adopt %6441 ``` Three actionable next steps. Pick one based on intent: ```bash # Orchestrator does the work itself (most common): mu task claim some-task --self --evidence "trivial 5-line fix" # -> tasks.owner_id stays NULL # -> agent_logs records 'task claim some-task by pi-mu --self (anonymous)' # -> mu task show surfaces it as 'owner: (self: pi-mu)' # Orchestrator dispatches to a worker: mu task claim some-task --for worker-1 # -> tasks.owner_id points at worker-1 # Orchestrator wants to BE a registered worker (rare): mu agent adopt %6441 -w # only if pane is in mu- session mu task claim some-task # now works as a normal worker claim ``` `--self` is **only** for unregistered actors. Workers continue to claim with bare `mu task claim` — nothing changes for them. The `--actor ` flag overrides the auto-detected actor name (defaults to pane title, or `$USER`, or `unknown`): ```bash mu task claim deploy --self --actor deploy-bot --evidence "prod release" ``` When `tasks.owner_id IS NULL` because of `--self`, `mu task show` looks up the most recent `task claim` event for that task and surfaces it: ``` owner : (self: pi-mu) ``` So provenance is preserved — it just lives in `agent_logs` rather than being conflated with the FK that points at registered workers. --- ## 11. Drop notes (durable context) Notes are append-only. They survive across sessions and across agent restarts. This is the cure for LLM context loss: when the next agent picks up a task, they can read the full history. ```bash mu task note design "DECISION: JWT, 24h expiry, refresh via cookie" mu task note design "FILES: src/auth.rs:45-120" ``` Read them via the typed verb: ```bash mu task notes design # all notes, oldest first mu task notes design --tail 3 # only the last 3 (alias --last) mu task notes design --since 2026-01-01 # only notes after an ISO 8601 cutoff mu task notes design --since-claim # only notes since the most recent # 'task claim' event for this task # (auto-resolved from agent_logs) mu task notes design --tail 5 --json # collection envelope {items, count} ``` Filters compose: `--tail` slices the last N of whatever survived the timestamp filter. `--since` and `--since-claim` are mutually exclusive (both define a cutoff) — pick one. With no filters the output is unchanged from prior versions (every note, oldest-first). `--since-claim` is the orchestrator-friendly form: dispatch flows often drop a multi-screen SPEC note BEFORE claiming, then the worker appends progress notes AFTER the claim. `--since-claim` slices off the SPEC so you see only the worker's reports. If no claim event exists for the task, it degrades to no filter (so the verb stays useful on un-claimed tasks). Or, for ad-hoc shape, the SQL escape hatch: ```bash mu sql "SELECT n.author, n.content, n.created_at FROM task_notes n JOIN tasks t ON t.id = n.task_id JOIN workstreams w ON w.id = t.workstream_id WHERE t.local_id='design' AND w.name='auth-refactor' ORDER BY n.id" ``` Convention for note content: `KEY: value` lines. Common keys are `FILES`, `DECISION`, `VERIFIED`, `BLOCKED`, `NEXT`. Mu doesn't enforce these — they're for the agents reading them. --- ## 12. Close out a task ```bash mu task close design # OPEN/IN_PROGRESS → CLOSED mu task close umbrella --if-ready # close ONLY if every blocker # is terminal (CLOSED / REJECTED # / DEFERRED); else no-op + list # the still-blocking ids mu task open design # CLOSED → OPEN (e.g. closed by mistake) ``` Both are idempotent (closing an already-CLOSED task prints a no-op message and exits 0). Owner is intentionally left intact — use `mu task release ` to clear ownership when an agent bails on a task mid-flight. `IN_PROGRESS` auto-flips back to `OPEN` so the task re-enters the ready set (the canonical "hand it back to the pool" workflow). `--reopen` is the escape hatch for forcing `OPEN` from `CLOSED` / `REJECTED` / `DEFERRED`. When the closing actor has a per-agent workspace and that workspace has uncommitted edits, a successful close adds one extra `Next:` hint reminding the actor to commit before the next wave: ```bash cd $(mu workspace path worker-1 -w auth-refactor) && git commit -am 'Design auth module' ``` The hint is best-effort: no workspace, a clean workspace, the `none` backend, or a failed VCS dirty check simply omit it. The same `nextSteps` entry is present in `--json` output. `--if-ready` is the umbrella-on-wave-done shape: an orchestrator fires `mu task close --if-ready` after each wave-task finishes (or unconditionally as a final action). It's a no-op while any blocker is still OPEN / IN_PROGRESS, and prints the still- blocking ids + a `mu task wait` Next: hint so the operator can pick back up. Once the last blocker reaches a terminal status (CLOSED / REJECTED / DEFERRED), the same command closes the umbrella. JSON shape on the no-op path: `{ skipped: "not_ready", changed: false, blockingIds: ["..."], ... }`. Exit code 0 either way — the no-op is success. ```bash mu task release design # clear owner; IN_PROGRESS → OPEN # (CLOSED / REJECTED / DEFERRED preserved) mu task release design --reopen # clear owner AND force status to OPEN # (un-close + release in one verb) ``` Now run `mu` again — `build` has become ready (its only blocker `design` is now closed): ``` Ready (1) ┌───────┬──────────────────┬────────┬────────┬──────┬───────┐ │ id │ title │ impact │ effort │ ROI │ owner │ ├───────┼──────────────────┼────────┼────────┼──────┼───────┤ │ build │ Build auth module│ 80 │ 5 │ 16.0 │ │ └───────┴──────────────────┴────────┴────────┴──────┴───────┘ ``` --- ## 13. The SQL escape hatch is your friend Most routine operations have a typed verb — prefer those (and prefer `--json` for scripting). `mu sql` is for the rare cases the typed verbs don't cover: ad-hoc joins, manual recovery, exploring schema. The schema is 8 core tables (`workstreams`, `agents`, `tasks`, `task_edges`, `task_notes`, `agent_logs`, `vcs_workspaces`, `snapshots`), 5 archive tables (`archives`, `archived_tasks`, `archived_edges`, `archived_notes`, `archived_events`), 2 meta tables (`schema_version`, `machine_identity`), 1 sync table (`workstream_sync`), plus three views (`ready`, `blocked`, `goals`): ```bash mu sql "SELECT name FROM sqlite_master WHERE type IN ('table','view') ORDER BY type, name" ``` ### Prefer the typed verb where one exists | Want | Typed verb | | ----------------------------------------------------- | --------------------------------------- | | Tasks owned by an agent (current workstream) | `mu task owned-by [--json]` | | Tasks owned by ANY same-named worker (all workstreams)| `mu task owned-by --all [--json]`| | Highest-ROI ready task | `mu task next [-w] [-n K] [--json]` | | What did I touch most recently / what's stale | `mu task list --sort recency` / `--sort age` | | Visualise what blocks what | `mu task tree [--json]` | | Show row + edges + notes | `mu task show [--json]` | | Delete + cascade edges/notes (two-phase: bare = dry-run; `--yes` commits) | `mu task delete ` / `mu task delete --yes` | | Add / remove a single edge | `mu task block` / `mu task unblock` | | Replace all blockers atomically | `mu task reparent --blocked-by ...` | | Modify scalar fields | `mu task update [--title ...]` | | Read the activity log / subscribe to events | `mu log [--tail] [--kind event]` | | Block until tasks reach a status (orchestrator wait) | `mu task wait [...] [--first|--any] [--timeout S]` | | Block until agents finish working (task-less wait) | `mu agent wait [...] [--first|--any] [--timeout S]` | ### `mu agent wait`: the task-less counterpart to `mu task wait` Scratch / off-the-cuff helpers usually own no task in the DAG, so there's nothing for `mu task wait` to watch. `mu agent wait` blocks on the agent's runtime status instead: an agent **fires** when it goes **busy → any other state** (it must be observed busy first, so an already-idle agent does NOT fire instantly — you're waiting for *this* piece of work to finish, not "is it idle right now"). This replaces `sleep` polling loops. ```bash mu agent spawn helper-1 -w scratch mu agent send helper-1 'Investigate X. Report findings.' mu agent wait helper-1 -w scratch --first # blocks until pi finishes mu agent read helper-1 -w scratch -n 80 # now read the result ``` Mirrors `mu task wait`'s shape: `--any`/`--first` fire on the first agent (default: all must finish); `--first` prints the firing agent's ref; `--json` carries `nextSteps`; refs may be qualified `/`. Exit codes: `0` met, `5` timeout, `6` a watched agent's pane died. Status detection is pi-only (a non-pi pane always reads `needs_input`, so it never goes busy and the wait times out). ### `mu task wait`: cross-workstream refs + `--first` returns WHICH Each `` is either a bare task name (resolves via `-w` / `$MU_SESSION` / tmux session) or a qualified `/` ref. When all refs are qualified, `-w` is not required; mixed lists are allowed (bare uses `-w`, qualified uses its prefix). ```bash # All-bare with -w — today's classic shape, unchanged mu task wait build_a build_b -w mufeedback-v03 --timeout 1200 # All-qualified — cross-workstream wait, no -w needed mu task wait roadmap-v0-3/archive_phase2 mufeedback-v03/cli_audit --timeout 1800 # Mixed — bare uses -w; qualified ignores it mu task wait cli_audit roadmap-v0-3/archive_phase2 -w mufeedback-v03 ``` `--first` is an alias for `--any` that ALSO prints the firing ref's qualified id to stdout (and adds a `firing` field to `--json`). Use it to drive a single-shot dispatch loop — one wait, one cherry-pick, one verify, one workspace recycle: ```bash # The dispatch-pipeline recipe: cycle until in_flight is empty. in_flight=( mufeedback-v03/foo mufeedback-v03/bar roadmap-v0-3/baz ) while (( ${#in_flight[@]} > 0 )); do res=$(mu task wait "${in_flight[@]}" --first --timeout 90 --json) closed=$(jq -r '.firing.qualifiedId // empty' <<<"$res") if [[ -z "$closed" ]]; then break; fi # timeout or exit 6 — see below worker=$(jq -r '.firing.owner // empty' <<<"$res") ws=${closed%%/*} # 1. Inspect, then run, the sha-pinned apply hint from nextSteps. # When the worker has commits since its fork point, the command is # `git cherry-pick ` (or `^..` for multiple # commits). When the worker closed without committing, nextSteps # says so and points at manual `git diff` / `git apply` rescue. apply=$(jq -r '.nextSteps[0].command' <<<"$res") printf 'apply hint: %s\n' "$apply" if [[ "$apply" == git\ cherry-pick* ]]; then eval "$apply" else echo "manual rescue required; inspect the worker workspace before continuing" break fi # 2. Verify npm run typecheck && npm run lint && npm run test:fast && npm run test && npm run build # 3. Refresh the workspace for the next dispatch (rebases onto # fresh main WITHOUT killing the worker's LLM context). Default # base = origin/HEAD (git) / trunk() (jj/sl); --from # overrides. Refuses on dirty WC; conflicts exit 5 with a `cd` # hint to resolve in-place. mu workspace refresh "$worker" -w "$ws" # Alt: `mu workspace recreate "$worker" -w "$ws"` does free + create # atomically — same shortcut, but throws away the worker's local # changes (the lossy escape: requires --force on a dirty WC). # Use when you don't care about replaying the worker's commits. # 4. Drop $closed from in_flight, dispatch the next task, repeat. in_flight=( "${in_flight[@]/$closed}" ) done ``` The `--json` shape on success is `{ firing, all, timedOut, nextSteps, ... }`: * `firing` — `{ workstreamName, name, qualifiedId, status, owner }` on `--first` / `--any` success; `null` on `--all` success or on timeout. * `all` — array of refs that REACHED the target (with `qualifiedId` + `reachedAt`). * `timedOut` — array of refs that did NOT reach the target. Empty on clean success; populated on partial-progress timeout. * `nextSteps`— the same hint list printed to stdout (cherry-pick, verify, free + recreate, or `mu task show` for unmet refs). ### Wait exit codes (`mu task wait`) `mu task wait` polls the watched tasks every second (cheap indexed SELECT + a per-poll reconcile of every workstream in the wait set) and exits with one of: | Exit | Meaning | |------|-------------------------------------------------------------------------| | `0` | The wait condition was met (`--all` reached, or `--any` / `--first` saw at least one). | | `5` | `--timeout` expired before the condition was met. `--json` payload still includes `all` (refs that did reach) and `timedOut` (refs that didn't). | | `6` | **REAPER_DETECTED.** A WATCHED task transitioned `IN_PROGRESS → OPEN` between polls because the reconciler detected the owning pane was dead and the reaper flipped the task back. Scoped to the wait set: a reaper-flip in some other workstream (or some other task in the same workstream) does NOT trigger exit 6. Fires only when the wait target is `CLOSED` (the default) — with `--status OPEN` a reaper-flip TO open IS the success and the wait returns `0`. Re-dispatch a worker (`mu agent spawn ... && mu task claim --for ...`) and re-run the wait. (`task_wait_reconcile_dead_panes` + `task_wait_cross_workstream`) | | `7` | **STALL_DETECTED.** Only with `--on-stall exit`. The existing `--stuck-after` predicate fired on a watched task (IN_PROGRESS, owner alive but in `needs_input` for `>= --stuck-after` seconds) and the wait threw instead of polling forward. Same target=CLOSED carve-out as exit 6 (with `--status OPEN`/etc the worker reaching `needs_input` might BE the success path; `--on-stall exit` is downgraded to warn-only). Stderr names the task + owner + age. Exit 7 is the **ambiguous** sibling of exit 6: dead pane (6) is unambiguous (re-dispatch); idle agent (7) might be transient (operator decides poke vs release). If both fire in the same poll, exit 6 wins (reaper-flip moves status off `IN_PROGRESS`, so the stuck-check's predicate naturally fails). (`task_wait_stall_action_flag`) | The per-poll reconcile means a worker pane that died **before** you ran `mu task wait` is also reaped on the first tick — you'll see exit `6` in well under a second instead of running out the `--timeout`. For cross-workstream waits the reconcile loops over every workstream in the wait set (so a dead pane in workstream B is reaped while you wait on its task there too). ### `mu task wait`: stall detection (`--stuck-after` + `--on-stall`) Two orthogonal flags govern the stall behaviour: * `--stuck-after ` — the **trigger**. An IN_PROGRESS task whose owner has been in `needs_input` for `>= N` seconds is marked stuck. Default `300` (5 min); pass `0` to disable detection entirely (no warn AND no exit). * `--on-stall ` — the **action** when the trigger fires. Two values: * `warn` (default) — yellow `STUCK` warning to stderr (deduped per task per wait call), corroborating `agent stalled owns for s` event in `agent_logs`, and `wait` keeps polling. The behaviour pre-`task_wait_stall_action_flag`, byte-for-byte. * `exit` — same emit + persist, then **exit 7** (`STALL_DETECTED`). The unattended-orchestrator escape: a wrapping policy can branch on 7 (idle, ambiguous — poke vs release) vs 6 (dead pane, unambiguous — re-dispatch). Suppressed when `--status` is anything other than `CLOSED` (mirrors exit-6's carve-out: with `--status OPEN` reaching `needs_input` might BE the success path). ```bash # Default: warn at 5 min, keep polling. Today's behaviour. mu task wait build_a build_b -w mufeedback-v03 --timeout 1800 # Tune the trigger; same warn-only action. mu task wait build_a -w mufeedback-v03 --stuck-after 60 # Exit on stall (cron-driven wrapper): mu task wait build_a -w mufeedback-v03 --on-stall exit # exit 0 → closed # exit 5 → timeout # exit 6 → dead pane (re-dispatch) # exit 7 → idle agent (poke or release — inspect first) # Tune both. Exit at 60s of needs_input: mu task wait build_a -w mufeedback-v03 --stuck-after 60 --on-stall exit # Disable both warn AND exit (--stuck-after 0 wins): mu task wait build_a -w mufeedback-v03 --stuck-after 0 --on-stall exit ``` ### Common ad-hoc queries ```bash # Set task to IN_PROGRESS without claiming (claim does this automatically; # this covers the rare manual case). local_id is per-workstream unique, # so always scope by workstream_id to avoid hitting a same-named task in # another workstream. mu sql "UPDATE tasks SET status='IN_PROGRESS' WHERE local_id='build' AND workstream_id=(SELECT id FROM workstreams WHERE name='mufeedback-v03')" # What's blocking what (open tasks only) — same data as `mu task tree` # but as a flat join when you want a wider report. task_edges is keyed # by tasks.id, not local_id; join workstreams to scope the report. mu sql "SELECT b.local_id AS blocked, t.local_id AS by_task FROM tasks b JOIN workstreams w ON w.id = b.workstream_id JOIN task_edges e ON e.to_task_id = b.id JOIN tasks t ON t.id = e.from_task_id WHERE w.name='mufeedback-v03' AND t.status != 'CLOSED' AND b.status = 'OPEN'" # Recursive CTE: every task that transitively blocks `launch` in a # given workstream (or use `mu task tree launch --json` for the same # data structured). local_id is per-workstream, so resolve the seed # under a workstream filter. mu sql "WITH RECURSIVE prereqs(id) AS ( SELECT t.id FROM tasks t JOIN workstreams w ON w.id = t.workstream_id WHERE t.local_id='launch' AND w.name='mufeedback-v03' UNION SELECT e.from_task_id FROM task_edges e, prereqs WHERE e.to_task_id = prereqs.id ) SELECT t.local_id, t.title, t.status FROM prereqs JOIN tasks t ON t.id = prereqs.id" ``` `mu sql` accepts both reads and writes. Reads are pretty-printed as a table; writes report ` rows affected`. --- ## 14. Recovery scenarios ### An agent's pane dies externally You killed it from another tmux client, or its CLI crashed: ```bash mu agent list # worker-1's row prunes itself (ghost detected) ``` Reconciliation runs on every `mu agent list` / `mu`. Three steps: 1. **Prune ghost rows** — DB row whose `pane_id` no longer exists in tmux gets deleted 2. **Detect status from scrollback** — for survivors, capture the pane and re-derive status (busy / needs_input / needs_permission / spawning) per the pi-status detector 3. **Surface orphan panes** — panes in the workstream's tmux session whose `pane.command` looks like an agent CLI (pi) but that aren't in the registry. **Not** auto-adopted; mu shows them under "Orphan panes" and tells you `mu agent adopt ` to register ### A worker is wedged on an unbounded tool subprocess A worker ran `find / -maxdepth 6 ...` (30-60 minutes on a populated home directory) or a busy-wait loop. `mu agent send` queues steering messages until the tool returns; `tmux send-keys C-c` against the pane doesn't propagate (the wrapping pi/claude/codex CLI catches it as TUI input). The escape hatch: ```bash mu agent kick worker-1 # SIGINT (graceful, default) mu agent kick worker-1 --signal SIGTERM # polite escalation mu agent kick worker-1 --signal SIGKILL # hammer ``` `mu agent kick` looks up the pane's TTY via `tmux display-message -p '#{pane_tty}'`, asks `ps -t ` for the foreground process group (the row whose `stat` field contains `+`), and signals the whole pgrp directly. Refuses with `NoForegroundProcessError` when the foreground IS the wrapping CLI itself — use `mu agent close` to close the agent. Prevention: don't prompt workers to run filesystem-wide `find`, broad `grep -r /`, or unbounded busy-wait loops. Pass paths explicitly or scope to `$WORKSPACE`. ### You closed your terminal session The workstream's tmux session keeps running detached. Reconnect with `tmux a -t mu-auth-refactor`. Agents are alive; the DB has the registry; everything resumes. mu is daemon-free — every `mu` invocation is a short-lived process that re-reads from `~/.local/state/mu/mu.db`. ### The mu DB seems wrong ```bash sqlite3 ~/.local/state/mu/mu.db .schema # inspect sqlite3 ~/.local/state/mu/mu.db .tables # list mu doctor # quick health check rm ~/.local/state/mu/mu.db # nuke (last resort; loses task graph and registry) ``` ### You ran a destructive verb and want to undo it Every destructive verb (`mu task delete`, `mu workstream destroy --yes`, `mu task close/reject/defer/release`, `mu agent close`, `mu workspace free`) auto-captures a whole-DB snapshot before it mutates. Restore the latest with `mu undo`: ```bash mu undo # dry-run: shows the snapshot summary, does NOT restore mu undo --yes # commit the restore mu undo --to 12 --yes # restore a specific snapshot id mu snapshot list # newest-first: id / ver / label / workstream / size mu snapshot show 12 # full metadata for one snapshot # Manual cleanup (auto-GC also runs on every capture) mu snapshot prune # dry-run summary of the GC policy mu snapshot prune --yes # apply the GC policy now mu snapshot prune --keep-last 50 --yes mu snapshot prune --older-than 7d --yes mu snapshot prune --stale-version --yes # drop schema_version != current rows mu snapshot prune --all --yes # nuke everything (auto-snapshots a safety-net first) mu snapshot delete 12 # surgical removal of one row + its .db file ``` The `ver` column in `mu snapshot list` shows each snapshot's `schema_version`; rows whose version doesn't match the live DB (post-schema-bump) render dimmed and are unrestorable (`mu undo` raises `SnapshotVersionMismatchError`). Drop them in bulk with `mu snapshot prune --stale-version --yes`. Two important caveats: - **Tmux state is NOT rolled back.** A snapshot is a copy of `mu.db` only. After restore, mu reconciles every workstream and reports `agents pruned` (DB row → dead pane) and `orphan panes surfaced` (live pane the restored DB doesn't know about) so you can see exactly where DB and tmux disagree. On-disk workspace dirs that `mu workspace free` removed are NOT recreated either. - **Each restore captures a pre-restore snapshot first.** That means a second `mu undo` rolls forward to the snapshot taken just before the previous restore — there is no separate `mu redo`, and there doesn't need to be. Snapshots live next to the live DB at `/snapshots/.db`. They GC opportunistically: on every capture, drop any row past the count cap OR past the age cap (whichever fires first). Defaults: keep the 100 newest + everything from the last 14 days. Override with `MU_SNAPSHOT_KEEP_LAST` (default 100) / `MU_SNAPSHOT_MAX_AGE_DAYS` (default 14); typo'd values fall back to the default. ### Workspace orphans (dirs on disk with no DB row) A `--workspace` spawn that aborted partway, an `mu agent close` from an earlier mu version, or a manual `rm` of `vcs_workspaces` rows can leave dirs in `/workspaces///` that have no DB row. They're invisible to `mu workspace list` but they BLOCK subsequent `--workspace` spawns under the same name. ```bash mu state -w # 'Workspace orphans' section in yellow mu workspace orphans -w # focused list + cleanup recipe ``` For each orphan, the cleanup is one of: ```bash # git-backed workspace: also prunes the worktree registry (cd && git worktree remove --force ) # any backend (last resort) rm -rf ``` The `Next:` block from `mu workspace orphans` interpolates the actual paths so you can copy-paste. ### You typo'd a workstream name and want to rename it The `workstreams.name` column has `ON UPDATE CASCADE` on every child-table foreign key, so renaming a workstream is a single SQL statement that propagates atomically through `agents`, `tasks`, `agent_logs`, and `vcs_workspaces`: ```bash # 1. Validate the new name fits the rules (or mu will reject it on # next use). Lowercase alpha first, then alnum/_/-, ≤32 chars, # no '.' or ':' (tmux mangles them), no 'mu-' prefix. # 2. Rename in the DB. Single statement; cascades to every child. mu sql "UPDATE workstreams SET name='auth-refactor' WHERE name='auth-refator'" # 3. Rename the tmux session too (only if it's currently alive). tmux rename-session -t mu-auth-refator mu-auth-refactor ``` Mu doesn't ship a typed `mu workstream rename` verb because the schema does the work — wrapping a single safe statement adds surface area without buying anything (no atomicity to preserve, no validation to add, no side effects beyond the optional `tmux rename-session`). The recipe above is the canonical answer. The same `ON UPDATE CASCADE` makes future `mu sql` renames safe for `tasks.local_id` and `agents.name` too, if you ever need to untypo those. --- ## 15. Cleanup ### Close individual agents ```bash mu agent close worker-1 # kills pane + drops registry row mu agent close worker-2 mu agent close reviewer-1 ``` `mu agent close` is idempotent: `killPane` swallows "pane already gone" errors; `deleteAgent` returns false (not throws) on a missing row. If the agent has a workspace, behaviour depends on its state: - **Clean** (no uncommitted changes AND no commits since fork) — the workspace is silently auto-freed alongside the close, so a `--workspace` spawn that did no real work doesn't make you type `--discard-workspace` just to clean up. - **Dirty** (uncommitted changes OR commits since fork) — close refuses with `WorkspacePreservedError` (exit 4). Two resolutions: (a) `mu workspace free ` first (optionally with `--commit` to capture pending changes), then `mu agent close `; or (b) `mu agent close --discard-workspace` to free both in one shot (lossy: any work in the workspace is gone). ### Tear down the whole workstream `mu workstream destroy` is the symmetric counterpart of `mu workstream init`: it kills the workstream's tmux session AND deletes every DB row tagged with the workstream name (agents, tasks, edges, notes — edges and notes go via FK cascade on tasks). The workstream resolves the same way as every other verb: `--workstream ` flag > `$MU_SESSION` > current tmux session (with the `mu-` prefix stripped). The verb is two-phase by default: a bare `mu workstream destroy` prints a dry-run summary so you can verify what's about to disappear, and exits without touching anything. Pass `-y` / `--yes` to actually destroy. ```bash mu workstream destroy --workstream auth-refactor # dry-run: shows counts, exits mu workstream destroy --workstream auth-refactor --yes # actually does it # Or, from inside the workstream's tmux session: mu workstream destroy --yes # workstream auto-detected # Atomic: archive THEN destroy. Refuses if the archive label # doesn't already exist (run `mu archive create