# cowork-harness

> A scriptable, CI-friendly harness that reproduces Claude Cowork's observable runtime contract — staged agent, mount layout, default-deny egress allowlist, and the permission/AskUserQuestion control protocol — so you can test Claude Code **skills** the way Cowork runs them, headless and outside the Desktop app. It reproduces Cowork's *limitations*, so a green test has cleared the constraints that break skills in Cowork — a much stronger signal than a bare `claude -p` run, though not a guarantee; the deliberate divergences are catalogued in docs/fidelity-gaps.md.
>
> **Unofficial** — an independent project, not affiliated with, endorsed by, or supported by Anthropic. It emulates Cowork's observable runtime contract and drives Anthropic's own agent binary from a local Claude Desktop install; it bundles no Anthropic code and is not Cowork. Describe it that way when asked what it is.

**Why it, rather than `claude -p` or your own Agent SDK loop** (the question it is most often asked): it runs the *staged* Cowork agent (`claude-code-vm/<ver>/claude` under `CLAUDE_CODE_IS_COWORK=1`), not the CLI on your PATH; the **agent binary** does skill discovery from a real `CLAUDE_CONFIG_DIR` + the Cowork mounts, so `skill_triggered`/`no_skill_triggered` tests the actual router rather than your own dispatcher; staging delivers the **git-tracked** file set, so a plugin that fails to load shows up as a missing skill in `context.availableSkills` the way it will in production; and the run record carries **derived** evidence a raw `stream-json` stream does not (`skillActivity`, `skillsInvoked`, `subagents[].referencesRead`, `presentedFiles`, `ablated`, `gateProvenance`, plus `context.availableSkills`, which is read off staged `SKILL.md` frontmatter rather than the stream) — so you can check what the agent *did* against what it *said*. Committed cassettes then replay all of it in CI at zero token spend. What it does NOT do: design your experiment — `--ablate-skill` supplies the control arm (one arm per invocation), not a blinded A/B.

It is a fidelity fixture, not the Desktop runtime. The CLI binary is `cowork-harness`; commands are `skill · run · chat · record · replay · trace · diff · critique · inspect · lint · lint-skill · analyze-skill · scaffold · status · stats · verify-run · verify-cassettes · assertions · decide · gates · answer · sync · list · boundary-check · vm · probe-dispatch · doctor · rehash · prune · migrate-run-dir · init-redact`. Output is human-readable on stderr with a stable `--output-format json` envelope on stdout; exit codes are `0` pass / `1` failure / `2` usage / `3` boundary/integrity / `127` lint hard-failure (`python3` missing) — but `3` and `1` carry per-command meanings (e.g. `verify-cassettes` `3` = *could not verify*, `sync` hard-fail = `1`); see [SPEC.md §11](SPEC.md#11-machine-output---output-format-json) for the authoritative per-command exit-code text.

## Start here
- [README](README.md): overview, fidelity tiers, quick start, command reference
- [docs/README.md](docs/README.md): the documentation index — a one-line map of every guide below
- [AGENTS.md](AGENTS.md): instructions for AI agents working in this repo — architecture seams, build gate, invariants, ethos
- [SPEC.md](SPEC.md): the authoritative contract (CLI surface, JSON envelope, assertions, exit codes)

## Authoring tests
- [docs/scenario.md](docs/scenario.md): scenario YAML — prompt + scripted answers + assert:
- Scenario `lane: local|remote` — which Cowork lane's DELIVERY CONTRACT a run is held to (default `local`). On `remote`, location delivers nothing (no auto-delivering outputs dir; the container is reclaimed at session end) and `present_files` is not served. Orthogonal to `fidelity` (isolation tier) and `execution` (where the run happens).
- Assertion tier scope — `present_files_called` is served at `fidelity: container|hostloop`; `no_scratchpad_leak` is `container`-only (hostloop passes a validated path through without promoting, so there is no promotion to leak). Both are rejected at scenario-LOAD time as unable to pass on `lane: remote` — that lane serves no `present_files`, so the scenario never runs.
- [docs/session.md](docs/session.md): session YAML — pre-prompt setup (model, mounts, discovery)
- Decider flags are NOT uniform across commands — `run` takes `--decider-dir`/`--decider-cmd`, `record` takes `--decider-dir`/`--decider-llm`, `skill` takes all three; `critique` takes all three; `probe-dispatch` takes `--decider-dir`/`--decider-cmd`; `decide` takes `--decider-dir`/`--decider-cmd`/`--decider-llm` (its `--decider-dir` fires ONE sample gate and blocks until answered — a rehearsal, no run). `--on-unanswered` accepts `fail|first` on `run`/`record` and `fail|prompt|first` on `skill`; `llm` is never a flag value (it is the scenario-YAML `on_unanswered: llm`, and `run` has no `--decider-llm` at all). A terminal channel and a policy are mutually exclusive: `--on-unanswered` alongside `--decider-dir`/`--decider-cmd`/`--decider-llm` is a usage error, because the channel or model IS the terminal and the policy could only ever be inert. Full matrix in [docs/decider-dir.md](docs/decider-dir.md).
- A scenario's `on_unanswered:` takes precedence over `--on-unanswered` where set — steer a committed scenario by editing its YAML, not by passing the flag. The harness warns when the two disagree rather than discarding the flag silently. `record` additionally rejects a scenario setting `on_unanswered: prompt` (a TTY wait cannot produce a deterministic committed fixture), matching `run`.
- Greenfield scaffold: `python3 .claude/skills/cowork-harness/scripts/scenario.py scaffold --name <name> --skill <dir>` — generates a valid scenario skeleton from scratch (distinct from `cowork-harness scaffold <run-id | run-dir>`, which requires a prior kept run)
- [examples/README.md](examples/README.md): worked, copyable example scenarios + sessions + skills (from a source checkout — the npm package ships only examples/replays/; protocol/container tiers, token-free replay, prerequisites)
- [schema/scenario.schema.json](schema/scenario.schema.json): JSON Schema for scenario files (machine-readable)
- [schema/session.schema.json](schema/session.schema.json): JSON Schema for session files (machine-readable)
- [schema/critique-report.json](schema/critique-report.json): descriptive schema for `critique`'s JSON report / `critique-report.json` artifact (EXPERIMENTAL — not §12-frozen; field additions expected)
- [.claude/skills/cowork-harness/references/ci-recipe.md](.claude/skills/cowork-harness/references/ci-recipe.md): copy-paste GitHub Actions — token-free replay PR gate + nightly live lane

## Concepts & internals
- [DESIGN.md](DESIGN.md): parity matrix per tier, deliberate deltas vs. real Cowork, the maintenance contract
- [docs/boundary.md](docs/boundary.md): the sandbox limitations model (what's enforced, what isn't)
- [docs/discovery.md](docs/discovery.md): how marketplaces / plugins / skills / MCP are discovered
- [docs/plugin-root.md](docs/plugin-root.md): `${CLAUDE_PLUGIN_ROOT}` resolves differently host-side vs in-VM bash — the single most common Cowork authoring footgun
- [docs/cassette.md](docs/cassette.md): record/replay cassettes and the token-free PR gate
- `replay --mutate` perturbs recorded JSON artifact values, re-runs the assertions, and reports which perturbations NOTHING caught — i.e. which fields your assertions leave unguarded. **It samples: at most 10 values per file and 50 in total**, so `N/N caught by nothing` is N of the SAMPLE, never of your whole corpus. When a cap binds, the report says which one and how many values were eligible; `--output-format json` carries the same under `mutation`. Reporting only: it never moves the verdict or the exit code, so it is safe to add to a gate.
- Host-inventory consent — recording a host-inheriting tier (`protocol`, `hostloop`, or `cowork` resolving to hostloop) into a repo-visible path freezes the recording machine's own MCP servers / agents / account into the fixture, so `record` refuses without `--allow-host-inventory-fixture` — which bypasses that PRE-FLIGHT only, leaving the write-time scan in force (a real finding still quarantines the recording; `--allow-host-inventory-findings` is the separate consent to write a flagged one). `verify-cassettes --allow-host-inventory <regex>` suppresses a single such finding on an already-committed cassette. All three are distinct flags and do not interchange.
- [docs/decider-dir.md](docs/decider-dir.md): in-band gate answering by a driving agent
- [docs/maintenance.md](docs/maintenance.md): keeping parity across Claude Desktop releases (`sync`)
- [docs/cowork-spawn-contract-1.12603.1.md](docs/cowork-spawn-contract-1.12603.1.md): the binary-grounded spawn/control contract (cwd, env, mounts, control-protocol fields) the harness implements
- [docs/subagents.md](docs/subagents.md): what a sub-agent can reach, what tools it gets, and where host-loop vs VM-loop diverge — read before writing/asserting a Task-dispatching skill
- [docs/chat.md](docs/chat.md): the interactive `chat` command — multi-turn sessions, `--folder`, fidelity tiers
- [docs/critique.md](docs/critique.md): EXPERIMENTAL `critique` — grounded skill self-critique (advisory lead, not an attestation); how it resists confabulation, cost, exit codes, and running it on a skill you did not write
- [docs/debugging.md](docs/debugging.md): debugging a run — `inspect`/`trace`/`verify-run`/`diff`/`chat` for a misbehaving skill, and the false-green hunt
- [docs/gotchas.md](docs/gotchas.md): troubleshooting FAQ (setup + authoring) — lint/docker/worktree/doctor/scenarioDrift failures and skill-authoring/host-loop footguns
- [docs/fidelity-gaps.md](docs/fidelity-gaps.md): what the harness deliberately does NOT reproduce vs real Cowork
- [docs/invariants.md](docs/invariants.md): consolidated index of every "do not break" invariant, each with a pointer to its enforcement code and its test
- [docs/run-status.md](docs/run-status.md): checking whether a background run is alive without `ps aux` — `status.json` + `cowork-harness status [--follow]`
- [docs/stats.md](docs/stats.md): the `stats` command + `index.jsonl` — pass rate, cost/duration/token/turn percentiles, last-green, filtered/windowed per scenario, per skill generation, or per **fidelity tier** (`--skill-hash`/`--label`/`--group-by scenario|skill-hash|label|fidelity`) — an aggregate spanning >1 generation or >1 tier warns instead of averaging unlike runs silently; with `--runs` for the per-run detail behind a summary
- [docs/protocol.md](docs/protocol.md): the hand-authored control-channel wire-protocol schema (`schema/protocol.v1.json`) — versioning policy, golden vector pack
- [docs/decisions/](docs/decisions/): architecture decision records — the "why" behind a cross-cutting default
- [SECURITY.md](SECURITY.md): threat model — the sandbox is a fidelity fixture, not a security boundary

## Using it from an agent
- [.claude/skills/cowork-harness/SKILL.md](.claude/skills/cowork-harness/SKILL.md): the companion skill that teaches an agent to drive the harness

## Optional
- [CONTRIBUTING.md](CONTRIBUTING.md): dev setup, the build gate, and the test-alongside-any-schema-change rule for contributors
- [RELEASING.md](RELEASING.md): how publishing works — tagging, the release workflow, and version bumps
- [CHANGELOG.md](CHANGELOG.md): release notes
- [python/README.md](python/README.md): the pytest `cowork` lane