# Contributing

Thanks for helping make Cowork skill-testing reproducible.

For the full documentation map, see [docs/README.md](./docs/README.md). Read [AGENTS.md](./AGENTS.md) —
the canonical agent/architecture contract (architecture seams, invariants) — before changing code.

## Development setup

```bash
npm ci
npm run format:check  # prettier over src/ + test/ .ts — the most common first-pass CI red
npm run typecheck     # tsc --noEmit
npm test              # vitest
npm run build         # -> dist/
npm run ci            # typecheck + build + test — the core TS gate (CI's `build` + `test` jobs run more; see below)
```

Before pushing: `npm run format:check` and `npm run ci` (see RELEASING.md's release checklist for the full pre-tag list).

Node ≥ 22. The project is ESM TypeScript; no transpiler magic — `tsc` only.

Extra prerequisites for specific stages:

- **python3** — required for `cowork-harness lint` (the scenario linter shells out to `python3`; it hard-fails with exit `127` when `python3` is missing). PyYAML is **bundled** with the linter — no separate install.
- **Docker (arm64)** — required for `boundary-check` and the **L1 `container`** + `hostloop` fidelity tiers (the container sandbox + agent image). **`hostloop` additionally needs a staged native macOS agent binary** (its agent loop is a host process, not a container one), so Docker alone is not enough for it and a Linux box cannot run it — see README Prerequisites.
- **Lima (`limactl`, macOS arm64)** — required only for the **L2 `microvm`** tier and the `vm` commands; the guest runs on Apple Virtualization.framework (`vmType: vz`). `microvm` does **not** use Docker. (`cowork-harness doctor --tier microvm` checks for Lima, not Docker.)

CI Stage 1 (the `build` job in `.github/workflows/ci.yml`) does **not** invoke `npm run ci`. It runs the
gate steps individually — e.g. `check:versions`, `check:skill-doc-links`, `format:check`, `typecheck`, `build`, a CLI smoke
(`node dist/cli.js list`), three token-free `replay` gate fixtures, `verify-cassettes`, `lint`, and
source-guard checks — while the unit suite runs separately as the 4-shard `test` job (see `ci.yml` for
the authoritative list). Only `release.yml` calls `npm run ci`. For a copy-paste GitHub Actions setup,
see [ci-recipe.md](./.claude/skills/cowork-harness/references/ci-recipe.md).

> **Cutting a release?** See [RELEASING.md](./RELEASING.md) for the branch → PR → tag → publish flow.

## Project layout

```
src/
  cli.ts              command entry — thin wrapper over executeScenario (run / skill / sync / list / decide / …)
  types.ts            PlatformBaseline + Scenario zod schemas
  session.ts          SessionConfig schema + buildLaunchPlan
  baseline.ts         parity-baseline loader
  agent/              AgentSession — the stream-json control protocol (session.ts)
  decide/             Decider — answer policy (scripted / LLM / external-channel deciders)
  run/                Run — turn loop + RunRecord (run.ts); executeScenario (execute.ts); cassette replay
  runtime/            protocol (L0) / container (L1) / microvm (L2) / hostloop / lima
  hostloop/           Cowork host-loop handlers — can_use_tool gate, web_fetch dedup, workspace/path hooks
  critique/           critique — run a skill, then grade its self-report against a frozen run record
  staging/            agent-binary resolution + mount naming for the sandboxed live tiers
  egress/             default-deny allowlist proxy
  boundary.ts         sandbox self-test probes
  assert.ts           synchronous assertion evaluator
  sync/               cowork-sync — derive platform baselines from the live app
test/                 vitest unit tests
  fixtures/           unit-test fixtures
baselines/             committed platform baselines (one per Desktop release)
examples/             user-facing worked examples (CI-verified): scenarios/ sessions/ skills/ data/
e2e/                  the harness's own fidelity self-tests: scenarios/ sessions/
fixtures/             harness runtime fixtures (protocol/ golden control-response vectors)
docs/                 guides + references
```

Paths inside a scenario/session resolve relative to that file (see [docs/session.md](./docs/session.md#path-expansion)), so each `examples/`/`e2e/` bundle is self-contained.

## Guidelines

- **Keep the seam.** Release-specific facts belong in `baselines/*.json` (synced), not in code. See [docs/maintenance.md](./docs/maintenance.md).
- **Don't weaken the boundary.** Changes to `src/runtime/container.ts`, `src/egress/sidecar.ts` (the live per-run network/egress enforcer — `docker/compose.yml` is a standalone reference shape only, not invoked), or `docker/compose.yml` must keep the default-deny network + sealed FS. Run `cowork-harness boundary-check` and add/adjust a probe in `src/boundary.ts` if you change the model.
- **Mark unverified code.** Anything not yet run end-to-end against a live agent gets a `// UNVERIFIED` comment so reviewers know.
- **Add a test.** New schema fields, `Decider` rules, or egress logic need a unit test in `test/`. Examples must validate (`test/examples.test.ts`).
- **Consumer-visible workflow changes update the skill.** A change a scenario author would act on — a new assertion key, cassette field, CLI command, or a changed record/replay/verify workflow — must land with a matching update to `.claude/skills/cowork-harness/` (SKILL.md or `references/`). The machine-checkable slices are enforced (`test/skill-docs-sync.test.ts` pins the skill against the assertion-key catalog and the cassette schema's field list; `test/cli-help.test.ts` pins the README command table); prose workflows are on you — this checklist line exists because `effectiveFidelity` shipped consumer-visible and stayed undocumented in the skill until an external consumer flagged it.
- **Typecheck.** `npm test` is `vitest run` — it strips types and does **not** typecheck. `tsconfig.json` covers only `src`; test files are typechecked solely by `tsconfig.test.json`, i.e. only via `npm run typecheck` (which `npm run ci` runs). After editing `src/`, run `npm run typecheck`, not just `npm test`.
- **Format.** `npm run format:check` must pass (`npm run format:write` to fix).

## Validating a companion-skill edit (answer quality)

A behavioral change to `.claude/skills/cowork-harness/` (a SKILL.md refactor, a moved reference) can quietly make the *advice* worse without failing any deterministic test. Two dev instruments over `test/evals/` (live, real money — not part of `npm run ci`, never gate a PR) catch that:

- `npm run eval-gate -- --rebaseline --dotenv .env` records a per-claim pass-rate baseline; `-- --calibrate` tags which claims the skill actually drives (skill-ablation); then a plain `npm run eval-gate -- --dotenv .env` gates a candidate edit with a per-claim Fisher-exact test and refuses to diff across a judge/answerer model change. Commit `test/evals/baseline/profile.json` when you intend to move the baseline.
- `npm run skill-critique -- <skill-dir> --prompt "…" --dotenv .env` surfaces triaged, evidence-grounded improvement *ideas* for a skill (it never edits anything, always exits 0) — a discovery aid, not a gate.

## Extending the sync extractor

When a Desktop release moves something `sync` doesn't read, it reports an `unknown delta`. Extend `src/sync/cowork-sync.ts` to parse the new shape, add the field to the baseline, and note it in the CHANGELOG.

## Commit & PR

- **A pre-commit hook gates cassettes, and it fails CLOSED.** `npm install` points `core.hooksPath` at
  `.githooks/`. Staging a `baselines/desktop-*.json`, any `*.cassette.json`, or any `.json` whose staged
  content carries the `"generator": "cowork-harness"` marker runs `verify-cassettes` — and **anything that
  is not a proven clean exit `0` blocks the commit**, including a missing `dist/cli.js`. That last one
  catches people out: `npm run build` does `rm -rf dist` first, so a tree with a failing typecheck has no
  built CLI at all. Run `npm run build` and re-commit. The check is scoped to commits that stage one of
  those files, so an ordinary source commit is unaffected.

  It is deliberately strict because for a maintainer it is the *only* pre-publication gate: CI triggers on
  `push: [main]` and `pull_request`, but the local flow lands with `merge --ff-only` and pushes afterwards,
  so by the time CI reds, a leaked recording is already in public history. `git commit --no-verify`
  bypasses it — if you need that for anything other than a broken hook, say so in the PR.
- Conventional, imperative commit subjects (`add …`, `fix …`, `parity: sync to <ver>`).
- Open PRs against `main`. CI runs a nine-stage pipeline (`build`, `test`, `action-self-test`, `python`, `image-recipe`, `boundary`, `scenarios`, `parity-drift`, `floor` — see `ci.yml`) on every PR including forks (no secrets needed) except `scenarios`; live scenarios only run on same-repo PRs/pushes with `ANTHROPIC_API_KEY` set.
- **Which stages actually block a merge is narrower than the pipeline.** The branch ruleset requires status *contexts*, not jobs, and the `ci-green` aggregator that carries the `typecheck · test · build` context declares `needs: [build, test, floor, image-recipe]` — so `action-self-test`, `boundary`, `scenarios`, and `parity-drift` run and report but do **not** gate. (`python` **does** gate: its job name is the second required context, `pytest helper lane (-m 'not cowork')` — this line previously listed it as non-gating, which was wrong.) Read a red one as a real signal anyway: `boundary` in particular is the secrets-free proof that the sandbox enforces Cowork's limits. `ci.yml`'s comment above `ci-green` is the authority on the current wiring.
- Describe *what changed and why*; link issues.

## Reporting issues

Use the issue templates. For anything security/sandbox related, see [SECURITY.md](./SECURITY.md).

## This repo's own CI pipeline

Contributor-facing. For *consuming* the harness in your own CI — the token-free gate and the packaged
Action — see [docs/ci.md](./docs/ci.md).

The provided [GitHub Actions workflow](https://github.com/yaniv-golan/cowork-harness/blob/main/.github/workflows/ci.yml) runs a **nine-stage pipeline**. The **build** + **test** stages are the token-free gate you can copy into your skill repo; the `floor`, `action-self-test`, `python`, `image-recipe`, `boundary`, `scenarios`, and `parity-drift` stages are this repo's own fidelity self-tests and are not directly portable (they build the harness's Docker image and run harness-specific e2e scenarios — see [`ci-recipe.md`](./.claude/skills/cowork-harness/references/ci-recipe.md) for the skill-repo template):

| Stage | Runs | Needs | Gates |
|---|---|---|---|
| **build** | format check · version-lockstep guard · typecheck · source guards · build · CLI smoke · token-free `replay` · `verify-cassettes` · `lint` | nothing | every push/PR |
| **test** | the unit suite (vitest), sharded 4-way | nothing | every push/PR |
| **floor** | the unit suite once, unsharded, on Node 22 — the version `engines.node` declares — so the floor is exercised rather than asserted (other jobs run Node 24, the Active LTS line) | nothing | every push/PR; gates the merge context |
| **action-self-test** | packs this commit and runs the packaged `uses: ./` Action across its full case set — pass (committed example cassette), usage-error fail (nonexistent path), assertion fail (checks the reporter renders a ❌ row), `lint`, and `analyze-skill`; `ci.yml` currently carries 11 `command:` invocations, so re-count here rather than trusting this sentence | nothing | every push/PR |
| **python** | `pytest` helper self-checks (`python/`, run with `-m 'not cowork'` — the token-free subset; the Docker/token `@pytest.mark.cowork` tests are excluded) | nothing (token-free assertions only) | every push/PR |
| **boundary** | builds the pinned agent image, brings up the default-deny network, runs `boundary-check`, then `npm run test:live` (live contract tests that guard the binary-resolution assumptions, no token needed) | Docker, arm64 runner | proves the sandbox enforces Cowork's limits — **no API key** |
| **image-recipe** | compiles `docker/Dockerfile.agent` in **both** variants (lean/core and `COWORK_FULL_PARITY=1`) on an arm64 runner, so a recipe change cannot reach the merge gate uncompiled | Docker, arm64 runner | every push/PR; gates the merge context |
| **scenarios** | the live scenario suite (mixed `protocol` + `container` fidelity across `examples/scenarios/`), plus the `e2e/scenarios/*.yaml` smoke set (this repo's own L0/L1/hostloop self-tests, `microvm` excluded — needs a real VM); uploads transcripts/egress logs as artifacts; relies on a runner-local staged agent binary (no in-workflow download step). | `ANTHROPIC_API_KEY` | fork PRs: the whole job is skipped (`if:` guard); same-repo without a key: warns and exits 0 |
| **parity-drift** | reminder to re-`sync` when Desktop updates | nothing | **goes red** if the newest committed baseline is &gt; 90 days old, but sits outside `ci-green`'s `needs:` list, so a red run does not block a merge |

This ordering means cheap checks fail fast, the **boundary parity gate runs without secrets** (so forks get it too), and expensive live runs only happen when a key is present.


## The harness's own suite

```bash
npm run ci            # typecheck + build + test (run format:check separately; NOT the same set as CI's `build` job — see CONTRIBUTING.md)
npm test              # vitest: decider, egress allowlist, launch plan, example validation
cowork-harness boundary-check   # self-verify the sandbox (needs Docker; not part of `npm run ci`)
```

Unit tests cover the scripted-answer logic, the egress allowlist matcher, the session→launch-plan materialization (mounts + discovery settings + env-strip), and a **schema guard** that fails if any shipped baseline/session/scenario stops validating. Add a test alongside any new schema field or `Decider` rule — see [CONTRIBUTING.md](./CONTRIBUTING.md).

> Copy your starting scenarios/sessions from **`examples/`**. The **`e2e/`** directory is the harness's *own* fidelity self-tests (smoke scenarios per tier) — not a template to copy.