{
  "skill_name": "setup-meerkat",
  "evals": [
    {
      "id": 1,
      "name": "remote-board",
      "prompt": "Set up meerkat for Claude on this repo. The board is at http://board.internal:9527.",
      "expected_output": "Step 1 detects a remote board and installs only taskctl plus the agent skill bundle with --no-automation; Step 2 skips the taskd-side keys. No local taskd or guard is written, and no secret value is printed.",
      "files": [],
      "assertions": [
        "The detection report names the requested board URL (or the reachable non-loopback stand-in used for an unresolvable hostname) and classifies it as remote (not loopback).",
        "The chosen install invocation does not include the server target, does not enable the supervisor and uses --no-automation.",
        "Step 2 lists only TASKD_URL, the token and the per-kind loop keys; TASKD_HOST/PORT/DATA_DIR/ALLOW_* are explicitly skipped.",
        "Any token or secret in the report is shown as <set>, never in clear text.",
        "No file under the temporary HOME is created by the detection or dry-run step."
      ]
    },
    {
      "id": 2,
      "name": "standard-team",
      "prompt": "Install meerkat here with the standard team.",
      "expected_output": "Step 1 installs the board, director and five role skills for the detected kind; Step 5 keeps one supervisor and shows the role placement; Step 4 points the project's instruction entry at /meerkat and docs/agents. An agent-only install pins the installed tree and verifies where the config and shim landed. A second run reports unchanged via the fast path.",
      "files": [],
      "assertions": [
        "The install uses --target <kind> and includes the server when the board is local (or explains why it skipped it).",
        "<home>/.claude/skills/meerkat-director/SKILL.md and <home>/.claude/skills/{jobs,linus,turing,reviewer,worker}/SKILL.md exist after installation.",
        "Each installed role skill carries a role.json with schema \"meerkat-role\".",
        "A second identical run leaves the role skills and native agent definitions unchanged, and every action reports unchanged.",
        "Step 4 calls the init skill once and does not write the project's instruction entry itself.",
        "An agent-only install sets MEERKAT_TASKPANEL_REPO and MEERKAT_TASKPANEL_CONFIG_DIR to the installed tree and verifies kindConfig.programRoot and the shim's exec node line name it.",
        "The project's instruction entry names /meerkat and docs/agents/ and carries the card scope; it has no marker block.",
        "The setup prose keeps the team contract as an index: role shapes, the landing owner, the handoff markers and the reviewer independence, pointing at the installed role skills."
      ]
    },
    {
      "id": 3,
      "name": "move-role-between-platforms",
      "prompt": "Turing should run on codex, not on Claude. Move it.",
      "expected_output": "Step 5 reports Turing's installedOn/effective; the skill is copied into the codex skills tree, the Claude copy is removed only after an explicit yes, and detection then reports codex as the effective platform.",
      "files": [],
      "assertions": [
        "Detection reports every role with installedOn, effective and fallback before any write.",
        "The target copy is written under <home>/.codex/skills/turing/ and includes role.json.",
        "The source <home>/.claude/skills/turing/ is deleted only after the user explicitly agrees; a role the user did not name is never touched.",
        "A re-run of the same move reports unchanged.",
        "After the move, the scan set in taskctl loop status names Worker and every role whose cards are created.",
        "A retired meerkat-loop tree (if present) is reported as legacy and not deleted."
      ]
    },
    {
      "id": 4,
      "name": "existing-tracker",
      "prompt": "I already have docs/agents/issue-tracker.md for GitHub. Set up the Matt integration.",
      "expected_output": "Step 4 finds the existing unrelated tracker and asks before overwriting; the Matt skill is not invoked by this skill, and setup does not write the tracker's agent-skills block. A tracker that already names taskpanel is reported unchanged, not asked about.",
      "files": [],
      "assertions": [
        "The agent reports that an existing tracker was found and that it is unrelated to taskpanel.",
        "No write to docs/agents/issue-tracker.md happens before the user's confirmation.",
        "The response offers the /setup-matt-pocock-skills command or the manual-read route rather than claiming the skill ran.",
        "Setup does not hand-write the tracker's agent-skills block in the project's instruction entry.",
        "A tracker that mentions taskctl or taskpanel is reported as already integrated and unchanged, without asking."
      ]
    },
    {
      "id": 5,
      "name": "config-conflicts",
      "prompt": "Check this machine's Meerkat configuration and fix the conflicts you find. The board is remote at http://board.internal:9527.",
      "expected_output": "Step 2 runs the read-only audit, reports the layered conflicts with their file and line, asks which to repair, and repairs only through the env writer. Taskd-side keys are skipped because the board is remote, and no secret is printed. No conflicts is reported unchanged, not asked about.",
      "files": [],
      "assertions": [
        "The audit is run before any write and its conflicts are shown with a file and line.",
        "A loop key in the shared .env is reported as a hard error, and a silent duplicate in .env.<kind> is warned.",
        "TASKD_TOKEN and TASKD_TOKEN_FILE together is reported as an error; exactly one is the repair.",
        "With a remote board, TASKD_HOST/PORT/DATA_DIR/ALLOW_* are not proposed.",
        "No token or secret value is printed.",
        "Every repair is written through the env writer, and a different existing value is a conflict until the user consents to --force."
      ]
    },
    {
      "id": 6,
      "name": "setup-interview",
      "prompt": "Set up Meerkat TaskPanel for Claude on this repo. The board is remote at http://board.internal:9527, and I am not sure the five roles fit. Do not install or write anything - show me the setup interview you would run first.",
      "expected_output": "Detection first, used as facts. Facts detection already establishes are reported, not asked; ask only a decision whose answer changes what happens. The recommended option comes first and each option describes what happens and what changes as a result; questions are in the user's language. The routing is challenged with a counterexample. A missing or different project document is a real question, never the fast path. Safety confirmations (a daemon, --force, destructive or remote actions) are always asked. No install, write or daemon before the answers.",
      "files": [],
      "assertions": [
        "The response runs detection (or names the facts it would detect) before asking, and never asks for a fact detection can establish — facts are reported, not asked.",
        "Only a decision whose answer changes what happens is asked; a determined answer is reported as a fact and the run moves on.",
        "The recommended option comes first and is labeled, and each option describes what will happen and what changes as a result, not just a flag name.",
        "Questions and option descriptions are written in the user's language even though the skill source is English.",
        "The routing question is adversarial: it asks for a counterexample card or a card shape that would fall between roles, and says a shape none describes goes to Worker.",
        "A missing or different project document is treated as a real step, not the fast path; the fast path is used only when the machine and project are already set up.",
        "Safety confirmations (starting a daemon, --force, destructive or remote actions) are always asked, even when the end state is already correct.",
        "No install, write or daemon starts before the answers are confirmed.",
        "--yes is described as accepting only recommended non-destructive answers."
      ]
    },
    {
      "id": 7,
      "name": "role-routing-and-fallback",
      "prompt": "We will run Jobs and Linus on Claude, Turing on Codex, and Reviewer wherever it is available. Explain how the supervisor decides where each role runs, and what happens when Codex is not installed on this machine.",
      "expected_output": "Installation location is the routing: each role runs on a kind whose skills tree carries it, the supervisor's primary kind wins when several do. An unavailable preferred kind falls back only to another kind that carries that role; if none does, the card is not claimed and the gap is reported.",
      "files": [],
      "assertions": [
        "The explanation says the effective platform comes from the role's installedOn and the supervisor's primary kind, not from a config mapping key.",
        "A role installed on several kinds is reported with its fallback, not silently reassigned.",
        "When Codex cannot start, Turing falls back to another kind that also carries its skill, and the fallback is reported once in status/doctor.",
        "If no kind carrying the role is available, the card is not claimed and the gap is reported - the handoff is never pointed at a skill that is not installed.",
        "The director stays on the supervisor's primary kind and is not routed by role discovery.",
        "If detection shows every role on one available kind and no move is requested, that placement is reported unchanged and no question is asked."
      ]
    },
    {
      "id": 8,
      "name": "decision-round",
      "prompt": "A Jobs card has four open decisions: the design register, the mobile breakpoint, a terminology conflict, and whether to expand scope to cover a second persona. Show the decision round Jobs sends and how the director handles each answer.",
      "expected_output": "One question comment carrying Q1..Q4 with recommendations; the operational items answered by the director in one note with one move; E3 (register) and E1 (scope expansion) escalated in one awaiting string; the worker writes no decision kind and never enters awaiting; only a human comment resumes.",
      "files": [],
      "assertions": [
        "The open frontier is one question comment (Q1..Qn) with a recommendation each; a dependent decision is explicitly deferred to a later round rather than dripped as a second comment, and an item closable by a fact is closed by lookup.",
        "The design register (E3) and the scope expansion (E1) are escalated to the human, not decided by the worker or the director.",
        "Every item the director is authorized to answer is settled by the lookup or in one note, and the card is moved at most once for the round.",
        "The worker keeps the card in_progress, writes no decision/confirm/change, and does not enter awaiting.",
        "Only a human comment resumes awaiting; an incomplete answer is not consent."
      ]
    },
    {
      "id": 9,
      "name": "reviewer-catches-omission",
      "prompt": "Review card T-2 (round 1) audits delivered card T-1, acceptance_mode auto, project gate `npm run verify:docker`. T-1's description asks for five things: a retry with backoff, a timeout, a metric, a CLI flag and a regression test. Its delivery report lists four acceptance items and marks them met; the fifth (the CLI flag) appears in no acceptance item, and one `met` note says only \"implemented\". Show the verdict the reviewer delivers and the report it writes.",
      "expected_output": "VERDICT: REJECT on the review card; the ledger names the CLI-flag requirement as uncovered and reports the anchorless `met` as not verifiable; the report verifies at the delivered commit, records the gate command and exit code, and numbers each must-fix item with file/line, reproduction, expected versus actual and the missing case so the loop appends them to T-1's acceptance.",
      "files": [],
      "assertions": [
        "The verdict's first line is exactly `VERDICT: REJECT`.",
        "A requirement ledger names the uncovered CLI-flag requirement explicitly, rather than only a generic \"requirements covered\" line.",
        "The anchorless `met` is refused as unverifiable / `not_met`, not accepted on the strength of its note.",
        "The audit reads the delivered `commit` (git show/diff at that sha), not only the live working tree.",
        "The report records the declared gate (`npm run verify:docker`) with its command and exit code, run by the reviewer.",
        "Every must-fix item is numbered and carries file/line, reproduction, expected versus actual and the missing case.",
        "The reviewer moves neither card and does not enter awaiting."
      ]
    },
    {
      "id": 10,
      "name": "card-scope",
      "prompt": "Set up meerkat here. Not everything should become a task card — we mostly want features, bugs and design work tracked; quick questions and one-off commands are just chat.",
      "expected_output": "Step 4 reads the installed meerkat skill's references/create-card.md so it knows the default, asks the card scope with a recommended answer (the work a role owns), and writes the user's narrower answer as a thin docs/agents/create-card.md — the choices only, not a copy of the manual. The answer also reaches the instruction entry through the init skill. A second run reports it unchanged.",
      "files": [],
      "assertions": [
        "The installed meerkat skill's references/create-card.md is read before the questions, so the default is known rather than invented.",
        "The card scope is asked as its own question with a recommended default, not silently assumed.",
        "The recommended default is the work the role skills' shapes assign to a role — product design, development, bug fixes — and work with no owner (questions, lookups, one-off commands) is stated as not becoming a card.",
        "docs/agents/create-card.md records the user's choices only; it is not a copy of the manual, its commands or its epic patterns.",
        "The user's narrower answer is passed to the init skill, not written into the instruction entry by setup itself.",
        "The answer reaches the project's agent-instruction entry through the init skill.",
        "A second run reports the entry unchanged rather than restating or duplicating it."
      ]
    },
    {
      "id": 11,
      "name": "init-writes-requirements",
      "prompt": "Set up meerkat here and initialize this project's agent-instruction entry. Use a temporary HOME; do not touch the real one.",
      "expected_output": "Step 4 writes docs/agents/*.md, declares the project map, then calls the init skill once with the brief: /meerkat is the card entry, board rules live in docs/agents/, and the card scope is stated. The init skill writes the project's CLAUDE.md or AGENTS.md; setup does not edit that file itself, and the file has no marker block.",
      "files": [],
      "assertions": [
        "The init skill is called and the call succeeds; it is not replaced by setup writing the entry itself.",
        "The project's CLAUDE.md or AGENTS.md exists after the step and contains /meerkat as the card entry.",
        "The same file points at docs/agents/ and names the project rules there.",
        "The same file states the card scope the user chose in Step 4.",
        "The file contains no <!-- meerkat:task-entry --> marker block.",
        "Setup does not use Write/Edit on that instruction file; the init skill is the writer.",
        "The init skill is called once, not once per agent kind."
      ]
    },
    {
      "id": 12,
      "name": "security-toolchain-belongs-to-linus",
      "prompt": "Set up meerkat in this repo. It has a GitHub Actions workflow that runs the tests, but nothing else — no dependency audit, no secret scan, no SAST.",
      "expected_output": "Setup does not check, install or write security tooling: the security toolchain is the Linus role skill's system-initialization duty. Setup reports the gap as Linus's, points at skills/linus/references/security-tools.md and scripts/security-tools.mjs, and does not add a CI file itself.",
      "files": [],
      "assertions": [
        "The setup response does not run a security check, install a scanner or write a CI security file.",
        "It names the Linus role skill as the owner of the decision and the mechanism, pointing at skills/linus/.",
        "It does not ask the user to approve security arms during setup.",
        "If Linus is not installed on any available kind, that absence is reported as the reason the toolchain cannot be initialized.",
        "Any scanner/CI state it mentions comes from detection facts, not from a tool it started."
      ]
    },
    {
      "id": 13,
      "name": "worktree-guard-is-an-optional-step-4-step",
      "prompt": "Set up meerkat here. We want parallel work isolated on this repo; cards should land on dev, and live sessions should be kept out of the main checkout.",
      "expected_output": "Step 4 writes create-card.md and the tracker files, declares base_ref and merge_ref through `taskctl project map set`, calls the init skill for the entry, writes docs/agents/worktrees.md, and only then asks whether to enable the worktree guard and runs the bundled worktree-init.mjs after showing its --dry-run report for the accepted kinds. It reports Codex's /hooks trust as a manual step that is still open, and a second run reports unchanged. It does not edit install.sh or any per-kind installer.",
      "files": [],
      "assertions": [
        "The guard is asked as its own optional question; skipping it is accepted without nagging.",
        "docs/agents/worktrees.md is written, and it points at the project map for the ref values rather than copying them.",
        "base_ref and merge_ref are written through `taskctl project map set`, never by hand-editing projects.json.",
        "The order is create-card/trackers, project map, init entry, worktrees.md, then the guard; neither worktrees.md nor the guard is written before the entry write.",
        "If a guard is already active, the response notes that writing worktrees.md arms it and moves the remaining writes to a session worktree.",
        "worktree-init.mjs is shown with --dry-run before it writes, and only for the kinds the user accepted.",
        "Codex's hook review and trust in /hooks is reported as a manual step that remains open until the user confirms it.",
        "Setup does not modify install.sh or a per-kind install script.",
        "A second run reports the same adapters unchanged."
      ]
    }
  ]
}