# Open Orchestra MVP Open Orchestra is a local-first, provider-agnostic framework for governed multi-agent software delivery. The public CLI is `orchestra`. It stores workflow state in `.agent-workflow/` and coordinates agents through files, events, handoffs, reviews, evidence, gates, locks, and model provenance. See [architecture.md](architecture.md) for the system diagram and layer overview. ## Compatibility - `orchestra` is the only public CLI name. - Existing `.agent-workflow/` data remains valid. - Existing `AGENTS.md`, `CLAUDE.md`, Cursor rules, and generated instruction files remain supported entry points. - `ORCHESTRA.md` is the intended future primary guide name; it is not required for existing projects. ## Prompt Registry Open Orchestra initializes `.generated-prompts/` beside `.agent-workflow/`. The prompt registry stores the latest prompt intent and generation context by artifact type so future agents can preserve conventions without loading all history into the main instruction files. Generated register files: ```text .generated-prompts/ cicd.md code.md diagrams.md docs.md evals.md services.md tests.md ui.md ``` Existing register files are preserved unless `orchestra init --force` is used. Generated runtime guidance uses managed instruction blocks instead of taking ownership of complete files. If a project already has `AGENTS.md`, `CLAUDE.md`, Cursor rules, or `ORCHESTRA.md`, Open Orchestra appends its marked block and preserves user-authored content outside the block. Later refreshes update only the matching Open Orchestra block. Manual edits inside a managed block are reported as drift and are not overwritten unless `--force` is used; forced updates still replace only that block. ## Skills and Context Loading Open Orchestra should treat skills as demand-loaded capabilities. Main files such as `AGENTS.md`, `CLAUDE.md`, Cursor rules, and `ORCHESTRA.md` should contain a compact index and activation rules, while detailed procedures live in skill files. See [skill-loading-strategy.md](skill-loading-strategy.md). ## Commands Use the installed CLI form in project documentation and automation: ```bash orchestra init orchestra health --json orchestra commands manifest --json orchestra status --json ``` For installed-package dogfooding, run `npm run test:e2e:init` after installing the package under test. The suite resolves `orchestra` from `PATH`, uses real temporary workspaces, and avoids Unix shell features so the coverage can run on Windows, Linux, and macOS. The first-use E2E path starts from an empty temporary project and exercises the installed binary through `orchestra init`, `orchestra health --json`, `orchestra task add`, `orchestra status --json`, `orchestra handoff`, `orchestra evidence add`, and `orchestra gate --gate release-readiness`. The same test intentionally checks the blocked release-readiness output before evidence is attached, so missing fields such as QA evidence or a developer to QA handoff remain visible as recovery steps. Supported local development platforms are macOS, Linux, and Windows with a current Node.js runtime. The core validation scripts are written in Node where shell differences matter; in particular, `npm run validate:workflow` does not require `sh` and runs the local build plus `orchestra validate` whenever `.agent-workflow/` exists. CI runs installed-package dogfooding and workflow validation on `ubuntu-latest`, `macos-latest`, and `windows-latest`. When developing Open Orchestra itself, run `npm install`, `npm run build`, `npm run lint`, `npm run typecheck`, `npm run secret-scan`, `npm test`, and use `node bin/orchestra.js ...` when you need to exercise the local checkout before installing or packing it. Core task and graph commands: ```bash orchestra task add --id TASK-1 --title "First task" --owner developer --paths src/commands.ts orchestra task list --json orchestra task update --id TASK-1 --status in_progress orchestra task deps --id TASK-1 --json orchestra graph plan --json orchestra graph run-next --json orchestra graph run-ready --json orchestra github sync --issue 169 --task GITHUB-SYNC --comment --transport gh --dry-run --json orchestra lock claim --task TASK-1 --role developer --path src/commands.ts --reason "editing commands" orchestra lock list --json ``` Issue sync maps visible backlog into workflow state with explicit, reviewable mutations. By default `orchestra github sync --issue ` uses `--transport gh`, reads issue title, body sections, labels, milestone, URL, and state through the GitHub CLI, and creates or updates task `GH-`. Use `--task ` to preserve a named backlog item such as `GITHUB-SYNC`, `--owner` and `--status` to override local task metadata, `--comment` to upsert a single idempotent issue comment with status and evidence counts, and `--close` only when release readiness passes. If release readiness is blocked, closure requires `--accepted-risk `. `--dry-run --json` prints planned commands without writing local tasks, comments, or issue state. GitHub comment payloads are file-backed. Generated commands use `gh issue comment --body-file ` for new comments and `gh api --input ` for comment updates instead of embedding multiline markdown after `--body` or `-f body=...`. Agents should keep this pattern for any copied or derived command: write markdown or JSON payload bytes to a temporary file, pass the file path as an argv value, and avoid logging the payload contents when command execution fails. The transport boundary is intentionally tracker-agnostic. The local CLI can execute `gh` because it is a child process with stable arguments. MCP-backed trackers such as GitHub MCP, Jira, Bitbucket, GitLab, or a custom work tracker belong to the agent runtime rather than the Node process, so they should be exposed through a tracker skill/adapter that implements equivalent issue-read/comment-upsert/close operations. `--transport mcp-skill` records that intent and fails locally with recovery guidance unless a runtime injects a compatible runner. Default mapping is intentionally small: issue title -> task title, issue URL -> backlog item, `## Acceptance Criteria` bullets -> task acceptance criteria, labels -> risks, milestone -> scope, and `## Test Evidence Required` -> test strategy. Teams can make the mapping explicit per run with command flags until project-level mapping policy is added. Advisory mode is the consulting or audit entry point. It creates `.agent-workflow/advisory/` artifacts without writing root instruction files such as `AGENTS.md` or `ORCHESTRA.md`. Use it when the target repository still needs evaluation or owner approval: ```bash orchestra init --advisory orchestra status --json orchestra health --json orchestra advisory convert --json ``` The advisory artifacts include a README, role guides, portable decisions, conversion guide, and `project-task.json`. After the target project is confirmed, `orchestra advisory convert` promotes that JSON into a real workflow task while keeping writes inside workflow-owned paths. Context, memory, skills, and runtime rendering: ```bash orchestra context --task TASK-1 --json orchestra context --task TASK-1 --budget 12000 --json orchestra memory query --task TASK-1 --role developer --budget 2000 --json orchestra memory hook --point before_plan --task TASK-1 --role qa --json orchestra skills plan --task TASK-1 --json orchestra skills render --target codex --task TASK-1 orchestra protocol render --target codex --task TASK-1 orchestra workflow render --target codex --task TASK-1 orchestra runtime brief --task TASK-1 --runtime codex-cli --json orchestra runtime delegate-plan --task TASK-1 --runtime claude-cli --roles developer,qa --json orchestra runtime sessions --task TASK-1 --json orchestra runtime session --session TASK-1:claude-cli --action suspend --json ``` Memory hooks are explicit retrieval points for lessons learned and prompt registry entries. Bundles are scoped by role (`product_owner`, `architect`, `developer`, `qa`, or `release_manager`) and report token budget, estimated tokens, kept and omitted sections, and trimming rationale in both JSON and human output. Runtime delegation sessions are derived from append-only events. Session operations record suspend, resume, cancel, or close events for the parent runtime and web console to reconcile active work, stale sessions, and handoff state without creating a second session store. Delivery evidence, handoff, and gates: ```bash orchestra readiness --task TASK-1 orchestra handoff --task TASK-1 --from developer --to qa --changed "..." --behavior "..." --tests "..." --commands "..." --update-owner orchestra evidence add --task TASK-1 --role developer --type command --summary "npm test passed" --command "npm test" --exit-code 0 orchestra review --task TASK-1 --role qa --result approve --findings "..." --recommendation "..." orchestra gate --gate architecture --task TASK-1 orchestra gate --gate qa-release --task TASK-1 orchestra gate --gate risk-review --task TASK-1 orchestra gate --gate release-readiness --task TASK-1 ``` `orchestra release check --json` includes a structured `releaseReadiness` report when the workspace is initialized. It maps each acceptance criterion to traceable evidence, flags uncovered AC, unresolved blocker reviews, active locks, and missing smoke or rollback evidence. Browser/UI criteria require Playwright-style screenshot, trace, or video evidence unless the task has an accepted risk. For `1.0.0` release candidates, the same report includes `gaReadiness`. This is the repeatable go/no-go layer for production promotion. A GA-ready workspace must have no uncovered acceptance criteria, no blocking reviews, no active locks, smoke and rollback evidence, documentation or release notes evidence, observability evidence, security and permissions review evidence, package provenance evidence, public CLI contract evidence, migration or upgrade readiness evidence, and release test matrix evidence. Accepted risks must include an owner, a rationale, and a follow-up or expiry signal in the rationale. The human release check summary reports both the legacy release readiness gaps and the number of GA blockers. Use JSON output for the full criterion list: ```bash orchestra release check --json ``` ## Config Schema Migrations Open Orchestra workspace config is versioned through `.agent-workflow/config.json`. For pre-1.0 workspaces, inspect migrations before writing anything: ```bash orchestra config migrate --json ``` Apply a supported migration only after reviewing the plan: ```bash orchestra config migrate --apply --json ``` Migration apply writes a backup under `.agent-workflow/backups/` before updating `config.json`. Newer unsupported schema versions are reported as errors and are not modified. Upgrade dogfooding: ```bash orchestra upgrade --json orchestra upgrade --beta --json orchestra upgrade --smoke --json ``` Stable installs use `npm install -g @jterrats/open-orchestra@latest`; beta dogfooding uses `npm install -g @jterrats/open-orchestra@beta` or the exact version reported by `orchestra upgrade --beta`. After upgrading, run the smoke command shown by the CLI to verify the installed binary and health report. The upgrade output also includes the exact rollback install command for the previous version plus `orchestra config migrate --json` as the migration compatibility check before accepting the new package. Set `SKIP_NETWORK_TESTS=1` in CI jobs that must avoid external network calls; package update checks and provider smoke tests honor it, while localhost E2E coverage remains available. Release go/no-go: - Release tags are CI-owned for normal releases. Humans and local agents should bump `package.json` and `package-lock.json`, commit, and push the release candidate; they should not create or push the release tag locally unless a documented break-glass release decision approves it. - The `Create release tag` workflow resolves the tag from `package.json`, checks that it matches `v`, runs the configured precommit gate, then creates the annotated tag, creates the GitHub Release, and triggers npm publish. - If the intended tag already exists remotely, do not move or overwrite it. Bump to the next patch/prerelease version or record an explicit rollback or break-glass decision. - Run `npm run precommit` locally before pushing release changes. - Confirm `npm run security:audit` passes or attach an accepted security risk. - Confirm `orchestra release check --json` passes the package provenance gate, which runs `npm pack --dry-run --json` and rejects private workflow state, generated prompts, env files, and missing public package entry points. - Confirm `npm run package:validate` passes before publishing. This rebuilds the package dry-run file list without lifecycle scripts and verifies the CLI bin, compiled dist output, legacy and React web console assets, built site assets, docs, rules, skills, package metadata, README, and changelog are included. - Confirm GitHub Actions CI is green for the latest pushed HEAD. - Confirm installed-package dogfooding passes on `ubuntu-latest`, `macos-latest`, and `windows-latest`. - Confirm `Create release tag`, site publish, and npm publish workflows are successful for the intended release commit. - After publish, confirm the registry exposes the exact version from `package.json` with `npm view @jterrats/open-orchestra@$(node -p "require('./package.json').version") version`. - Confirm the npm publish workflow authenticates with either a maintainer `NPM_TOKEN` that has read-write access plus CI 2FA bypass for `@jterrats/open-orchestra`, or npm Trusted Publishing/OIDC configured with organization/user `jterratsdev`, repository `open-orchestra`, workflow filename `publish-npm.yml`, and no environment. If npm returns `404 Not Found` during publish for an existing scoped package, treat it as a token or scope permission failure until proven otherwise; do not print token values in logs or evidence. If `npm whoami` returns `E401 Unauthorized`, regenerate the GitHub Actions `NPM_TOKEN` secret from the npm maintainer account before rerunning publish. If `npm publish` returns `EOTP`, the token is valid but cannot bypass 2FA; switch to a granular token with bypass 2FA enabled or run `publish-npm.yml` with `auth_mode=trusted` after configuring npm Trusted Publishing for the package. Support diagnostics: ```bash orchestra diagnostics bundle --json orchestra diagnostics bundle --output support/diagnostics.json --json ``` The diagnostics bundle is written inside the workspace and includes local health, workflow validation, task and lock counts, run failure summaries, and a sanitized config summary. It redacts token-like values, email addresses, user home paths, and credential fields before writing the JSON artifact so it can be attached to an issue or support handoff. Performance budgets: ```bash npm run performance:bench -- --json npm run performance:bench -- --tasks 500 --json ``` The benchmark seeds a synthetic workspace and measures core CLI commands plus read-only web API routes against documented millisecond budgets. Use the JSON result as release evidence for large-workspace readiness and investigate any failed budget before promoting a production release. Model routing, budget, telemetry, and release: ```bash orchestra model providers --json orchestra model set-role --role developer --provider openai --model gpt-example orchestra budget check --task TASK-1 --json orchestra telemetry status --json orchestra release check --json orchestra release candidate --version 1.0.0-beta.1 --json orchestra release tag --version 1.0.0-beta.1 --approve --approver "user" --json ``` Use `orchestra commands manifest --json` as the canonical machine-readable command surface. It includes stable JSON contracts, required flags, optional flags, and examples generated from the current CLI. ## Autonomous Workflow Engine `orchestra workflow run` executes a full story lifecycle as a governed multi-phase sequence. Each phase creates a sub-task, generates handoff artifacts, and persists state in an append-only run log. ``` PM → PO [gate] → Architect [sizing gate] → Developer → QA [gate] → Release ``` ```bash # Inspect the phase graph without persisting state orchestra workflow run --task FEAT-001 --dry-run --gates phase # Fully autonomous — no human approval required orchestra workflow run --task FEAT-001 --gates none # Gate-controlled — pauses at po→architect and qa→release orchestra workflow run --task FEAT-001 --gates phase # Resume a paused or clarification-suspended run orchestra workflow run --task FEAT-001 --resume # List all runs with status and phase trace orchestra workflow runs ``` **Architect sizing gate:** always enforced regardless of `--gates` mode. The architect must record a sizing decision (`xs/s/m/l/xl`) before the developer phase starts. If missing, the run fails with the exact command to resolve it: ```bash orchestra decision add --task FEAT-001 --owner architect \ --title "Story sizing" --decision "m [5 points]" \ --context "..." --consequences "..." --status accepted ``` ### Clarification Loop Developers or QA engineers can surface blocking questions to the PO or architect mid-phase without abandoning the run. ```bash # Open a clarification (suspends the active developer/qa phase) orchestra workflow clarify --run --from developer --to po \ --question "Should empty input return null or throw?" # Answer the clarification (resumes the phase) orchestra workflow clarify-respond --run --clarification \ --answer "Return null — downstream handles it." # Resume execution after the answer orchestra workflow run --task FEAT-001 --resume # Inspect all clarifications for a run orchestra workflow clarify-list --run ``` ## Workflow Files ```text .agent-workflow/ config.json roles.json tasks.json locks.json events.jsonl workflow-runs.jsonl ← autonomous run state (append-only) clarifications.jsonl ← clarification loop records (append-only) estimates.jsonl ← declared effort baselines (append-only) source-of-truth.json agent-lessons.jsonl approvals/ decisions/ handoffs/ evidence/ reviews/ runs/ ``` ## Stable JSON Contracts The VS Code Control Center should consume stable JSON outputs instead of parsing human-readable text or duplicating file reads. Current UI-facing commands include: - `orchestra status --json` - `orchestra validate --json` - `orchestra graph plan --json` - `orchestra summary --json` - `orchestra context --task --json` - `orchestra approvals list --json` - `orchestra evidence list --json` - `orchestra roles list --json` - `orchestra config show --json` The role catalog JSON includes capabilities, required handoff fields, blocking authority, activation criteria, expected evidence, and gate participation. ## Role Activation Open Orchestra initializes a broad role catalog but does not require every role to participate in every task. The parent/orchestrator should activate roles based on task type, risk, touched paths, impact areas, and gate requirements. Default roles include delivery and specialist roles such as Product Manager, Product Owner, Business Analyst, Architect, Developer, Tech Lead, Frontend Specialist, Backend Specialist, Mobile Specialist, QA, SDET, Security, DevOps, Platform Engineer, SRE, DBA, UX/UI Designer, Release Manager, Compliance/Privacy, Technical Writer, AI Evaluation Engineer, and Support/Customer Operations. They also include orchestration roles for modern multi-agent systems: Planner, Reviewer/Critic, Toolsmith, Context Curator, Policy/Governance, Observability/Incident Response, Data/Privacy Officer, Domain Expert, UX Researcher/Accessibility Reviewer, Performance Engineer, and Game Designer. Specialist profiles and their source rationale are documented in [dev-team-specialist-role-profiles.md](dev-team-specialist-role-profiles.md). Each default role declares: - activation criteria for when the role should be used; - expected evidence that should be produced or reviewed; - gate participation for readiness, architecture, QA, risk, or release decisions. This keeps collaboration explicit even when the underlying LLM provider does not support native subagent-to-subagent communication. ## VS Code Control Center The VS Code Control Center scaffold is under `extensions/vscode-open-orchestra`. It follows the same boundary as the CLI: extension UI calls stable JSON commands and renders results, while workflow state remains in Open Orchestra services and files. The first supported commands are status, validate, graph plan, summary, roles, approvals, evidence, config inspection, and Playwright evidence attachment. ## Reusable Core - `src/types.ts` is the compatibility barrel for public TypeScript contracts shared by CLI, web, VS Code, and future adapters. New domain-owned contracts should live in focused modules under `src/types/` and be re-exported from the barrel; existing consumers should continue importing from `src/types.ts` unless they intentionally depend on a narrower internal domain module. - `src/workflow-services.ts` owns reusable workflow behavior for tasks, locks, readiness, handoffs, reviews, evidence, status, and summary. - `orchestra task deps` checks whether task dependencies are approved or done before downstream work starts. - `orchestra graph plan` classifies current tasks as dependency-ready, blocked, or complete before execution. - `orchestra graph run-next` executes the first dependency-ready task through the existing run flow. - `orchestra graph run-ready` executes the current dependency-ready task snapshot sequentially through the existing run flow. - Graph scheduling omits tasks with active task locks or path conflicts and reports them as locked. - Graph batch runs write summary artifacts under `.agent-workflow/runs/batches/`. - `orchestra decision` records ADR-like decision artifacts and append-only events for task context. - `orchestra context` aggregates task state, dependencies, locks, decisions, handoffs, reviews, evidence, gates, model provenance, memory, and risks before an agent starts work, with a configurable context budget. - `orchestra memory query` and `orchestra memory hook` retrieve bounded lessons and prompt-registry memory at lifecycle checkpoints without loading the full history into the prompt. - `orchestra plan` generates an ordered parent-agent execution plan by role without invoking model providers. - `orchestra run` checks task dependencies and usage budgets before execution, then can execute an approved budget fallback with the deterministic fake provider, recording step events and model provenance without network calls. - Budget fallback requests write reviewable approval proposals under `.agent-workflow/approvals/`. - Stored approved budget fallback proposals can be consumed by a later `orchestra run` without repeating approval flags. Timeout errors do not fall back to another provider automatically. - Run steps write artifacts under `.agent-workflow/runs//` so downstream agents can consume prior role output. - `orchestra handoff --update-owner` records the handoff and transfers task ownership to the receiving role explicitly. - `orchestra pr-summary` generates PR or release-review context from task state, evidence, reviews, gates, handoffs, locks, rollout notes, and rollback notes. - `orchestra playwright plan` generates deterministic Playwright test plans from task acceptance criteria and QA context. - `orchestra playwright evidence` attaches Playwright screenshots, traces, videos, or reports through the existing evidence workflow. - `orchestra usage` aggregates request counts, token totals, and estimated model cost from recorded model provenance events. - `orchestra budget check` enforces configured usage budgets from local provenance events and fails when limits are exceeded. - `orchestra telemetry` is disabled by default and requires explicit consent before export or submission. - `orchestra approvals` lists, shows, approves, and rejects approval proposal artifacts through append-only decision events. - `src/workflow-gates.ts` owns reusable gate evaluators. Current gates include `architecture`, which blocks non-trivial work until proposal, user approval, and architect approval are present; `qa-release`, which requires QA plan, execution status, evidence, and no unresolved critical QA block; `risk-review`, which maps impact areas to required risk-owner reviews; and `release-readiness`, which composes readiness, architecture, QA, risk, evidence, handoff, review, and lock checks. - `src/model-providers.ts` owns provider-agnostic model contracts, an in-memory provider registry, and a fake provider for deterministic tests. - `orchestra model providers` lists configured provider routing without invoking real model APIs. - `orchestra model set-role` configures provider/model routing per role without invoking real model APIs. - `orchestra model complete-fake` simulates provider fallback behavior without invoking real model APIs. - `orchestra model provenance` records model usage metadata in the append-only event log without storing raw prompts, raw responses, or secrets. - `orchestra release check`, `release candidate`, `release tag`, and `release evidence` provide a local release-readiness flow for stable, beta, and other prerelease tags. Candidate plans include rollout, rollback, post-release, changelog, and semver-impact sections; release evidence accepts `smoke`, `rollback`, `post-release`, and `release-notes` kinds. - `orchestra health --network` can check for a newer published package version and print the exact `npm install -g @jterrats/open-orchestra@` update command when one is available. - Package identity is intentionally split: `@jterrats/open-orchestra` is the canonical npm package and `orchestra` is the installed CLI binary. Do not use `@jterrats/orchestra` in install snippets unless a future alias package is explicitly published and documented. - `src/commands.ts` is the CLI adapter: it parses command options, delegates to services, and renders terminal output. - Services accept an explicit repo root, so future web, GitHub Actions, Playwright, or multi-model orchestration layers can reuse the same core without depending on `process.cwd()`. ## Benchmark & Sprint Burndown `orchestra estimate` declares the three-mode effort baseline at story start. After the autonomous run completes, `orchestra benchmark` joins the declared estimate with the actual cycle time and quality signals automatically computed from the event log. Quality signals collected automatically: - `REVIEW_RECORDED` events → review count, blocking reviews (result=block or severity high/critical) - `EVIDENCE_ADDED` events → evidence artifact count - `LESSON_RECORDED` events → lesson count - `GATE_BLOCKED` events → gate block count - `MODEL_PROVENANCE_RECORDED` events → total tokens, estimated cost ```bash # Declare baseline at story start (once per story) orchestra estimate --task TASK-1 --sizing m --solo-days 5 --ai-unguided-days 3 --ai-guided-days 2 # Per-story benchmark after run completes orchestra benchmark --task TASK-1 # Sprint summary table across all stories with estimates orchestra benchmark --summary # Sprint burndown (developer points > architect points as fallback) orchestra burndown --sprint TASK-1,TASK-2,TASK-3 ``` See [benchmark.md](benchmark.md) for the full reference. ## Current Scope - Autonomous workflow engine (`workflow run`) executes the full PM→PO→Architect→Developer→QA→Release phase sequence with configurable human gates and an architect sizing gate. - Clarification loop (`workflow clarify`) allows developer and QA phases to surface blocking questions to PO or architect without abandoning the run. - Benchmark (`orchestra estimate` + `orchestra benchmark`) compares declared effort baselines against actual cycle time and automatically collected quality signals from the event log. - Sprint burndown (`orchestra burndown`) computes ideal vs actual lines from developer or architect story point estimates. - Provider-backed workflow execution supports deterministic `fake`, OpenAI Responses API, and Anthropic Messages API adapters when configured. The default provider remains `none`, so workflows stay deterministic unless routing is explicitly enabled. - No automatic code editing. - Python workers are represented in config only and disabled by default. - Static analysis is enforced locally through `.githooks/pre-commit` after running `npm run hooks:install`.