# External Baseline Dogfooding Findings

This file tracks issues found while using Open Orchestra to coordinate an
external improvement backlog.

## Finding 1: Concurrent `task add` commands can corrupt `tasks.json`

- **Open Orchestra issue:** https://github.com/jterrats/open-orchestra/issues/85
- **Status:** fixed in Open Orchestra

- **Date found:** 2026-05-06
- **Source context:** external local repository
- **Command pattern:** multiple `node_modules/.bin/orchestra task add ...` commands executed concurrently
- **Observed behavior:** `.agent-workflow/tasks.json` ended up with trailing partial JSON and subsequent Orchestra commands failed with `Unexpected non-whitespace character after JSON`.
- **Impact:** users or automation that parallelize CLI calls can corrupt workflow state.
- **Expected behavior:** state writes should be serialized, locked, or written atomically so concurrent commands either succeed safely or fail without corrupting existing state.
- **Workaround used:** manually rewrote `.agent-workflow/tasks.json` as valid JSON and continued with sequential state-changing commands.
- **Suggested fix:** introduce file-level locking plus atomic write through a temp file and rename for all workflow state mutations.
- **Resolution:** workflow state writes are serialized with per-file lock directories and JSON state files are written through temporary files followed by atomic rename.

## Finding 2: Evidence type validation is stricter than the CLI help implies

- **Open Orchestra issue:** https://github.com/jterrats/open-orchestra/issues/84
- **Status:** fixed in Open Orchestra

- **Date found:** 2026-05-06
- **Source context:** external local repository
- **Command pattern:** `orchestra evidence add --type validation ...`
- **Observed behavior:** the command failed with `type must be one of: command, file, screenshot, trace, video, log, report`.
- **Impact:** users must know the closed evidence type list before calling the command.
- **Expected behavior:** CLI help should show the allowed values, and validation failures should suggest the nearest valid type when possible.
- **Workaround used:** recorded validation evidence as `--type command`.
- **Suggested fix:** update command help to include the enum and consider aliases such as `validation -> command`.
- **Resolution:** `validation` is accepted as an alias for `command`; CLI help lists canonical types and the alias; invalid type errors include valid types and aliases.

## Finding 3: `reviewer` is not accepted as a review role

- **Open Orchestra issue:** https://github.com/jterrats/open-orchestra/issues/86
- **Status:** fixed in Open Orchestra

- **Date found:** 2026-05-06
- **Source context:** external local repository
- **Command pattern:** `orchestra review --task SA-90 --role reviewer --result approve ...`
- **Observed behavior:** the command failed with `unknown reviewer role: reviewer`.
- **Impact:** the role catalog and README describe a Reviewer / Critic capability, but the review command does not accept the intuitive `reviewer` role id.
- **Expected behavior:** `reviewer` should either be a valid role alias or the error should list accepted role ids.
- **Workaround used:** record the approval with an existing valid role.
- **Suggested fix:** add `reviewer` as a role alias or expose accepted roles in validation errors and CLI help.
- **Resolution:** `reviewer` is accepted as an alias for `reviewer_critic`; CLI help lists the alias; unknown reviewer role errors include accepted role ids and aliases.

## Finding 4: Concurrent `lock claim` commands can create duplicate lock ids

- **Open Orchestra issue:** https://github.com/jterrats/open-orchestra/issues/87
- **Status:** fixed in Open Orchestra

- **Date found:** 2026-05-06
- **Source context:** external local repository
- **Command pattern:** multiple `node_modules/.bin/orchestra lock claim ...` commands executed concurrently
- **Observed behavior:** two distinct locks were created with the same id, `lock-1778052846730`, for different paths.
- **Impact:** `lock release --id <id>` becomes ambiguous and can release the wrong lock or require manual state repair.
- **Expected behavior:** lock ids should be collision-resistant even when lock claims happen in the same millisecond.
- **Workaround used:** avoid concurrent state-mutating Orchestra commands and continue sequentially.
- **Suggested fix:** generate lock ids with a random suffix or monotonic per-process sequence in addition to the timestamp.
- **Resolution:** lock ids now include a timestamp plus `crypto.randomUUID()`, and concurrent lock claim regression coverage verifies uniqueness.

## Finding 5: Runtime bootstrap examples are not portable after package install

- **Open Orchestra issue:** https://github.com/jterrats/open-orchestra/issues/96
- **Status:** fixed in Open Orchestra

- **Date found:** 2026-05-06
- **Source context:** external local repository
- **Command pattern:** `node_modules/.bin/orchestra init --force`
- **Observed behavior:** generated `AGENTS.md` and `ORCHESTRA.md` bootstrap blocks recommend commands such as `node bin/orchestra.js health --json` and `node bin/orchestra.js task list --json`.
- **Impact:** those examples only work inside the Open Orchestra repository layout. When Open Orchestra is installed into another repository as a package or local file install, the consumer repo does not have `bin/orchestra.js`, so agents following the generated bootstrap will run the wrong command.
- **Expected behavior:** generated runtime bootstrap guidance should use the installed executable form, such as `orchestra health --json`, or derive a target-appropriate command prefix from the runtime/install context.
- **Workaround used:** run `node_modules/.bin/orchestra ...` directly from the consuming repo.
- **Suggested fix:** update command manifest examples and runtime bootstrap rendering to prefer `orchestra` for installed usage, with repo-local `node bin/orchestra.js` reserved for Open Orchestra development docs.
- **Resolution:** command manifest examples and generated runtime bootstrap blocks now use the portable installed executable form, `orchestra ...`; regression coverage verifies generated bootstrap content does not include `node bin/orchestra.js`.

## Finding 6: Concurrent `evidence add` commands can overwrite artifact files

- **Open Orchestra issue:** https://github.com/jterrats/open-orchestra/issues/98
- **Status:** fixed in Open Orchestra

- **Date found:** 2026-05-06
- **Source context:** Open Orchestra repository
- **Command pattern:** multiple `orchestra evidence add ...` commands executed concurrently
- **Observed behavior:** two distinct `EVIDENCE_ADDED` events pointed to the same artifact file when they were created in the same millisecond.
- **Impact:** the later evidence write overwrote the earlier evidence content, leaving one event pointing to mismatched artifact content.
- **Expected behavior:** every evidence event should keep a unique artifact path even under concurrent CLI usage.
- **Workaround used:** rerun evidence recording sequentially.
- **Suggested fix:** include a collision-resistant suffix in generated evidence artifact filenames.
- **Resolution:** evidence artifact filenames now include a UUID in addition to task, timestamp, and type; concurrent CLI regression coverage verifies unique artifact paths.

## Workspace State Safety Notes

- Open Orchestra serializes local workflow state writes with per-file lock directories next to the target file.
- JSON state files are written through temporary files and atomic rename to avoid partial JSON on interruption.
- The lock mechanism is intended for local filesystems. Network filesystems or stale lock directories after a killed process may require manual cleanup if a command times out waiting for a lock.
- State-mutating Orchestra commands should still prefer explicit task/path locks when multiple agents edit code, because file serialization protects workflow metadata, not source-code merge conflicts.