--- install: core --- # /orchestrate — Phase 3: Dependency Analysis & Execution Plan ## Phase 3: Dependency Analysis & Execution Plan ### Step 3A: Analyze explicit dependencies For each issue (including newly spawned ones from Phase 2), check: 1. **Explicit dependencies**: Issue body contains "Depends on #X" or "Blocked by #X" 2. **Milestone ordering**: Issues within a milestone may have a natural order (foundation → features → UI) 3. **File conflicts**: If two issues modify the same files, they should be sequential (not parallel) 4. **Parent-child links**: If issue body contains "Parent investigation: #{X}", it was spawned by an investigation — no special ordering needed unless explicitly stated Read each issue's body briefly to check for dependency markers: ```bash for NUM in {issue_numbers}; do gh issue view $NUM --json body --jq '.body' | grep -iE 'depends on|blocked by|after #|parent investigation' || echo "no deps" done ``` ### Step 3B: Domain estimation For each issue, estimate which domains it touches based on title, body, and labels. This improves DAG construction — issues in the same domain likely touch the same files and should be serialized (one becomes a predecessor of the other). ```bash declare -A ISSUE_DOMAIN # issue → comma-separated matched domain tags, or "NONE" (forge#1913) for NUM in {issue_numbers}; do ISSUE=$(gh issue view $NUM --json title,body,labels --jq '{title: .title, labels: [.labels[].name], body: (.body[:300])}') echo "=== #$NUM ===" MATCHED_DOMAINS=() if echo "$ISSUE" | grep -qiE "credit|billing|pricing|stripe|charge|refund"; then echo " BILLING"; MATCHED_DOMAINS+=("BILLING"); fi if echo "$ISSUE" | grep -qiE "auth|session|jwt|login|permission|oauth"; then echo " AUTH"; MATCHED_DOMAINS+=("AUTH"); fi if echo "$ISSUE" | grep -qiE "worker|queue|job|task|background|consumer"; then echo " WORKER"; MATCHED_DOMAINS+=("WORKER"); fi if echo "$ISSUE" | grep -qiE "migration|\.sql|database|postgres|alembic"; then echo " DATABASE"; MATCHED_DOMAINS+=("DATABASE"); fi if echo "$ISSUE" | grep -qiE "component|page|layout|dashboard|ui|ux|frontend|web/src"; then echo " FRONTEND"; MATCHED_DOMAINS+=("FRONTEND"); fi if echo "$ISSUE" | grep -qiE "docker|deploy|traefik|nginx|ci|cd|infra|github.action"; then echo " INFRA"; MATCHED_DOMAINS+=("INFRA"); fi if echo "$ISSUE" | grep -qiE "llm|extract|schema|format|embedding|model"; then echo " AI"; MATCHED_DOMAINS+=("AI"); fi # For project-specific domains, configure keywords in forge.yaml → review.domains and extend above # Materialize the result — every issue gets an explicit ISSUE_DOMAIN entry, even when no # keyword matched ("NONE"). This is what Step 3D.6's completion gate checks for key presence # (not truthiness) — an un-set key means this loop never ran for that issue. if [ "${#MATCHED_DOMAINS[@]}" -eq 0 ]; then ISSUE_DOMAIN[$NUM]="NONE" else ISSUE_DOMAIN[$NUM]=$(IFS=,; echo "${MATCHED_DOMAINS[*]}") fi done ``` **Use domain info for DAG edge construction:** - Issues in the SAME domain (especially WORKER, BILLING, DATABASE) are more likely to touch the same files → add predecessor edges to serialize them - Issues in DIFFERENT domains are more likely independent → safe to parallelize - BILLING + AUTH issues should be prioritized early (security-critical) - **DATABASE issues are ALWAYS serialized — hard rule, no exceptions.** Multiple agents writing migrations simultaneously will produce duplicate migration numbers (e.g., two `0067_*.sql` files), which breaks the migration runner. DATABASE issues form a linear predecessor chain in the DAG. If 3 DATABASE issues are in a batch: A has no predecessors, B has {A} as predecessor, C has {B} as predecessor. **Domain tags are stored in `ISSUE_DOMAIN[$NUM]`** (materialized above) for use in the plan presentation (Step 3E) and the Step 3D.6 completion gate. ### Step 3C: Multi-layer conflict detection Domain estimation (above) catches broad category overlap but misses cases where two issues modify the exact same file without mentioning the same keywords, or where two issues touch different files that share indirect dependencies (imports, config, barrel exports). This step uses three layers of structural analysis to catch conflicts the keyword heuristic misses. #### Layer 1: Explicit file-overlap extraction **For issues that already have a `FORGE:CONTRACT` comment** (built in a prior wave or session, or re-extracted mid-batch), extract its `### Deliverables` table first — that table states intent to change, so it outranks every other source (forge#2848). **For issues that already have an INVESTIGATOR comment** (from Wave 0 or a prior session) and no contract-derived paths, extract their Affected Files list, scoped to that comment's own `### Affected Files` section. **For issues WITHOUT an investigation comment**, fall back to parsing the issue body, scoped to a deliverables-shaped heading (`## Affected Files`, `## Deliverables`, or `### Files to change`) — never the whole body. Both code paths accumulate into a single `LAYER1_FILES` array (declared once, before the loop) — this is the batch-wide file set that Layer 5's co-change query (below) reuses, per the "file list already extracted in Layer 1" reference in Layer 2 and Layer 5. Extraction runs through `scripts/extract-affected-files.sh` (not an inline `grep -oP` over the whole text) so that stray paths mentioned in `## Context` / `## Prior art` / `## Related` / `## Root Cause` — or anywhere outside a deliverables-shaped heading — are never collected as if they were files the issue changes. A populated-but-wrong file list is strictly worse than an empty one: it clears Layer 4's `<2 paths` conservative-serialization threshold with false confidence, defeating the safety net that threshold exists to provide (forge#2436). Each issue's extraction also carries a **provenance** tag, in descending order of confidence — `contract-deliverables` (from a `FORGE:CONTRACT` comment's own `### Deliverables` table), `affected-files-section` (from the INVESTIGATOR comment's own scoped section), `body-fallback` (from the raw issue body's scoped section, pre-investigation), or `none` (no scoped section found; correctly yields zero paths so Layer 4 fires). This is recorded per issue in `FILE_SOURCE[$NUM]` and consumed by Layer 5 (below) and Step 3E's plan presentation. **Why `contract-deliverables` ranks highest** : a deliverables table states *intent to change* ("I will edit this file"), whereas an Affected Files list — and far more so a raw issue body — routinely names files as *context*: "this interacts with X", "similar to the check in Y". The extractor cannot tell those apart, which is what makes a body-derived file list over-predict overlap. **Temporal caveat — do not over-claim this source**: `FORGE:CONTRACT` is posted at *build* time (`work-on/build.md` Phase B2), which is **after** this phase builds the DAG. On a cold first-pass plan no contract exists, so this provenance is inert by construction and extraction lands on one of the three values below. It pays off only on paths that explicitly re-extract later: mid-batch re-derivation on a dropped edge (`phase-4-execution.md`'s DONE-arm handling, lines 1208-1268) and `IN_PROGRESS` predecessors built in an earlier wave or session. Wake/compaction reconstruction does not retain the in-memory edge metadata and does not re-extract. A contract whose deliverables table yields zero paths **falls through** to the investigator source rather than blackholing to `none` — a contract is an upgrade over that source, never a replacement for it. ```bash LAYER1_FILES=() declare -A EDGE_KIND # "{PRED}:{SUCCESSOR}" → same-file | directory | shared-module (forge#1860) declare -A EDGE_FILES # "{PRED}:{SUCCESSOR}" → the specific file(s) that triggered the edge (forge#1860) declare -A FILE_SOURCE # {NUM} → contract-deliverables | affected-files-section | body-fallback | none (forge#2436, forge#2848) declare -A ISSUE_FILES # {NUM} → newline-separated declared file set (forge#2844) # Resolve ForgeDock's helper from the runtime installation before falling back to # the target repository. The orchestrator runs inside the project being worked on, # so a bare `scripts/extract-affected-files.sh` silently fails when that project # has not copied ForgeDock's helper scripts into its own repository (observed in # OpenCode runs against installed ForgeDock). Keep the precedence aligned with # phase-4-execution.md's classify-lane resolver. resolve_extract_affected_files() { local candidates=() [ -n "${FORGE_HOME:-}" ] && candidates+=("$FORGE_HOME/scripts/extract-affected-files.sh") [ -n "${REPO_PATH:-}" ] && candidates+=("$REPO_PATH/scripts/extract-affected-files.sh") candidates+=("$PWD/scripts/extract-affected-files.sh") local candidate for candidate in "${candidates[@]}"; do if [ -f "$candidate" ]; then printf '%s\n' "$candidate" return 0 fi done echo "ERROR: extract-affected-files.sh is not installed in any configured runtime path." >&2 return 1 } AFFECTED_FILES_SCRIPT=$(resolve_extract_affected_files) || { echo "ERROR: cannot build file-overlap edges without extract-affected-files.sh" >&2 exit 1 } for NUM in {issue_numbers}; do echo "=== #$NUM ===" EXTRACT_OUT=$(bash "$AFFECTED_FILES_SCRIPT" "$NUM" -R "{GH_REPO}") FILE_SOURCE[$NUM]=$(echo "$EXTRACT_OUT" | head -1 | sed 's/^PROVENANCE=//') FILES_FOR_NUM=$(echo "$EXTRACT_OUT" | tail -n +2) ISSUE_FILES[$NUM]="$FILES_FOR_NUM" if [ "${FILE_SOURCE[$NUM]}" = "error" ]; then echo "ERROR: affected-file extraction for #$NUM was inconclusive after a GitHub/API failure." >&2 echo " Do not treat the issue as file-independent; retry the extraction or rerun /orchestrate." >&2 exit 1 fi echo "$FILES_FOR_NUM" echo " (source: ${FILE_SOURCE[$NUM]})" # Accumulate into the batch-wide array — read line-by-line so each extracted path # becomes one array element (paths here don't contain spaces, but this stays robust). while IFS= read -r f; do [ -n "$f" ] && LAYER1_FILES+=("$f") done <<< "$FILES_FOR_NUM" done ``` **Cross-reference all extracted file lists:** - If two issues share ANY affected file → one MUST be a predecessor of the other (serialized) - The issue with lower issue number goes first (stable ordering), unless an explicit `Depends on #` says otherwise - Add a conflict note to the DAG plan: "#{A} and #{B} both modify `{file}` — #{A} is predecessor of #{B}" - **Record the edge kind and shared file(s)** : alongside the predecessor relationship, set `EDGE_KIND["${A}:${B}"]="same-file"` and `EDGE_FILES["${A}:${B}"]="{file}"` (space-separated if multiple files overlap). Layer 1 is the only point where the specific overlapping file is known — Step 3D's DAG only records that a predecessor relationship exists, not why. `EDGE_KIND`/`EDGE_FILES` have two consumers in `phase-4-execution.md` Step 4B: (1) the DONE-case "same-file current-state brief" forwarding (forge#1860, unchanged), and (2) `verify_file_overlap_edge()` (forge#1904, extended to the DONE path by forge#2848), which re-checks this guessed file list against the predecessor's *actual* PR diff once it reaches FAILED, GATED, **or DONE** — since Layer 1's file list is extracted from a pre-build investigation guess or a raw issue-body parse, never from real code, it can be wrong, and a predecessor that concludes without ever touching the guessed file should not keep a dependent gated/skipped on it, nor hand it a same-file brief about a file it never wrote. See `phase-4-execution.md`'s "File-overlap edge re-verification" section for the full mechanism. #### Layer 2: Directory-proximity detection Two issues that modify different files in the **same leaf directory** have a high probability of conflicting through shared `__init__.py`, `index.ts`, barrel re-exports, or tightly coupled sibling modules. This layer catches indirect conflicts that Layer 1 misses. **Extract the leaf directory for each affected file:** ```bash # For each issue's file list, extract unique directories for NUM in {issue_numbers}; do echo "=== #$NUM directories ===" # From the file list already extracted in Layer 1: echo "$FILES_FOR_NUM" | xargs -I{} dirname {} | sort -u done ``` **Cross-reference directory lists:** - If two issues share a leaf directory AND that directory is "small" (contains fewer than 10 tracked files), flag as **probable conflict** → serialize - If the shared directory is a broad container (e.g., `services/api/app/routers/` with 15+ files), downgrade to **possible conflict** → serialize only if same domain tag (from Step 3B) - **Known high-conflict directories** (always serialize if shared): - `services/api/app/models/` (SQLAlchemy models often share Base, imports) - `services/api/app/core/` (shared dependencies) - `services/worker/worker/` (tightly coupled consumer modules) - `web/src/lib/` (shared utilities) - `shared/` (volume-mounted, affects all services) - `infra/migrations/` (already covered by DATABASE hard rule, but explicit here too) - **Record the edge kind and shared directory** : whenever this layer actually serializes a pair (small-directory match, broad-directory + same-domain match, or a known high-conflict directory), set `EDGE_KIND["${A}:${B}"]="directory"` and `EDGE_FILES["${A}:${B}"]="{shared_directory}"`. Same consumption contract as Layer 1 — read by Step 4B's same-file brief forwarding. #### Layer 3: Shared-module inference When two issues modify different files that **import from the same utility/init module**, both agents often end up modifying that shared module (adding imports, updating re-exports). This creates merge conflicts invisible to Layer 1. **Heuristic rules (no git operations needed — pattern-based):** | Pattern | Inference | Action | |---------|-----------|--------| | Issue A modifies `routers/billing.py`, Issue B modifies `routers/auth.py` | Both likely import from `routers/__init__.py` or `dependencies.py` | Flag as possible conflict if same service | | Issue A modifies `models/user.py`, Issue B modifies `models/subscription.py` *(Python/SQLAlchemy example)* | Both likely modify `models/__init__.py` (model registry) | Flag as probable conflict → serialize | | Issue A modifies `routes/users.js`, Issue B modifies `routes/billing.js` *(Node.js example)* | Both likely import from `routes/index.js` barrel | Flag as possible conflict if same service | | Issue A modifies `internal/user/handler.go`, Issue B modifies `internal/billing/handler.go` *(Go example)* | Both likely register in `internal/router/router.go` | Flag as possible conflict if same service | | Issue A modifies `web/src/components/X.tsx`, Issue B modifies `web/src/components/Y.tsx` | May share `index.ts` barrel export | Flag only if same parent directory | | Issue A modifies the app entrypoint (e.g. `main.py`, `main.go`, `server.js`, `app.ts`) | Entrypoint is a high-fan-in file (router registration, middleware) — set the path via `forge.yaml → review.layout.api_main` | Serialize with ANY other same-service issue | | Issue A modifies `docker-compose.yml` or `docker-compose.prod.yml` | Global config — any concurrent modification conflicts | Serialize with ALL other issues that touch infra | **Apply inferences:** ``` # High-fan-in files — if ANY issue touches these, serialize it with all same-service issues. # Read layout paths from forge.yaml review.layout; fall back to sensible generic defaults. # Example (pseudo-code — adapt to your forge.yaml parsing method): # API_MAIN = forge_yaml.review.layout.api_main ?? "src/main.py" # set in forge.yaml; no stack-specific default # WORKER_MAIN = forge_yaml.review.layout.worker_main ?? "src/worker.py" # set in forge.yaml; no stack-specific default # PAGES_ROOT = forge_yaml.review.layout.pages ?? "web/src/app" # Next.js default; override in forge.yaml for other frameworks HIGH_FAN_IN = [ API_MAIN, # app entrypoint — router/middleware registration (set forge.yaml review.layout.api_main) WORKER_MAIN, # worker entrypoint (set forge.yaml review.layout.worker_main) PAGES_ROOT + "/layout.tsx", # root layout for all pages (Next.js; adapt for your framework) "docker-compose.yml", "docker-compose.prod.yml", ".env.example" ] # For each issue, check if affected files include a high-fan-in file # If yes: that issue cannot be parallelized with any other issue touching the same service # Record edge metadata for any pair actually serialized by this layer : # EDGE_KIND["${A}:${B}"]="shared-module" # EDGE_FILES["${A}:${B}"]="{the shared high-fan-in file, or the inferred barrel/init/registry module}" # Same consumption contract as Layers 1-2 — read by Step 4B's same-file brief forwarding. ``` #### Layer 4: Conservative fallback (low-confidence cases) When file extraction yields **fewer than 2 file paths** for an issue (common for issues without investigation comments, or issues described in prose without backtick-wrapped paths), the conflict detection has low confidence. Rather than assuming independence, apply conservative serialization. **Rules:** - If an issue has 0-1 extracted file paths AND shares a domain tag (from Step 3B) with another issue → serialize them - If an issue has 0 extracted file paths AND no domain tag could be determined → add it as a predecessor of the next same-domain issue, or if no domain match exists, serialize it after the most recently added issue (safest default) - Add a note to the plan: "#{N} — low file-extraction confidence, serialized conservatively" **Rationale**: The cost of a false-negative (two agents conflict → one fails with merge error, wasting the full agent run) far exceeds the cost of a false-positive (an issue waits for one predecessor before starting). Always err toward serialization when uncertain. **Cohort-confidence guidance — N issues from one audit citing the same file is WEAK evidence, not strong** The rule above is correct and stays. What this guidance qualifies is *what counts as evidence* feeding it, because there is a cohort shape that turns the rule into a total order: When a **single audit, sweep, or analysis run files N issues at once**, those issues are written by one author against one mental model, so they all cite the same handful of files — as background, as "the thing this interacts with", as the example of the pattern. That shared citation reflects **the common origin of the issues, not a real overlap in the code each one will change**. Treat it as *low*-confidence evidence of conflict and prefer a `contract-deliverables` or `affected-files-section` list for every member of the cohort before serializing on it. **Why this matters more here than anywhere else in Step 3C**: a single-domain cohort compounds through the two rules above. Every issue in it shares a domain tag, and audit-filed issues are typically prose-heavy and pre-investigation, so most yield 0–1 extracted paths. That combination — `<2 paths` **and** shared domain — matches the first rule for *every pair in the cohort*, which degenerates to a total order over all N issues. Observed in batch `20260725T180538-orchestrate`: sixteen issues from one backup/DR audit produced a **13-deep serial chain**, an estimated 5–9 hours during which at most one agent could work that track while 16 concurrency slots sat idle. When the chain head merged, its actual diff touched **neither** of the two files the chain had been serialized on; re-splitting into five file-scoped groups dropped the critical path from ~11 to ~4 with no conflict. **Practical rule**: before applying conservative serialization across a cohort, check whether the shared file citation traces to a common filing origin (same audit, same day, same author, near-identical `## Context` prose). If it does, that citation is not independent corroboration from N issues — it is one claim repeated N times. Serialize on a real `EDGE_FILES` entry backed by a contract or a diff, not on the repetition. Layer 1's same-file **hard conflict** rule is unaffected — this guidance never weakens an edge backed by ground truth, and `phase-4-execution.md`'s DONE-arm re-verification is the automatic backstop that unwinds such a chain mid-batch when the first real diff disproves it. #### Layer 5: Historical co-change coupling Layers 1-4 infer conflict risk from structure — path overlap, directory nesting, hard-coded high-fan-in lists. They miss the case where two files with no directory or naming relationship have historically changed together in the same commits (e.g. `models/user.py` and `services/billing/charge.py`), and they over-serialize the inverse case where files merely sit near each other but have never actually co-changed. Git commit history answers both questions directly and empirically. This layer reads **commit metadata only** — the list of files touched per commit — never file contents, so it does not violate Hard Rule 2 ("dispatcher, not a builder... never read code"). **Primary path — persisted co-change matrix** (O(1) lookup, repo-wide coverage): Check whether `~/.forge/index/cochange.jsonl` (produced by `scripts/danger-zones.mjs`) exists before falling back to the live `git log` query. The matrix is repo-wide and covers file pairs outside the current batch — providing broader coupling signal than the batch-scoped live query alone. ```bash COCHANGE_INDEX="${HOME}/.forge/index/cochange.jsonl" COCHANGE_META="${HOME}/.forge/index/cochange-meta.json" if [ -f "$COCHANGE_INDEX" ]; then echo "Layer 5: co-change matrix found — using persisted index for pair lookups" LAYER5_SOURCE="matrix" TOTAL_COMMITS=$(cat "$COCHANGE_META" 2>/dev/null | python3 -c "import sys,json; d=json.load(sys.stdin); print(d.get('totalCommits',0))" 2>/dev/null || echo "0") else echo "Layer 5: co-change matrix absent — falling back to live git query (run 'node scripts/danger-zones.mjs' to build)" LAYER5_SOURCE="live" TOTAL_COMMITS=0 fi ``` **Matrix lookup per file pair** (when `LAYER5_SOURCE=matrix`): For each pair of files across issues in the batch, query the matrix for coupling verdict: ```bash # Query a single file pair from the matrix query_cochange_pair() { local FILE_A="$1" local FILE_B="$2" # Find the record for FILE_A (matrix stores each pair once in lexicographic order, # but also includes reverse-lookup entries for O(1) lookup from either side) local RECORD RECORD=$(grep -m1 "\"file\":\"${FILE_A}\"" "$COCHANGE_INDEX" 2>/dev/null || true) if [ -z "$RECORD" ]; then echo "unknown" # FILE_A not in matrix — insufficient history return fi # Extract n(A) — sum of monthly ring buffer local N_A N_A=$(echo "$RECORD" | python3 -c " import sys, json r = json.loads(sys.stdin.read()) n = r.get('n', [0,0,0]) print(sum(n)) " 2>/dev/null || echo "0") # Extract c(A,B) — co-occurrence count with FILE_B local C_AB C_AB=$(echo "$RECORD" | python3 -c " import sys, json r = json.loads(sys.stdin.read()) partners = r.get('partners', {}) c = partners.get('${FILE_B}', [0,0,0]) print(sum(c)) " 2>/dev/null || echo "0") # Apply thresholds: cold-start check (n < 5), support (c >= 3), confidence if [ "$N_A" -lt 5 ]; then echo "unknown" # Insufficient history for FILE_A return fi local FILE_B_RECORD FILE_B_RECORD=$(grep -m1 "\"file\":\"${FILE_B}\"" "$COCHANGE_INDEX" 2>/dev/null || true) local N_B=0 if [ -n "$FILE_B_RECORD" ]; then N_B=$(echo "$FILE_B_RECORD" | python3 -c " import sys, json r = json.loads(sys.stdin.read()) n = r.get('n', [0,0,0]) print(sum(n)) " 2>/dev/null || echo "0") fi if [ "$N_B" -lt 5 ]; then echo "unknown" # Insufficient history for FILE_B return fi # Use danger-zones.mjs --query for the authoritative normalization verdict # (handles ubiquity, directional confidence, companions — avoids re-implementing) VERDICT=$(node "$(git rev-parse --show-toplevel 2>/dev/null)/scripts/danger-zones.mjs" \ --query "$FILE_A" 2>/dev/null \ | python3 -c " import sys, json d = json.loads(sys.stdin.read()) partners = d.get('cochangePartners', []) for p in partners: if p['file'] == '${FILE_B}': print(p['verdict']) sys.exit(0) print('unknown') " 2>/dev/null || echo "unknown") echo "$VERDICT" } ``` **Fallback path — live git query** (when matrix is absent or file pair not found in matrix): ```bash # Union of affected files across all issues in the CURRENT batch only (already # extracted per-issue in Layer 1) — never the whole repo. Built as an array # (not a newline-joined scalar) so each path survives as a single pathspec # argument below — a plain string here would be word-split and glob-expanded # by the shell when handed to `git log --`, silently mangling or dropping any # path containing a space or glob metacharacter. mapfile -t ALL_AFFECTED_FILES < <(printf '%s\n' "${LAYER1_FILES[@]}" | sort -u | grep -v '^$') # Guard: an empty array expands to nothing after `--`, which git interprets as # "no pathspec restriction" (i.e. the whole repo) rather than "match nothing". # Skip the query entirely in that case instead of letting it silently widen to # a full-repo scan — this can happen when upstream file-extraction (Layer 1) # yields zero paths for every issue in the batch. # NOTE: `grep -v '^$'` above is required — without it, `printf '%s\n'` on an # empty array emits one blank line (printf runs its format string once even # with zero variadic args), which `mapfile` captures as a 1-element array of # [""], making this guard evaluate to false and causing `git log -- ""` to run. if [ "${#ALL_AFFECTED_FILES[@]}" -eq 0 ]; then echo "Layer 5: no affected files extracted by Layer 1 for this batch — skipping co-change query, falling back to Layers 1-4." else # Bounded window: last 90 days, capped at 200 commits — whichever is smaller. # Each commit's file list is delimited by a marker so co-occurring files can be # grouped per-commit in a single pass. The array is expanded quoted # ("${ALL_AFFECTED_FILES[@]}") so every path is passed as one literal argument. git log --name-only --since="90 days ago" --max-count=200 \ --pretty=format:'---%H---' -- "${ALL_AFFECTED_FILES[@]}" \ > /tmp/cochange_log.txt # Parse into commit → file-set groups, then increment a co-occurrence counter # for every unordered pair of files that appear in the SAME commit's file list. # (Illustrative — an agent executing this reads /tmp/cochange_log.txt and tallies # pairs; no separate script is shipped, matching the pseudo-code style of Layers 1-4.) fi ``` **Scoring rule**: A file pair is **co-change coupled** when it appears together in **3 or more** commits in the window. A pair with **zero** co-occurrences across the entire window (and both files have n ≥ 5 commits) is **verified independent**. Pairs where either file has fewer than 5 commits in the window are **unknown** — the matrix must NOT be used to downgrade edges for unknown pairs. **Apply the signal:** - **High co-change pair spans two different issues in the batch** → add a serialization edge between them (same directed-edge convention as Layers 1-4: lower issue number is predecessor), OR, if the pair also carries competing investigation recommendations, flag it for Phase 2.5 arbitration instead of a blind serialization edge (see cross-reference in Step 2.5B below). - **Verified-independent pair** → MAY be used to downgrade an existing Layer 2 "broad directory + different domain" or Layer 4 "conservative fallback" serialization to parallel. This downgrade is **never** applied to Layer 1 (same-file hard conflict, which is ground truth from the current batch, not historical inference) or to Layer 3 high-fan-in-file edges (a file can be structurally high-risk even with a thin history window, e.g. a newly added `main.py`). Ubiquitous-file pairs (n/N > 0.2 for either file) are **ineligible** for verified-independent downgrade even with zero co-occurrences. **Also ineligible**: a pair where either issue's `FILE_SOURCE[$NUM]` (Layer 1, above) is `body-fallback` — that file list came from a pre-investigation scrape of the raw issue body, lower-confidence than a post-investigation `affected-files-section` extraction, so a "verified independent" co-change verdict computed against it is not trustworthy enough to remove an edge. Weak provenance may only ever be used to *add* a conflict edge (Layer 1's own same-file cross-reference, unaffected by this rule), never to *remove* one — mirrors the existing "never overrides Layer 1/Layer 3" carve-out on this same line (forge#2436). `contract-deliverables` is **eligible** — like `affected-files-section` it is a **high**-confidence source, and strictly the stronger of the two (a deliverables table states intent to change rather than mere relevance), so it clears this bar for the same reason (forge#2848). Only `body-fallback` and `none` are excluded. - If `ALL_AFFECTED_FILES` is empty, the guard above skips the query and Layer 5 contributes nothing for the entire batch. If the matrix or live query returns no data for a pair → Layer 5 contributes nothing; fall back silently to Layers 1-4's existing verdict for that pair. **Wire-through proof (mandatory check)**: When `LAYER5_SOURCE=matrix`, confirm the matrix lookup path executes on at least one pair in the batch and log the verdict. This proves the path is live, not dead code. If no pairs are in the matrix, log that the live fallback ran instead. **Rationale**: The persisted matrix provides broader coverage (repo-wide, not batch-scoped) and O(1) lookup vs O(batch × commits) live query. The live fallback ensures no regression when the matrix is absent (cold start or first run). Bounding the live fallback to the batch file set keeps it cheap and deterministic. #### Combining all layers Build the final conflict graph by merging signals from all five layers: | Signal | Strength | Action | |--------|----------|--------| | Layer 1: Same file | **Hard conflict** | Always serialize | | Layer 2: Same small directory | **Probable conflict** | Serialize | | Layer 2: Same broad directory + same domain | **Probable conflict** | Serialize | | Layer 2: Same broad directory + different domain | **Possible conflict** | Parallelize (accept risk) — unless Layer 5 shows high co-change, then serialize | | Layer 3: High-fan-in file touched | **Probable conflict** | Serialize with same-service issues | | Layer 3: Shared model/init pattern | **Probable conflict** | Serialize | | Layer 4: Low confidence + same domain | **Conservative** | Serialize | | Layer 4: Low confidence + no domain | **Conservative** | Add predecessor edge to most recent issue — unless Layer 5 shows verified independence, then parallelize | | Layer 5: High co-change (3+ shared commits, cross-issue file pair) | **Probable conflict** | Serialize (or route to Phase 2.5 arbitration if a competing-recommendation conflict is also present) | | Layer 5: Verified independent (zero shared commits) | **Downgrade signal** | Permits downgrading a Layer 2 "broad directory + different domain" or Layer 4 verdict to parallel — never overrides Layer 1 or Layer 3 | **This supplements, not replaces, the domain keyword estimation.** Domain tags still help with broad sequencing decisions. Multi-layer conflict detection catches the specific cases keywords miss (e.g., two issues that both modify files in `services/api/app/models/` where one is labeled WORKER and the other BILLING — Layer 2 catches this even though Layer 1 shows no direct file overlap; or two issues touching files in unrelated directories that Layer 5 shows have co-changed in 4 of the last 12 commits touching either file). ### Step 3D: Build the dependency DAG Build a **directed acyclic graph (DAG)** of per-issue dependencies. Each issue gets a `predecessors` set — the specific issues that must reach a terminal state before this issue can dispatch. This replaces the previous wave-grouping model where all issues in a wave had to complete before any issue in the next wave could start. **DAG construction rules:** - Investigation issues already ran in Phase 2 — they are NOT included in this DAG - Each issue starts with an empty predecessor set - **Explicit dependencies**: If issue B says "Depends on #A" or "Blocked by #A", add A to B's predecessors - **File-conflict edges**: If two issues share affected files (from Step 3C Layer 1), add a directed edge: lower issue number → higher issue number (unless explicit deps say otherwise). The later issue has the earlier issue in its predecessors. - **Domain serialization edges**: DATABASE issues form a linear chain (each has the previous DATABASE issue as its predecessor). Same-small-directory issues (Layer 2) and high-fan-in file issues (Layer 3) get directed edges as per Step 3C rules. - **Edge-kind tagging for same-file/directory/shared-module briefing and re-verification** : Layer 1/2/3 edges are additionally tagged with `EDGE_KIND` (`same-file` / `directory` / `shared-module`) and `EDGE_FILES` at the point they're added (see Step 3C). Explicit-dependency edges, the DATABASE domain chain, Layer 4 conservative-fallback edges, and Layer 5 co-change edges deliberately do NOT receive one of these three `EDGE_KIND` values. This is what lets Step 4B distinguish "predecessor edge came from Step 3C Layer 1/2/3" from every other edge type, for two purposes: a same-file current-state brief (DONE case, forge#1860) and edge re-verification against the predecessor's actual PR diff once it reaches FAILED, GATED, or DONE (`verify_file_overlap_edge()`, forge#1904, DONE path added by forge#2848 — see `phase-4-execution.md` Step 4B). On the DONE path the two purposes are ordered: re-verification runs first and, on a `DROP`, suppresses the brief entirely. Non-`EDGE_KIND` edges are eligible for neither — they always `KEEP` in the re-verification check and are never candidates for the same-file brief. - **Conservative fallback edges**: Low-confidence issues (Layer 4) get edges to same-domain issues as per Step 3C rules. - **Co-change coupling edges** : High co-change file pairs (Layer 5, 3+ shared commits in the bounded window) that span two different issues get a directed edge using the same lower-issue-number-is-predecessor convention as Layer 1. Verified-independent pairs (Layer 5, zero shared commits) may instead REMOVE an edge that Layer 2 or Layer 4 would otherwise have added for that pair — Layer 1 and Layer 3 edges are never removed by a Layer 5 downgrade. - **Claims-board downgrade (Layer 2/4 edges only)** : After dispatch begins (Phase 4A), when both issues in a Layer-2 or Layer-4 serialized pair post `FORGE:CLAIM` annotations on the coordination issue and their claimed file sets are **disjoint** (no path appears in both claims), the serialization edge for that pair MAY be relaxed — the blocked issue becomes ready. This downgrade is **never** applied to Layer-1 (same-file) or Layer-3 (high-fan-in) edges. See Step 4B: Claims-board relaxation sweep for the runtime check. - **Concurrency is capped by default** — issues with empty predecessor sets are still all *eligible* to dispatch simultaneously (file overlap, explicit dependencies, and co-change coupling remain the only DAG-ordering constraints), but Phase 4's dispatch loop holds at most `MAX_CONCURRENT` in flight at once (default 12; `forge.yaml → orchestration.max_concurrent` overrides). Ready issues beyond the cap queue and dispatch as running workers complete (see Engine mode § Concurrency model, and `phase-4-execution.md` Step 4A-pre.0.2). **Materialize the DAG** : The rules above describe how edges are derived, but they must be applied into real, checkable data structures — not carried in prose or reconstructed from memory later. Step 3D.1 (coordination issue), Step 3D.5 (cycle detection), Step 3E.5 (scoring), and Step 3E (plan presentation) all read `ISSUES[]` and `PREDECESSORS[]` as if this already happened; this is the one place they're actually built: ```bash # --- Step 3D: Materialize the DAG --- ISSUES=({issue_numbers}) # all issues in this batch, excluding Phase 2 investigations declare -A PREDECESSORS # issue → space-separated predecessor issue numbers ("" = no predecessors) for NUM in "${ISSUES[@]}"; do PREDECESSORS[$NUM]="" # every issue gets an explicit entry — even an empty one — so # Step 3D.6's gate can tell "no predecessors" (valid) apart from # "never processed" (the bug this gate exists to catch) done # Apply edges in the order described above — explicit deps, then Layer 1/2/3 conflicts # (Step 3C), then the DATABASE domain chain, then Layer 4 conservative fallback, then # Layer 5 co-change coupling/downgrades. Each edge appends the predecessor's number to # the successor's entry: # PREDECESSORS[$SUCCESSOR]="${PREDECESSORS[$SUCCESSOR]:+${PREDECESSORS[$SUCCESSOR]} }$PREDECESSOR_NUM" # --- End Step 3D materialization --- ``` ### Step 3D.1: Create coordination issue (claims board) **When to run**: Immediately after DAG construction (Step 3D), before Step 3D.5 cycle detection. Run once per orchestration batch. Skip if `FORGE_COORD_ISSUE` is already set (e.g., resumed session). **Purpose**: Create a dedicated GitHub issue that serves as the shared claims board for the batch. Agents post `FORGE:CLAIM` annotations here when they begin implementation; they post `FORGE:CLAIM_RELEASED` when they reach a terminal state. The orchestrator reads active claims during the Layer-2/4 relaxation sweep (Step 4B) to determine whether serialized pairs can now run in parallel. ```bash # Create coordination issue for this orchestration batch. # Guarded on FORGE_COORD_ISSUE being unset: this is the actual enforcement of the # "Skip if FORGE_COORD_ISSUE is already set" contract stated above. Without this guard, # a re-entry into this block within the same logical session (resumed session, retry, # or any other re-run of Step 3D.1) would silently regenerate BATCH_ID — and because # check_orchestrator_lease() (Step 3D.2, forge#2627) keys "self" vs. "held" purely on # BATCH_ID string equality, a regenerated BATCH_ID causes the orchestrator's own # subsequent lease-refresh calls to see HOLDER_BATCH_ID != MY_BATCH_ID and self-lock-out # against its own still-live lease. if [ -z "${FORGE_COORD_ISSUE:-}" ]; then BATCH_ISSUE_COUNT="${#ISSUES[@]}" BATCH_ID="$(date -u +%Y%m%dT%H%M%S)-$$" COORD_ISSUE_BODY="## Orchestration Batch Claims Board This issue is the claims board for an orchestration batch of ${BATCH_ISSUE_COUNT} issues. Agents post \`FORGE:CLAIM\` here on build start and \`FORGE:CLAIM_RELEASED\` on terminal state. **Batch ID**: ${BATCH_ID} **Issues in batch**: ${ISSUES[*]/#/#} **Created**: $(date -u +%Y-%m-%dT%H:%M:%SZ) " # GOVERNOR-exempt: intentional coordination side-effect (best-effort lease/board/finding post), DRY_RUN-safe — reviewed & accepted for the check-command-side-effects gate. Flagged only by the staging->main full-diff; passes on every feature PR. forge#2627 COORD_ISSUE_URL=$(gh issue create -R {GH_REPO} \ --title "orchestrate: claims board for batch ${BATCH_ID}" \ --body "$COORD_ISSUE_BODY" \ --label "automation" 2>/dev/null || echo "") if [ -z "$COORD_ISSUE_URL" ]; then echo "WARNING: failed to create coordination issue — claims board disabled for this batch. Layer-2/4 relaxation will not run." FORGE_COORD_ISSUE="" else COORD_ISSUE_NUMBER=$(echo "$COORD_ISSUE_URL" | grep -oE '[0-9]+$') FORGE_COORD_ISSUE="$COORD_ISSUE_URL" echo "Coordination issue created: ${COORD_ISSUE_URL} (#${COORD_ISSUE_NUMBER})" export FORGE_COORD_ISSUE export COORD_ISSUE_NUMBER # BATCH_ID must survive compaction the same way FORGE_COORD_ISSUE/COORD_ISSUE_NUMBER do — # exporting it is necessary but not sufficient (export only survives within the *same* # process tree). The `` marker embedded in the issue body above # is the actual durable source of truth: see "Orchestrator state reconstruction on wake / # after compaction" below, which re-derives BATCH_ID from GitHub rather than trusting the # in-context variable, consistent with that section's own "do not rely on in-context # variables" contract. export BATCH_ID fi else echo "FORGE_COORD_ISSUE already set (${FORGE_COORD_ISSUE}) — skipping coordination-issue creation and BATCH_ID regeneration. Reusing existing batch identity." fi ``` **Idempotency**: If `FORGE_COORD_ISSUE` is already set in the environment (e.g., after a compaction / orchestrator restart, or any other re-entry into this block within the same batch), the bash block above skips creation entirely and reuses the existing `FORGE_COORD_ISSUE`/`BATCH_ID` — this is enforced by the `if [ -z "${FORGE_COORD_ISSUE:-}" ]` guard, not just documented in prose. The coordination issue persists for the lifetime of the batch. **Terminology:** - **Ready issues**: Issues whose predecessor set is empty (all predecessors have reached terminal state or were never added) - **Blocked issues**: Issues with one or more unresolved predecessors - **Critical path**: The longest chain of dependent issues in the DAG — determines minimum wall-clock time **Example DAG:** ``` Phase 2 (already done): #2644 (investigation) → spawned #2645, #2646, #2647 Dependency graph: #2633 (orphaned queues) → predecessors: {} ← READY #2636 (invalidation sub) → predecessors: {} ← READY #2645 (new finding) → predecessors: {} ← READY #2646 (new finding) → predecessors: {} ← READY #2634 (enable daemon) → predecessors: {#2633} ← blocked until #2633 completes #2647 (depends on #2645) → predecessors: {#2645} ← blocked until #2645 completes Critical path: #2633 → #2634 (2 steps) or #2645 → #2647 (2 steps) Initial dispatch: #2633, #2636, #2645, #2646 (all ready — launched simultaneously) ``` **Key advantage over waves**: When #2633 completes, #2634 dispatches immediately — it does not wait for #2636, #2645, or #2646 to finish. Similarly, when #2645 completes, #2647 dispatches immediately regardless of other issues' status. ### Step 3D.2: Acquire orchestrator lease (MANDATORY) **WHY THIS EXISTS**: The coordination issue created in Step 3D.1 exists purely for per-agent `FORGE:CLAIM` file-overlap serialization — nothing checks whether another live orchestrator instance is already dispatching against an overlapping issue set. Two concurrent loops (a stale survivor plus a restart, or two independent invocations) then both re-derive a ready set from GitHub and both dispatch, producing duplicate/backlog dispatch and truncating each other's view of local state. This step turns the same coordination issue into a single-instance lease, using the append-only comment stream (same idiom as `FORGE:CLAIM`/`FORGE:CLAIM_RELEASED`) rather than a body edit, so it does not reintroduce the read-then-write TOCTOU race documented in #2512. **Lease identity**: The lease holder is keyed on `BATCH_ID` (from Step 3D.1 — stable for the lifetime of this batch, including across compaction/wake within the *same* top-level session), not a fresh PID per invocation. This is deliberate: the same orchestrator resuming after its own compaction must always be able to refresh its own lease, never treated as a competing holder. **`check_orchestrator_lease()`** — shared helper, called from both the live Step 4A dispatch entry point (`phase-4-execution.md`) and the wake/compaction reconstruction block below. Declared once here; re-declare byte-identically wherever this file's context is not already sourced (same convention as `classify_predecessor_state()`/`verify_file_overlap_edge()` in `phase-4-execution.md` Step 4B). ```bash # Default lease TTL: 15 minutes. Refreshed once per dispatch chunk in phase-4-execution.md # Step 4A — generous enough to tolerate a normal chunk's wall-clock time without a false # takeover, short enough that a genuinely dead orchestrator's lease goes stale promptly. LEASE_TTL_SECONDS="${LEASE_TTL_SECONDS:-900}" # check_orchestrator_lease # Returns via stdout one of: "self" (lease already held by this BATCH_ID — safe to refresh), # "free" (no live lease — safe to acquire), "held:" (a different, unexpired # lease is held — do NOT dispatch). check_orchestrator_lease() { local COORD_NUM="$1" local MY_BATCH_ID="$2" # Last FORGE:LEASE / FORGE:LEASE_RELEASED comment, in chronological order, to determine # current state. Both are HTML-comment-tagged issue comments (never a body edit) — an # append-only log the same way FORGE:CLAIM/FORGE:CLAIM_RELEASED already work in this file. # # --paginate is REQUIRED here (not optional/cosmetic): the coordination issue also # accumulates one comment per FORGE:CLAIM/FORGE:CLAIM_RELEASED pair (Step 4B) plus one # FORGE:LEASE heartbeat per dispatch chunk (Step 4A in phase-4-execution.md) and per # wake/resume. GitHub's default comments listing returns only the first 30 (oldest) # comments without --paginate — on any batch with more than a handful of issues or a # couple of heartbeat cycles, `last` over an unpaginated page silently reads a STALE # lease event instead of the true most recent one, which can either falsely trigger the # "REFUSING TO DISPATCH" exit 1 below using stale holder data, or fail to see a genuinely # live competing lease and defeat the entire point of this mechanism. local LAST_LEASE_EVENT LAST_LEASE_EVENT=$(gh api --paginate "repos/{GH_REPO}/issues/${COORD_NUM}/comments" \ --jq '[.[] | select(.body | contains("FORGE:LEASE"))] | last | {body: .body, created_at: .created_at}' 2>/dev/null || echo "") if [ -z "$LAST_LEASE_EVENT" ] || [ "$LAST_LEASE_EVENT" = "null" ]; then echo "free" return fi local LEASE_IS_RELEASE LEASE_IS_RELEASE=$(echo "$LAST_LEASE_EVENT" | jq -r '.body | contains("FORGE:LEASE_RELEASED")' 2>/dev/null || echo "false") if [ "$LEASE_IS_RELEASE" = "true" ]; then echo "free" return fi local HOLDER_BATCH_ID LEASE_TIMESTAMP HOLDER_BATCH_ID=$(echo "$LAST_LEASE_EVENT" | jq -r '.body' 2>/dev/null | grep -oE '\*\*Holder Batch ID\*\*: [^[:space:]]+' | head -1 | sed -E 's/^\*\*Holder Batch ID\*\*: //') LEASE_TIMESTAMP=$(echo "$LAST_LEASE_EVENT" | jq -r '.created_at' 2>/dev/null) if [ "$HOLDER_BATCH_ID" = "$MY_BATCH_ID" ]; then echo "self" return fi # Staleness check — a lease older than LEASE_TTL_SECONDS with no refresh is treated as free. # A `date -d` parse failure must NOT silently fall through to "free" (LEASE_EPOCH=0 would # make AGE huge and every live lease report as free/expired, defeating the lease entirely # with no warning). Distinguish "genuinely parsed as epoch 0" from "failed to parse" by # checking the command's exit status explicitly, and fail SAFE (treat as held/unexpired) # rather than fail-open on a parse error. local LEASE_EPOCH if ! LEASE_EPOCH=$(date -u -d "$LEASE_TIMESTAMP" +%s 2>/dev/null); then echo "WARNING: check_orchestrator_lease(): failed to parse lease timestamp '${LEASE_TIMESTAMP}' — treating as held (fail-safe, not fail-open) to avoid masking a live lease." >&2 echo "held:${HOLDER_BATCH_ID}" return fi local NOW_EPOCH AGE NOW_EPOCH=$(date -u +%s) AGE=$((NOW_EPOCH - LEASE_EPOCH)) if [ "$AGE" -gt "$LEASE_TTL_SECONDS" ]; then echo "free" else echo "held:${HOLDER_BATCH_ID}" fi } ``` **Acquire (or refresh) the lease** — run once per batch, immediately after Step 3D.1's coordination-issue creation/reuse, before Step 3D.5's cycle detection: ```bash if [ -n "${FORGE_COORD_ISSUE:-}" ] && [ -n "${COORD_ISSUE_NUMBER:-}" ]; then LEASE_STATE=$(check_orchestrator_lease "$COORD_ISSUE_NUMBER" "$BATCH_ID") case "$LEASE_STATE" in free|self) HOSTNAME_ID=$(hostname 2>/dev/null || echo "unknown-host") # GOVERNOR-exempt: intentional coordination side-effect (best-effort lease/board/finding post), DRY_RUN-safe — reviewed & accepted for the check-command-side-effects gate. Flagged only by the staging->main full-diff; passes on every feature PR. forge#2627 gh issue comment "$COORD_ISSUE_NUMBER" -R {GH_REPO} --body " **Holder Batch ID**: ${BATCH_ID} **Holder**: ${HOSTNAME_ID} (pid ${$}) **Acquired/refreshed**: $(date -u +%Y-%m-%dT%H:%M:%SZ) **TTL**: ${LEASE_TTL_SECONDS}s (refreshed once per dispatch chunk — a lease with no refresh past this window is considered stale and may be taken over)" 2>/dev/null || \ echo "WARNING: failed to post FORGE:LEASE — continuing without a lease record (best-effort primitive, not a hard blocker on a gh API hiccup)" echo "Orchestrator lease acquired/refreshed for batch ${BATCH_ID} on coordination issue #${COORD_ISSUE_NUMBER}" ;; held:*) HELD_BY="${LEASE_STATE#held:}" echo "REFUSING TO DISPATCH: an unexpired orchestrator lease for this batch is already held by batch ${HELD_BY} on coordination issue #${COORD_ISSUE_NUMBER}." echo "Another live /orchestrate instance (or a not-yet-stale prior run) appears to be dispatching this batch already." echo "If you are certain the other instance is dead (not just idle), wait ${LEASE_TTL_SECONDS}s for the lease to go stale, or manually post a FORGE:LEASE_RELEASED comment on #${COORD_ISSUE_NUMBER} to force a takeover." exit 1 ;; *) # Defensive default (MANDATORY — do not remove): if check_orchestrator_lease() is not # re-declared correctly in a fresh context, or otherwise returns anything other than # free/self/held:*, this branch must fail LOUD, not silently no-op. Silently falling # through here would be worse than the documented "no coordination issue" fail-open # path above — it would look like the lease gate ran and passed when it never # evaluated a real state. echo "WARNING: check_orchestrator_lease() returned unexpected value '${LEASE_STATE}' — lease gate could not be evaluated. Proceeding without a confirmed lease (best-effort primitive) but this indicates a bug in check_orchestrator_lease() or its re-declaration; investigate rather than ignore." >&2 ;; esac else echo "INFO: no coordination issue available (Step 3D.1 failed or was skipped) — lease enforcement disabled for this batch. Proceeding without a single-instance guard." fi ``` **Known limitation (documented, not silently overclaimed)**: this is a best-effort lease built on GitHub issue comments, not a distributed-consensus lock. A genuinely simultaneous race — two orchestrators both calling `check_orchestrator_lease()` and both observing `free` before either posts `FORGE:LEASE` — has a narrow window it does not close. This is an accepted limitation of a prose/bash spec; it converts the common case (a stale survivor, a restart with the prior loop still technically alive, two deliberately-separate invocations) from silent parallel dispatch into a clear refusal, which is the actual failure mode reported in this issue. **Known limitation — spoofed lease comment (accepted risk, no authorship check)** : `check_orchestrator_lease()` trusts `**Holder Batch ID**` out of the *body* of the last `FORGE:LEASE`/`FORGE:LEASE_RELEASED` comment with no `comment.user.login` verification — any account with comment-write access to the coordination issue can post a spoofed `FORGE:LEASE` comment and force every legitimate orchestrator instance into the `held:*` refusal branch (`exit 1`) for up to `LEASE_TTL_SECONDS` (default 900s), repeatably. This is a deliberate, accepted tradeoff, not an oversight: (1) exploiting it already requires an existing repo collaborator, who has far higher-impact avenues than a bounded, self-expiring dispatch refusal; (2) the obvious fix — checking the comment author against an "expected" identity — would break legitimate leases posted by a second authorized teammate running `/orchestrate` under their own `gh auth` session, trading a bounded/self-healing DoS for a silent, unbounded correctness bug; (3) `FORGE:CLAIM`/`FORGE:CLAIM_RELEASED` (`phase-4-execution.md`) already uses the identical no-authorship-check, body-only trust model for the same reason and predates this lease mechanism. No authorship check is planned for either mechanism. ### Step 3D.5: Cycle Detection (MANDATORY) **Run immediately after Step 3D's DAG edge construction, before presenting the plan (Step 3E).** This step validates that the predecessor graph is acyclic. Without it, mutual `Depends on` declarations (e.g., A depends on B AND B depends on A) cause both issues to remain permanently blocked in Step 4B's dispatch loop — no error, no timeout, indefinite deadlock. **Algorithm**: Kahn's topological sort. Runs in O(V+E) — negligible overhead for typical batch sizes. ```bash # --- Step 3D.5: Cycle Detection --- # Inputs: # ISSUES[] — array of all issue numbers in the DAG # PREDECESSORS[N] — space-separated list of predecessor issue numbers for issue N # Step 1: Compute in-degree for each issue declare -A IN_DEGREE for NUM in "${ISSUES[@]}"; do IN_DEGREE[$NUM]=0 done for NUM in "${ISSUES[@]}"; do for PRED in ${PREDECESSORS[$NUM]:-}; do IN_DEGREE[$NUM]=$(( ${IN_DEGREE[$NUM]:-0} + 1 )) done done # Step 2: Seed the queue with zero-in-degree issues (no predecessors) KAHN_QUEUE=() for NUM in "${ISSUES[@]}"; do [ "${IN_DEGREE[$NUM]}" -eq 0 ] && KAHN_QUEUE+=("$NUM") done # Step 3: Process queue — reduce successor in-degrees, enqueue newly freed issues PROCESSED_COUNT=0 PROCESSED_ORDER=() while [ "${#KAHN_QUEUE[@]}" -gt 0 ]; do # Dequeue CURRENT="${KAHN_QUEUE[0]}" KAHN_QUEUE=("${KAHN_QUEUE[@]:1}") PROCESSED_ORDER+=("$CURRENT") PROCESSED_COUNT=$(( PROCESSED_COUNT + 1 )) # Reduce in-degree of all issues that depend on CURRENT (i.e., CURRENT is in their PREDECESSORS) for SUCCESSOR in "${ISSUES[@]}"; do for PRED in ${PREDECESSORS[$SUCCESSOR]:-}; do if [ "$PRED" = "$CURRENT" ]; then IN_DEGREE[$SUCCESSOR]=$(( ${IN_DEGREE[$SUCCESSOR]} - 1 )) [ "${IN_DEGREE[$SUCCESSOR]}" -eq 0 ] && KAHN_QUEUE+=("$SUCCESSOR") fi done done done # Step 4: Any issue not processed has in-degree > 0 — part of a cycle CYCLE_ISSUES=() EXCLUDED_CYCLE=() for NUM in "${ISSUES[@]}"; do FOUND=false for P in "${PROCESSED_ORDER[@]}"; do [ "$P" = "$NUM" ] && FOUND=true && break; done [ "$FOUND" = "false" ] && CYCLE_ISSUES+=("$NUM") done # Step 5: Handle cycles if [ "${#CYCLE_ISSUES[@]}" -gt 0 ]; then echo "CYCLE DETECTED in dependency graph — the following issues form a circular dependency:" for C in "${CYCLE_ISSUES[@]}"; do echo " #${C}: predecessors=[${PREDECESSORS[$C]}]" # Label each cyclic issue needs-human gh issue edit "$C" -R {GH_REPO} --add-label "needs-human" 2>/dev/null || true gh issue comment "$C" -R {GH_REPO} --body "**Cycle detected by /orchestrate**: This issue is part of a circular dependency chain involving issues: ${CYCLE_ISSUES[*]/#/#}. The orchestrator cannot dispatch it automatically. Please fix the \`Depends on\` declarations so that no cycle exists, then re-run /orchestrate." 2>/dev/null || true # Remove from DAG — store in EXCLUDED_CYCLE for Step 3E reporting EXCLUDED_CYCLE+=("$C") # Remove from ISSUES array for all downstream processing # Use exact-match filter loop — pattern substitution (${array[@]/pattern}) leaves blank # slots and corrupts partial matches (e.g., removing 100 changes 1000 to 0). NEW_ISSUES=() for I in "${ISSUES[@]}"; do [ "$I" != "$C" ] && NEW_ISSUES+=("$I") done ISSUES=("${NEW_ISSUES[@]}") done echo "" echo "These issues have been labeled needs-human and excluded from the DAG." echo "Fix their dependency declarations and re-run /orchestrate." else echo "DAG cycle check: PASS — no cycles detected. Proceeding with ${#PROCESSED_ORDER[@]} issues." fi # Guard: if all issues were cyclic, ISSUES[] is now empty — abort before presenting an empty plan if [ "${#ISSUES[@]}" -eq 0 ]; then echo "" echo "ERROR: All issues in this batch form circular dependencies and have been excluded." echo "Every issue has been labeled needs-human." echo "Fix the Depends on / Blocked by declarations so no cycle exists, then re-run /orchestrate." exit 1 fi # --- End Step 3D.5 --- ``` **After this step**: - `ISSUES[]` contains only acyclic issues — safe to dispatch - `EXCLUDED_CYCLE[]` contains cyclic issue numbers — reported in Step 3E, never dispatched - If `EXCLUDED_CYCLE` is non-empty, report it clearly in the Step 3E plan before asking for user confirmation - If `ISSUES[]` is empty after cycle exclusion (all issues were cyclic), the guard above aborts with `exit 1` — Step 3E is never reached with an empty plan - Proceed next to **Step 3D.6** before Step 3E.5's scoring pass. ### Step 3D.6: Phase 3 Completion Gate (MANDATORY) **Run immediately after Step 3D.5's cycle detection, before Step 3E.5's scoring pass and Step 3E's plan presentation.** Steps 3D.1, 3D.5, 3E.5, and 3E all read `ISSUES[]`, `PREDECESSORS[]`, and `ISSUE_DOMAIN[]` as if Step 3B/3D already built them for real. Without this gate, an orchestrator under time or context pressure (e.g. a 73-issue batch) can skip the actual extraction loops in Steps 3A–3D and hand-write a plausible-looking DAG from memory instead — Step 3E.5's scoring and Phase 4's dispatch would then run against fabricated data with no error, because the `${PREDECESSORS[$NUM]:-}` defaulting used throughout Step 3D.5 silently tolerates a missing entry instead of failing loudly. **Check**: every issue in `ISSUES[]` must have an explicit key in `PREDECESSORS` and in `ISSUE_DOMAIN` — checked by **key presence**, not by whether the value is non-empty. An issue with zero real predecessors (`PREDECESSORS[$NUM]=""`) or no matched domain (`ISSUE_DOMAIN[$NUM]="NONE"`) is a valid, common outcome and must NOT trip this gate — only a genuinely *absent* key means the upstream step never ran for that issue. ```bash # --- Step 3D.6: Phase 3 Completion Gate --- MISSING_PREDECESSORS=() MISSING_DOMAIN=() for NUM in "${ISSUES[@]}"; do # Key-presence test (${arr[key]+x}), NOT emptiness — PREDECESSORS[$NUM]="" and # ISSUE_DOMAIN[$NUM]="NONE" are valid values and must pass this check. [ -z "${PREDECESSORS[$NUM]+x}" ] && MISSING_PREDECESSORS+=("$NUM") [ -z "${ISSUE_DOMAIN[$NUM]+x}" ] && MISSING_DOMAIN+=("$NUM") done if [ "${#MISSING_PREDECESSORS[@]}" -gt 0 ] || [ "${#MISSING_DOMAIN[@]}" -gt 0 ]; then echo "FATAL: Phase 3 DAG construction is incomplete — cannot proceed to Step 3E.5, Step 3E, or Phase 4." [ "${#MISSING_PREDECESSORS[@]}" -gt 0 ] && echo " Missing PREDECESSORS[] entry for: ${MISSING_PREDECESSORS[*]/#/#}" [ "${#MISSING_DOMAIN[@]}" -gt 0 ] && echo " Missing ISSUE_DOMAIN[] entry for: ${MISSING_DOMAIN[*]/#/#}" echo "This means Step 3B (domain estimation) and/or Step 3D (DAG edge construction) were" echo "skipped, or did not run for every issue in the batch. Do NOT hand-write the missing" echo "entries from memory or by re-reading issue titles — go back and run Steps 3A-3D for" echo "the listed issues, then re-run this gate before proceeding." exit 1 else echo "Phase 3 completion gate: PASS — ${#ISSUES[@]} issues all have PREDECESSORS[] and ISSUE_DOMAIN[] entries." fi # --- End Step 3D.6 --- ``` **After this gate passes**: Step 3E.5 (scoring) and Step 3E (plan presentation) may proceed. Do not present the Step 3E plan, run Step 3E.5, or hand off to Phase 4 while this gate is failing. ### Step 3E.5: Value/Cost Scoring Pass (MANDATORY) **Run immediately after Step 3D.5, before presenting the plan (Step 3E).** This step scores every issue in `ISSUES[]` by its expected value/cost ratio and re-orders the ready-set (issues with an empty predecessors set) in descending value/cost order. Dependency constraints are **never overridden** — this is a reordering pass within the existing ready-set only, not an edge-insertion pass. No new edges are added; cycle detection (Step 3D.5) has already completed. **Purpose**: Ensure that when a budget is finite (see `--budget` in Phase 4), the highest-value-per-token work dispatches first. When no budget is set (the default, uncapped behavior), dispatch order still reflects value/cost — useful for observability even without a hard cap. #### Value function (transparent heuristic — deferral decisions must be explainable) ``` value(issue) = priority_weight × danger_zone_weight ``` **Priority weight** (from issue labels — matches both the canonical `priority:P` form and the bare `P` form some consumer repos use for externally/legacy-labeled issues; see `phase-1-resolve.md`'s "Priority label schema" note. `priority:P` wins if both are present on the same issue): | Label | Weight | |-------|--------| | `priority:P0` or `P0` | 4.0 | | `priority:P1` or `P1` | 3.0 | | `priority:P2` or `P2` | 2.0 | | `priority:P3` or `P3` | 1.0 | | *(neither form present)* | 1.5 | **Danger-zone weight** (from affected files via FORGE:INVESTIGATOR comment): Read the `### Affected Files` section and check each file path against the danger-zone list from `forge.yaml → review.danger_zones[]`. Each affected file that appears in a danger zone adds 0.5 to the weight (additive, capped at 2.0). Default (no matches): 1.0. ``` danger_zone_weight = min(2.0, 1.0 + 0.5 × count_of_danger_zone_files_affected) ``` If `forge.yaml → review.danger_zones` is absent: danger_zone_weight = 1.0 for all issues. #### Cost function (fallback hierarchy) ``` cost_estimate(issue) → expected_spend_usd ``` Resolve in this order (use the first that produces a non-null result): 1. **Cost-prior lookup** (primary): Read `~/.forge/index/cost-priors.json`. Compute key = `task_type:module` where: - `task_type` = FORGE:INVESTIGATOR `**Task Type**` field (lower-cased, spaces→hyphens). If no investigator comment: infer from issue labels (`feature` → `feature`, `bug` → `bug-fix`, else `unknown`). - `module` = basename (no ext, lowercase) of the primary affected file from FORGE:INVESTIGATOR. If absent: `_unknown`. - If the key exists in cost-priors.json: use `priors[key].mean`. Mark the issue as `has_prior: true` for exploration-reserve logic below. 2. **Label heuristic fallback** (when cost-priors.json absent or key not found): ``` bug/fix: $0.20 · feature: $0.40 · refactor: $0.30 · investigation: $0.50 · unknown: $0.35 ``` Mark the issue as `has_prior: false`. 3. **File-count proxy** (last resort — no labels and no prior): ``` estimated_cost = 0.10 + 0.05 × count_of_affected_files ``` Mark the issue as `has_prior: false`. #### Scoring and sorting ```bash # --- Step 3E.5: Value/Cost Scoring Pass --- # Requires: ISSUES[] (post-cycle-detection), PREDECESSORS[], GH_REPO # Outputs: ISSUE_SCORE[], ISSUE_COST_ESTIMATE[], ISSUE_HAS_PRIOR[], SORTED_READY_SET[] declare -A ISSUE_SCORE # issue → value/cost ratio (float) declare -A ISSUE_COST_ESTIMATE # issue → estimated cost (USD float string) declare -A ISSUE_HAS_PRIOR # issue → true|false declare -A ISSUE_VALUE # issue → value weight (float) COST_PRIORS_PATH="${HOME}/.forge/index/cost-priors.json" for NUM in "${ISSUES[@]}"; do # 1. Fetch issue data for scoring (labels, investigator comment) ISSUE_DATA=$(gh issue view "$NUM" -R {GH_REPO} --json labels,body \ --jq '{labels: [.labels[].name]}' 2>/dev/null || echo '{"labels":[]}') LABELS=$(echo "$ISSUE_DATA" | jq -r '.labels[]' 2>/dev/null || echo '') # --- Value: priority weight --- # Schema-tolerant (forge#2232): matches canonical "priority:P" (whole-line, via -E # anchors so "priority:P0" doesn't also satisfy a "priority:P0x"-style label) or bare # "P". $LABELS is already a flat newline-separated list of label names (not a JSON # array), so this stays a grep -E check rather than jq test() — no engine-mismatch risk # since there is no jq counterpart mirrored at this specific site. if echo "$LABELS" | grep -qE "^priority:P0$|^P0$"; then PRIO_WEIGHT=4.0 elif echo "$LABELS" | grep -qE "^priority:P1$|^P1$"; then PRIO_WEIGHT=3.0 elif echo "$LABELS" | grep -qE "^priority:P2$|^P2$"; then PRIO_WEIGHT=2.0 elif echo "$LABELS" | grep -qE "^priority:P3$|^P3$"; then PRIO_WEIGHT=1.0 else PRIO_WEIGHT=1.5 fi # --- Value: danger-zone weight --- DANGER_WEIGHT=1.0 DANGER_ZONES=$(yq '.review.danger_zones[]? // ""' forge.yaml 2>/dev/null || echo '') if [ -n "$DANGER_ZONES" ]; then # Fetch affected files from INVESTIGATOR comment AFFECTED=$(gh api repos/{GH_REPO}/issues/${NUM}/comments \ --jq '.[] | select(.body | contains("FORGE:INVESTIGATOR")) | .body' 2>/dev/null \ | grep -oP '`[^`]+\.(py|mjs|ts|md|sh|yaml|yml)`' | tr -d '`' | head -20 || echo '') ZONE_HIT_COUNT=0 while IFS= read -r dz; do [ -z "$dz" ] && continue if echo "$AFFECTED" | grep -q "$dz"; then ZONE_HIT_COUNT=$((ZONE_HIT_COUNT + 1)) fi done <<< "$DANGER_ZONES" DZ_ADD=$(echo "scale=1; if ($ZONE_HIT_COUNT * 0.5 > 1.0) 1.0 else $ZONE_HIT_COUNT * 0.5" | bc 2>/dev/null || echo "0") DANGER_WEIGHT=$(echo "scale=1; 1.0 + $DZ_ADD" | bc 2>/dev/null || echo "1.0") fi VALUE=$(echo "scale=4; $PRIO_WEIGHT * $DANGER_WEIGHT" | bc 2>/dev/null || echo "$PRIO_WEIGHT") ISSUE_VALUE[$NUM]="$VALUE" # --- Cost: prior lookup → label heuristic → file-count proxy --- COST="" HAS_PRIOR="false" if [ -f "$COST_PRIORS_PATH" ]; then # Derive task_type:module key TASK_TYPE=$(gh api repos/{GH_REPO}/issues/${NUM}/comments \ --jq '[.[] | select(.body | contains("FORGE:INVESTIGATOR")) | .body] | last // ""' 2>/dev/null \ | grep -oP '(?<=\*\*Task Type\*\*: )\S+' | head -1 | tr '[:upper:]' '[:lower:]' \ | tr ' ' '-' || echo '') [ -z "$TASK_TYPE" ] && { if echo "$LABELS" | grep -q "^feature$"; then TASK_TYPE="feature" elif echo "$LABELS" | grep -q "^bug$"; then TASK_TYPE="bug-fix" else TASK_TYPE="unknown"; fi } PRIMARY_FILE=$(gh api repos/{GH_REPO}/issues/${NUM}/comments \ --jq '[.[] | select(.body | contains("FORGE:INVESTIGATOR")) | .body] | last // ""' 2>/dev/null \ | grep -oP '`[^`]+\.(py|mjs|ts|md|sh|yaml|yml)`' | tr -d '`' | head -1 || echo '') MODULE=$(basename "${PRIMARY_FILE:-_unknown}" | sed 's/\.[^.]*$//' | tr '[:upper:]' '[:lower:]') [ -z "$MODULE" ] && MODULE="_unknown" PRIOR_KEY="${TASK_TYPE}:${MODULE}" COST=$(jq -r --arg k "$PRIOR_KEY" '.priors[$k].mean // empty' "$COST_PRIORS_PATH" 2>/dev/null || echo '') [ -n "$COST" ] && HAS_PRIOR="true" fi if [ -z "$COST" ]; then # Label heuristic fallback if echo "$LABELS" | grep -q "^feature$"; then COST="0.40" elif echo "$LABELS" | grep -q "^bug$"; then COST="0.20" elif echo "$LABELS" | grep -q "^refactor$"; then COST="0.30" else COST="0.35"; fi fi # Score = value / cost (protected against divide-by-zero) SCORE=$(echo "scale=4; if ($COST > 0) $VALUE / $COST else $VALUE / 0.01" | bc 2>/dev/null || echo "1.0") ISSUE_SCORE[$NUM]="$SCORE" ISSUE_COST_ESTIMATE[$NUM]="$COST" ISSUE_HAS_PRIOR[$NUM]="$HAS_PRIOR" echo "Score: #${NUM} value=${VALUE} cost_est=\$${COST} (prior=${HAS_PRIOR}) score=${SCORE}" done # --- ε-exploration reserve --- # ε = 10% of budget allocated to no-prior issues (high-variance unknowns). # A no-prior issue is guaranteed a dispatch slot within the ε reserve even if # its score would otherwise place it below the budget cutoff. This prevents # discovery starvation on novel modules with no cost history. # # Implementation: when --budget is set in Phase 4, the dispatch loop reserves # EPSILON_BUDGET = 0.10 × BUDGET_LIMIT for issues where ISSUE_HAS_PRIOR[N] == "false". # This step only MARKS the no-prior issues; Phase 4 reads ISSUE_HAS_PRIOR[] to apply the reserve. # # No-prior issues still compete in the main dispatch queue by score. The reserve # acts as a safety net: if no no-prior issue has been dispatched by the time # PROJECTED_SPEND reaches (BUDGET_LIMIT − EPSILON_BUDGET), the highest-scoring # no-prior issue is force-dispatched from the reserve before budget cutoff. NO_PRIOR_ISSUES=() for NUM in "${ISSUES[@]}"; do [ "${ISSUE_HAS_PRIOR[$NUM]:-false}" = "false" ] && NO_PRIOR_ISSUES+=("$NUM") done echo "Exploration reserve: ${#NO_PRIOR_ISSUES[@]} no-prior issues (ε=10% of budget reserved for these)" # --- Sort the ready-set by descending score --- # The ready-set is the subset of ISSUES with empty PREDECESSORS[]. # Dependency-constrained issues (non-empty PREDECESSORS[]) keep their original DAG ordering — # their dispatch is triggered by predecessor completion, not by score rank. # SORTED_READY_SET is consumed by Step 3E (plan) and Phase 4 (dispatch order). READY_SET=() for NUM in "${ISSUES[@]}"; do [ -z "${PREDECESSORS[$NUM]:-}" ] && READY_SET+=("$NUM") done # Sort by score descending (bc-based comparison via temporary file) SCORE_PAIRS="" for NUM in "${READY_SET[@]}"; do SCORE_PAIRS="${SCORE_PAIRS}${ISSUE_SCORE[$NUM]:-0} $NUM"$'\n' done SORTED_READY_SET=() while IFS=' ' read -r _score num; do [ -n "$num" ] && SORTED_READY_SET+=("$num") done < <(echo "$SCORE_PAIRS" | sort -rn -k1,1 | grep -v '^$') echo "" echo "Ready-set dispatch order (descending value/cost):" for NUM in "${SORTED_READY_SET[@]}"; do PRIOR_TAG="${ISSUE_HAS_PRIOR[$NUM]:-false}" [ "$PRIOR_TAG" = "false" ] && PRIOR_NOTE=" [ε-reserve eligible]" || PRIOR_NOTE=" [prior known]" echo " #${NUM} score=${ISSUE_SCORE[$NUM]:-?} cost_est=\$${ISSUE_COST_ESTIMATE[$NUM]:-?}${PRIOR_NOTE}" done # --- End Step 3E.5 --- ``` **Output state** (consumed by Step 3E and Phase 4): - `ISSUE_SCORE[N]` — value/cost ratio for each issue - `ISSUE_COST_ESTIMATE[N]` — estimated spend in USD - `ISSUE_HAS_PRIOR[N]` — `true` if cost-priors.json had an entry; `false` = no-prior (ε-reserve eligible) - `SORTED_READY_SET[]` — ready issues sorted by descending score (used in Step 3E plan table and Phase 4 dispatch loop) - `NO_PRIOR_ISSUES[]` — issues with no cost history (a subset of `SORTED_READY_SET` + blocked issues) **Important**: `ISSUE_SCORE[]` and `ISSUE_COST_ESTIMATE[]` are also set for blocked issues (for reporting). The sorted order only affects the ready-set — blocked issues dispatch when their predecessors complete, regardless of score. ### Step 3E: Present the plan to the user **Populate every row below by dereferencing `PREDECESSORS[$NUM]`, `ISSUE_DOMAIN[$NUM]`, `ISSUE_SCORE[$NUM]`, and `ISSUE_COST_ESTIMATE[$NUM]` computed in Steps 3B–3E.5 — do not fabricate values from memory or by re-reading issue titles at this point.** The Step 3D.6 gate having passed only guarantees the data structures exist; it does not substitute for actually reading them here. **Source-PR Hint column**: when `ISSUE_LIKELY_MOOT[$NUM]` (populated by `phase-1-resolve.md`'s "Source-PR Triage Hint" step — see that file) is present for an issue, dereference it into a `Source-PR Hint` column. This column is **informational only** — it never changes a row's `Status` (Ready/Blocked), never removes a row from the table, and never causes a row to be excluded from dispatch. It exists purely so the operator can see, before confirming the plan, which `staging-review`/`review-finding` issues cite a source PR that closed without merging — a reason to look closer during that issue's own investigation phase, not a reason to skip it (see `phase-1-resolve.md`'s counterexamples: #2339/#2342 vs. #2346/#2261 — the same signal was right in one case and would have been wrong as a verdict in the other). **File Source column** : dereference `FILE_SOURCE[$NUM]` (populated by Step 3C Layer 1, above) into a `File Source` column, rendered as `contract-deliverables` / `affected-files-section` / `body-fallback` / `none` / `—` (when the issue has no predecessors and no DAG edges depend on its file list at all). Same informational-only contract as `Source-PR Hint` — never changes `Status`, never removes a row. It exists so the operator can see, before confirming the plan, which edges rest on a pre-investigation `body-fallback` guess (lower confidence — see Layer 5's downgrade-eligibility carve-out) versus a post-investigation `affected-files-section` extraction or a `contract-deliverables` table (highest confidence — states intent to change). **Low-confidence serial-chain warning** : print this **above** the plan table, once, when **both** conditions hold: 1. The critical path (already computed for the `**Critical path**` line below) is **≥ 4 steps** deep, and 2. **Every** edge along that path has `FILE_SOURCE` = `body-fallback` for its predecessor. ``` ⚠️ LOW-CONFIDENCE SERIAL CHAIN — {DEPTH} steps, every edge from `body-fallback` provenance #{A} → #{B} → #{C} → #{D} → #{E} No issue on this chain has been investigated or contracted; every edge rests on file paths scraped from raw issue bodies, which name files as context as often as they name files that will actually change. A chain this deep built entirely on that source is the single most likely place for this plan to over-serialize. This is informational — the plan below is unchanged and every issue still dispatches. Consider before approving: • Did one audit/sweep file these issues? If so, see Layer 4's cohort-confidence guidance — a shared file citation across a cohort is one claim repeated N times, not N independent corroborations. • Investigating the chain head first upgrades its provenance and lets the DONE-arm re-verification in phase-4-execution.md unwind the rest automatically. ``` Both conditions are required. Depth alone over-fires on legitimately serialized chains (a DATABASE migration chain is deep *by design*), and `body-fallback` alone is unremarkable — most pre-investigation issues have it. It is the **conjunction** that identifies the specific failure mode from forge#2848: a deep chain where nothing along it has ever been checked against reality. A single non-`body-fallback` edge on the path suppresses the warning, because that edge is corroborated and the chain is no longer uniformly guesswork. This warning is **strictly informational**, on the same contract as the `File Source` and `Source-PR Hint` columns: it never changes a row's `Status`, never removes a row, never alters dispatch order, and never blocks the plan. It is output *in addition to* the table, never in place of any part of it. ``` ## Orchestration Plan **Scope**: {milestone name / "N issues" / "fast-lane"} **Total issues**: {count} ({investigation_count} investigations + {implementation_count} implementations) **Execution model**: Dependency-graph streaming (issues dispatch as predecessors complete) {IF investigations exist:} ### Investigations (run first, may spawn new issues) | # | Title | Expected Output | |---|-------|----------------| | #{INV1} | {title} | New issues → folded into dependency graph | ### Implementation (after investigations complete) {END IF} ### Domain Distribution | Domain | Issues | Notes | |--------|--------|-------| | FRONTEND | {N} | {Independent pages / Shared components} | | BILLING | {N} | {Critical — dispatches immediately} | | DATABASE | {N} | {Serialized chain — migration order matters} | | AUTH | {N} | {Critical — dispatches immediately} | | WORKER | {N} | {High overlap risk within worker service} | | AI | {N} | {Independent} | | INFRA | {N} | {Independent} | (Omit rows with 0 issues. Add project-specific domain rows from forge.yaml → review.domains.) ### Dependency Graph | Issue | Predecessors | Domain | Score | Est. Cost | File Source | Source-PR Hint | Status | |-------|-------------|--------|-------|-----------|-------------|-----------------|--------| | #{A} | — | FRONTEND | {score} | ${cost} | affected-files-section | — | Ready (dispatches 1st by score) | | #{B} | — | BILLING | {score} | ${cost} | affected-files-section | — | Ready (dispatches 2nd by score) | | #{C} | — | WORKER | {score} | ${cost} [ε] | body-fallback | — | Ready (dispatches 3rd — ε-reserve) | | #{D} | #{A} | FRONTEND | {score} | ${cost} | none | — | Blocked (waits for #{A} only) | | #{E} | — | DATABASE | {score} | ${cost} | affected-files-section | — | Ready (dispatches 4th by score) | | #{F} | #{E} | DATABASE | {score} | ${cost} | body-fallback | — | Blocked (serialized — waits for #{E}) | | #{G} | — | INFRA | {score} | ${cost} | affected-files-section | likely-moot (PR #{N} closed unmerged — verify first) | Ready (dispatches 5th by score) | **[ε]** = no cost prior; eligible for exploration reserve (10% of budget guaranteed for these) **File Source** = `${FILE_SOURCE[$NUM]:-none}` (Step 3C Layer 1) — one of `contract-deliverables` (highest confidence) / `affected-files-section` / `body-fallback` / `none`. Never affects `Status` or row inclusion — see Layer 1's provenance tracking and Layer 5's downgrade-eligibility carve-out (forge#2436, forge#2848) for how each value is actually consumed, and the low-confidence serial-chain warning above for when a run of `body-fallback` edges is worth an operator's attention. **Source-PR Hint** = `${ISSUE_LIKELY_MOOT[$NUM]:-unknown}` rendered as `—` when `unknown`/absent, or `likely-moot (PR #{ISSUE_SOURCE_PR[$NUM]} closed unmerged — verify first)` when `yes`. Never affects `Status` or row inclusion — see `phase-1-resolve.md`'s "Source-PR Triage Hint" step for how this is computed and why it stays a hint. **Score** = value / estimated_cost (value = priority_weight × danger_zone_weight; higher = dispatches first within the ready-set) **Est. Cost** = cost-prior mean for (task_type × module), or label heuristic if no prior **Critical path**: #{E} → #{F} (2 steps, determines minimum wall-clock time) **Initial dispatch**: #{A}, #{B}, #{C}, #{E} (all predecessors resolved — ordered by score) **Streaming**: #{D} dispatches as soon as #{A} completes — does NOT wait for #{B}, #{C}, or #{E} **Note**: Investigations may create additional issues that will be automatically added to the dependency graph. The final graph will be confirmed after investigations complete. **Excluded** (already in progress / ineligible): - #{X} — {reason} {IF EXCLUDED_CYCLE is non-empty:} ### ⚠ Circular Dependencies Detected — Manual Fix Required The following issues form a circular dependency chain and **cannot be dispatched** until the cycle is resolved: | Issue | Depends On | Problem | |-------|------------|---------| {rows: each EXCLUDED_CYCLE issue, its predecessor list, "mutual dependency — forms cycle with #{other}"} **Action required**: Edit each issue's body to remove or correct the `Depends on` / `Blocked by` declarations so no cycle exists. Each issue has been labeled `needs-human`. After fixing, re-run `/orchestrate` to dispatch them. {END IF} Proceed? (yes / adjust / pick specific issues) ``` **Wait for user confirmation before spawning agents unless the caller explicitly passed `--auto` or `--confirm`.** Those flags are machine-readable authorization for headless callers such as `/autopilot`; they do not change the plan or bypass any eligibility, dependency, or safety checks. Once the checkpoint is authorized, agents launch and run autonomously. **After confirmation**: If investigations exist, execute Phase 2B-E first. Interactive callers then re-present the expanded plan (with newly spawned issues added to the dependency graph) for a quick confirmation before launching implementation. An explicit `--auto`/`--confirm` caller proceeds through that second checkpoint without prompting, while still rebuilding the graph. --- ## Engine mode (default) Dispatch each issue via the durable execution engine. This is the **default execution mode** for both interactive (`/autopilot`, `/orchestrate`) and headless/CI paths: ```bash forgedock run-issue --lane ``` The engine drives every phase transition deterministically, mirrors state to the `FORGE:STATE` block on the issue, and holds a lease. Its **fail-closed review gate** (`phases.mjs → detectOutcome`) means the PR must be confirmed merged before the phase is committed — missing or unparseable review comments are treated as failures, not approvals. To recover stalls, scan in-flight issues' `FORGE:STATE`; any issue with an expired lease and a non-terminal state is re-dispatched with the same `forgedock run-issue ` command — it resumes from the last committed phase (idempotent). This replaces the label-heuristic "already in progress" check and the resume-with-nagging loop. **Why engine-first**: The engine's phase table enforces gate semantics in code — not via LLM interpretation of markdown specs. This eliminates the class of bug where the LLM assumes a review approved when the FORGE:REVIEW comment is absent or unreadable (issue #1714). Interactive sessions that run `/work-on` via Skill invocations additionally bridge to the engine run-log via the SubagentStop hook (`bin/hooks/interactive-engine.mjs`) — state is durable across compaction and context resets. **Fallback — best-effort, not all-or-nothing (fixed forge#2743)**: Engine-first is guaranteed to have an Agent-spawn fallback under two distinct trigger conditions, not just CLI absence: 1. **CLI absent at dispatch time**: If `forgedock` is not in PATH, `phase-4-execution.md` Step 4A falls back to spawning Agent sub-agents that run `Skill("work-on", ...)` per issue for the whole batch, before any dispatch happens. 2. **Backend unavailable despite CLI presence, or a runtime engine-error with empty committed state**: `command -v forgedock` only proves the orchestrator CLI is installed — it says nothing about whether the engine's execution backend (the `claude` CLI spawn) can actually run a phase (forge#2741 is a concrete case: `spawnSync claude` ENOENTs even though a shell `command -v` probe reports the binary present). Step 4A now runs a cheap backend preflight canary before committing the whole ready set to engine-first, downgrading the entire run to Agent-spawn on failure. And per-issue, Step 4B's completion handler auto-falls-back any individual engine-dispatched issue that completes at `workflow:engine-error` with an empty committed state (`committed=[] branch=null pr=null` — see `bin/engine-cli.mjs`'s `formatTerminalDiagnostics()`) to the same Agent-spawn template, rather than leaving it stuck waiting on a resume mechanism that only exists for Agent-spawn-dispatched issues. An engine-error with partial committed state (branch/PR non-null) is NOT auto-fallen-back, to avoid double-work — it surfaces via the existing stall-detection alert instead. In both cases the SubagentStop hook still bridges the fallback Agent-spawn runs to the engine run-log for state persistence. Engine-first is therefore best-effort: it is always backed by a working Agent-spawn path, whether the gap is discovered before dispatch (canary) or after (per-issue fallback) — a whole ready set is never permanently lost to an environmental engine failure. ### Concurrency model: in-process worker pool + worktree-per-issue **Decision** (recorded 2026-07-04, issue #1324): The durable engine uses an **in-process worker pool** model — a single control plane dispatches and monitors all concurrent issues, each isolated in its own git worktree. **Rationale over process-per-issue:** - Worktree isolation primitive already ships: `scripts/worktree-lifecycle.sh` (`ensure`/`cleanup` subcommands, merged #1268) provides deterministic filesystem isolation without forking a separate OS process per issue. - A single control plane can enforce **shared rate-limit backpressure** across all in-flight issues; per-process models require IPC to share API quota state. - Co-ordination primitives (DAG ready-set, completion callbacks, lease renewal) live in one place with no cross-process synchronisation overhead. - Aligns with the engine-first inversion (#1256): the engine owns correctness; the spec owns routing. **Filesystem isolation**: Before dispatching each issue, the engine calls: ```bash scripts/worktree-lifecycle.sh ensure # → creates or reuses .forgedock/worktrees/issue-/ ``` On completion or failure: ```bash scripts/worktree-lifecycle.sh cleanup ``` **Concurrency cap** (`forge.yaml → orchestration.max_concurrent`): - Default: **12** — the dispatch loop holds at most 12 in-flight workers unless overridden. This is an enforced default, not opt-in: earlier revisions defaulted to uncapped, which let a large ready set (e.g. 40+ issues) dispatch in one burst and saturate the Anthropic API rate limit. - When `max_concurrent: N` is set, the dispatch loop holds at most N in-flight workers instead of the default 12. Newly ready issues queue and start as running workers complete. It is a top-level worker cap only: each worker normally consumes roughly **8 total subagent spawns** across `/work-on`, build, quality-gate, and review. Set `N <= session_subagent_budget / 8`; for a 200-spawn session budget, use 25 or fewer. - Prevents wave-triggered rate-limit storms on large batches (e.g., 40-issue milestone dispatches). - See `phase-4-execution.md` Step 4A-pre.0.2 for the concrete initialization and headroom-gated dispatch logic that enforces this cap on both the engine-first and Agent-spawn-fallback dispatch paths. **Rate-limit backpressure** (pre-dispatch gate): Before dispatching each new worker, the engine runs: ```bash REMAINING=$(gh api rate_limit --jq '.resources.core.remaining') RESET_AT=$(gh api rate_limit --jq '.resources.core.reset') RATE_LIMIT_FLOOR=${FORGE_RATE_LIMIT_FLOOR:-200} if [ "$REMAINING" -lt "$RATE_LIMIT_FLOOR" ]; then echo "GitHub API headroom below floor ($REMAINING < $RATE_LIMIT_FLOOR). Pausing dispatch until reset at $RESET_AT." # Pause dispatch loop — already-running workers continue unaffected sleep_until "$RESET_AT" fi ``` - `FORGE_RATE_LIMIT_FLOOR` defaults to 200 remaining requests. Override in `forge.yaml → orchestration.rate_limit_floor`. - Already-in-flight workers are **never interrupted** by the backpressure gate — only new dispatches pause. - The gate is re-checked after each worker completion, not on a timer, so dispatch resumes immediately once the floor is cleared. - This core-quota check cannot detect GitHub secondary content-creation throttles. A `403` response containing `secondary rate limit` is a separate first-class signal: stop new dispatch, retain the queues, and require an operator resume rather than retrying or polling. In-flight workers must stop their GitHub write/create retry loops and report the error (see Phase 4 Step 4A-pre.0.3). **Configuration reference** (`forge.yaml`): ```yaml orchestration: max_concurrent: 8 # optional; default: 12 rate_limit_floor: 200 # optional; default: 200 ``` --- ## Background Dispatch Mode This section governs how the orchestrator dispatches DAG-ready issues as background agents and how it handles wake/compaction recovery. Read it before every Phase 4 dispatch decision. ### Feature gate Background dispatch (via `run_in_background=true` on each `Agent()` call) is the primary dispatch path. It is enabled when **both** of the following conditions hold: 1. **Version**: Claude Code >= v2.1.186 (the release that introduced background subagents with proper `agent_completed` completion notifications and the Notification hook). Below this version, background agents may not surface completion events correctly. 2. **Env var**: `CLAUDE_CODE_DISABLE_BACKGROUND_TASKS` is **not** set (or is empty). Check both at the start of Phase 4, before the first dispatch: ```bash # Runtime selection is explicit for non-Claude adapters. OpenCode runs nested # work-on phases through its native task path, so it must not be treated as a # failed Claude probe. FORGE_RUNTIME="${FORGE_RUNTIME:-}" if [ -n "${OPENCODE_SESSION_ID:-}" ] || [ -n "${OPENCODE_PID:-}" ] || [ -n "${OPENCODE:-}" ]; then FORGE_RUNTIME="opencode" fi if [ "$FORGE_RUNTIME" = "opencode" ]; then echo "OpenCode runtime detected: native task dispatch selected." # ForgeDock's OpenCode plugin opts into background task events by default so # each completed DAG node can wake this parent without a wave barrier. if [ "${OPENCODE_EXPERIMENTAL_BACKGROUND_SUBAGENTS:-true}" = "false" ]; then echo "OpenCode background subagents explicitly disabled: orchestration is degraded to foreground task dispatch." BACKGROUND_DISPATCH_ENABLED=false else BACKGROUND_DISPATCH_ENABLED=true fi fi # Feature gate check — run once before Phase 4 dispatch begins if [ "${BACKGROUND_DISPATCH_ENABLED:-}" != "false" ]; then BACKGROUND_DISPATCH_ENABLED=true fi if [ -n "${CLAUDE_CODE_DISABLE_BACKGROUND_TASKS:-}" ]; then echo "Background dispatch disabled: CLAUDE_CODE_DISABLE_BACKGROUND_TASKS is set." BACKGROUND_DISPATCH_ENABLED=false fi # Version check: if the Claude Code version can be read, compare it. # If the version cannot be determined, default to ENABLED (optimistic). CC_VERSION=$(claude --version 2>/dev/null | grep -oP '\d+\.\d+\.\d+' | head -1 || echo "") if [ "$FORGE_RUNTIME" != "opencode" ] && [ -n "$CC_VERSION" ]; then # Compare major.minor.patch numerically IFS='.' read -r CC_MAJOR CC_MINOR CC_PATCH <<< "$CC_VERSION" if [ "$CC_MAJOR" -lt 2 ] || \ { [ "$CC_MAJOR" -eq 2 ] && [ "$CC_MINOR" -lt 1 ]; } || \ { [ "$CC_MAJOR" -eq 2 ] && [ "$CC_MINOR" -eq 1 ] && [ "$CC_PATCH" -lt 186 ]; }; then echo "Background dispatch disabled: Claude Code ${CC_VERSION} < v2.1.186." BACKGROUND_DISPATCH_ENABLED=false fi fi ``` When `FORGE_RUNTIME=opencode` (or an OpenCode runtime marker is present), do not interpret a missing `claude` executable as an environment failure. The Phase 4 dispatcher must use OpenCode's native `task` contract instead of the Claude `Agent(...)` fallback. With background subagents enabled, each task is launched with `background=true` and its injected task-result event immediately drives predecessor classification and newly-ready dispatch. Claude remains the default when no runtime marker is present, preserving the existing Claude behavior. **When `BACKGROUND_DISPATCH_ENABLED=false`**: Claude falls back to its existing synchronous/polling behavior. OpenCode reaches this branch only after an explicit `OPENCODE_EXPERIMENTAL_BACKGROUND_SUBAGENTS=false`; use independent foreground `task` calls where the host can execute them concurrently, and report that per-completion streaming is unavailable. Do not use Claude's `Agent(...)` fallback and do not claim the event-driven DAG guarantee. **When `BACKGROUND_DISPATCH_ENABLED=true`**: Claude uses `run_in_background=true` and reacts to `agent_completed` notifications. OpenCode uses `task(..., background=true)` and reacts to each injected task-result event. Do NOT poll or wait for a whole wave in either enabled path. ### Orchestrator state reconstruction on wake / after compaction The orchestrator context window must stay small regardless of how many issues have been dispatched. Achieving this requires that all dispatch state is stored on GitHub — not in the orchestrator's context. **Contract**: After any compaction event or orchestrator wake (session resumed after idle/restart), do NOT rely on in-context variables. Instead, reconstruct the DAG dispatch state from GitHub before checking for newly ready issues: This reconstruction MUST use the same three-way **DONE / GATED / FAILED** predecessor classification defined in `phase-4-execution.md` Step 4B ("Predecessor Classification") — not a binary terminal/non-terminal grep. A binary grep is exactly the bug forge#1812 fixed: it let `needs-human` simultaneously satisfy "predecessor is done, dispatch the successor" (this block, pre-fix) and "predecessor failed, skip the successor" (Step 4B's failure handler, pre-fix) — with no way to represent "predecessor is human-gated, its PR is still open, and its dependents should wait but not be abandoned." That third case is exactly what wake/compaction reconstruction hits most often, since a merge approved by a human typically happens *after* the orchestrator session that dispatched the predecessor has already ended — this block, not the live Step 4B loop, is the realistic trigger point for "gating PR merged while nobody was watching." For the same reason, this reconstruction also MUST call `verify_file_overlap_edge()` (also defined in `phase-4-execution.md` Step 4B, alongside `classify_predecessor_state()` — re-declare it here too if this block runs in a fresh context) before treating a GATED or FAILED predecessor's `EDGE_KIND` edge as still blocking. A predecessor that reached `needs-human`/`workflow:invalid` with no PR, or whose PR never actually touched the guessed shared file, most realistically gets *discovered* at wake time — the session that would have caught it live has already ended. Re-verifying only in the live Step 4B loop and not here would leave this exact wake-time case unfixed. ```bash # Reconstruct dispatch state from GitHub after compaction / wake # Run this block at the top of every resumed Phase 4 loop iteration. # Keep this byte-identical to phase-4-execution.md Step 4A. A wake can start in a # fresh context, so it cannot rely on the live dispatch helper still being defined. read_active_claims() { local COORD_NUM="$1" local CLAIMS HOLDER TERMINAL CLAIMS=$(gh api --paginate --slurp "repos/{GH_REPO}/issues/${COORD_NUM}/comments" 2>/dev/null \ | jq -c ' flatten as $comments | [$comments[] | select(.body | contains("")) | . as $claim | ($claim.body | capture("\\*\\*Holder\\*\\*: #(?[0-9]+)").holder) as $holder | select([$comments[] | select(.body | contains("")) | select((.body | capture("\\*\\*Holder\\*\\*: #(?[0-9]+)").holder) == $holder) | select(.created_at > $claim.created_at)] | length == 0) | {holder: $holder, files: ($claim.body | capture("\\*\\*Files\\*\\*: (?[\\s\\S]*?)(?:\\n\\*\\*Interfaces\\*\\*:|$)").files)}]') || return 1 for HOLDER in $(echo "$CLAIMS" | jq -r '.[].holder'); do TERMINAL=$(gh issue view "$HOLDER" -R {GH_REPO} --json labels --jq \ '[.labels[].name | select(. == "workflow:merged" or . == "workflow:invalid" or . == "workflow:awaiting-merge" or . == "needs-human")] | length > 0' 2>/dev/null) [ "$TERMINAL" = "true" ] && CLAIMS=$(echo "$CLAIMS" | jq --arg holder "$HOLDER" '[.[] | select(.holder != $holder)]') done echo "$CLAIMS" } # 0. Lease check (MANDATORY, forge#2627) — run BEFORE any of the reconstruction below. # A wake/resume is either this same orchestrator continuing its own batch or, less # commonly, a fresh invocation resuming someone else's interrupted batch. Either way, # do not reconstruct or dispatch against a batch another *live* orchestrator still # holds the lease for. # check_orchestrator_lease() is declared in Step 3D.2 above — re-declare here if this # block runs in a fresh context that hasn't sourced Step 3D.2 yet. # # BATCH_ID reconstruction: this section's own contract is "do NOT rely on in-context # variables" — BATCH_ID is a plain shell variable set once in Step 3D.1 and is NOT # guaranteed to survive compaction/wake (export only propagates within the same process # tree, not across a genuinely new session). Re-derive it from GitHub here rather than # trusting an in-context value, the same way FORGE_COORD_ISSUE/COORD_ISSUE_NUMBER # themselves are treated as re-derivable state, not assumed-present variables: if [ -z "${BATCH_ID:-}" ] && [ -n "${FORGE_COORD_ISSUE:-}" ] && [ -n "${COORD_ISSUE_NUMBER:-}" ]; then BATCH_ID=$(gh issue view "$COORD_ISSUE_NUMBER" -R {GH_REPO} --json body \ --jq '.body' 2>/dev/null | grep -oE '' | head -1 | sed -E 's/^$//') if [ -n "$BATCH_ID" ]; then export BATCH_ID echo "Reconstructed BATCH_ID=${BATCH_ID} from coordination issue #${COORD_ISSUE_NUMBER} body (in-context value was lost, per this section's own contract)." else # KNOWN LIMITATION (not a silent oversight): this is exactly the stale-survivor-plus-restart # scenario the lease exists to catch, and it is the one case where the lease cannot be # evaluated at all — there is no safe synthetic BATCH_ID to substitute: guessing one risks # a false self-lockout (refusing to resume the orchestrator's own batch) which is a worse # failure mode than the gap being flagged. Fail-open is the deliberate choice for a # best-effort primitive built on GitHub comments (see Step 3D.2's "Known limitation" note), # but it must be loud, not a quiet log line — this WARNING is the operator's only signal # that single-instance protection did not run this cycle. echo "WARNING: could not reconstruct BATCH_ID from coordination issue #${COORD_ISSUE_NUMBER} (missing/malformed FORGE:BATCH_ID marker in the issue body) — lease check SKIPPED this cycle. This is the exact scenario the lease exists to protect against; single-instance protection is NOT active until the marker is present or a later cycle reconstructs it. This is a known, documented limitation of a best-effort primitive — not a silent bug." >&2 fi fi if [ -n "${FORGE_COORD_ISSUE:-}" ] && [ -n "${COORD_ISSUE_NUMBER:-}" ] && [ -n "${BATCH_ID:-}" ]; then LEASE_STATE=$(check_orchestrator_lease "$COORD_ISSUE_NUMBER" "$BATCH_ID") case "$LEASE_STATE" in held:*) HELD_BY="${LEASE_STATE#held:}" echo "REFUSING TO RESUME: an unexpired orchestrator lease for this batch is held by batch ${HELD_BY} (coordination issue #${COORD_ISSUE_NUMBER}), not this session's batch ${BATCH_ID}." echo "A different live orchestrator instance appears to already be dispatching this batch. Wait for its lease to expire or confirm it is dead before resuming." exit 1 ;; free|self) # Free (no live holder) or self (this exact batch already holds it) — safe to # refresh and continue reconstruction below. HOSTNAME_ID=$(hostname 2>/dev/null || echo "unknown-host") # GOVERNOR-exempt: intentional coordination side-effect (best-effort lease/board/finding post), DRY_RUN-safe — reviewed & accepted for the check-command-side-effects gate. Flagged only by the staging->main full-diff; passes on every feature PR. forge#2627 gh issue comment "$COORD_ISSUE_NUMBER" -R {GH_REPO} --body " **Holder Batch ID**: ${BATCH_ID} **Holder**: ${HOSTNAME_ID} (pid ${$}) **Acquired/refreshed**: $(date -u +%Y-%m-%dT%H:%M:%SZ) **TTL**: ${LEASE_TTL_SECONDS:-900}s (refreshed on wake/compaction resume)" 2>/dev/null || true ;; *) # Defensive default (MANDATORY — do not remove): see the matching comment at the # Step 3D.2 acquisition case block above. An unexpected LEASE_STATE here must warn # loudly, not silently fall through and let wake reconstruction proceed as if the # lease check had passed. echo "WARNING: check_orchestrator_lease() returned unexpected value '${LEASE_STATE}' during wake/compaction reconstruction — lease gate could not be evaluated. Proceeding without a confirmed lease; investigate rather than ignore." >&2 ;; esac fi # 1. Re-fetch all issue labels and classify each into DONE / GATED / FAILED / IN_PROGRESS # (same classify_predecessor_state() function defined in phase-4-execution.md Step 4B — # re-declare it here if this block runs in a fresh context that hasn't sourced Step 4B yet). declare -A ISSUE_CLASS declare -A ISSUE_FILES DONE_ISSUES=() GATED_ISSUES=() FAILED_ISSUES=() ACTIVE_ISSUES=() # IN_PROGRESS — still mid-pipeline, not yet terminal-for-this-agent for NUM in {all_issue_numbers_in_batch}; do CLASS=$(classify_predecessor_state "$NUM") ISSUE_CLASS["$NUM"]="$CLASS" case "$CLASS" in DONE) DONE_ISSUES+=("$NUM") ;; GATED) GATED_ISSUES+=("$NUM") ;; FAILED) FAILED_ISSUES+=("$NUM") ;; *) ACTIVE_ISSUES+=("$NUM") ;; esac done # 1.5. Rebuild the durable file-claim map. The claims board, not the orchestrator's # in-context memory, is authoritative before every later dispatch. Rebuild the target # declarations too because ISSUE_FILES is lost across compaction. if [ -n "${FORGE_COORD_ISSUE:-}" ] && [ -n "${COORD_ISSUE_NUMBER:-}" ]; then ACTIVE_CLAIMS=$(read_active_claims "$COORD_ISSUE_NUMBER") declare -A ACTIVE_CLAIM_FILES for HOLDER in $(echo "$ACTIVE_CLAIMS" | jq -r '.[].holder'); do ACTIVE_CLAIM_FILES["$HOLDER"]=$(echo "$ACTIVE_CLAIMS" | jq -r --arg holder "$HOLDER" '.[] | select(.holder == $holder) | .files') done for NUM in {all_issue_numbers_in_batch}; do EXTRACT_OUT=$(bash "$AFFECTED_FILES_SCRIPT" "$NUM" -R "{GH_REPO}") ISSUE_FILES[$NUM]=$(echo "$EXTRACT_OUT" | tail -n +2) done fi # 2. Re-derive the ready set: any non-terminal issue whose predecessors are ALL classified DONE # (or GATED-but-edge-dropped — see the re-verification gate below). # A GATED predecessor blocks dispatch but does NOT fail the dependent — see step 2.5 below. # # Edge re-verification (forge#1904): a GATED (or FAILED) predecessor's file-overlap edge is only # a real block if `verify_file_overlap_edge()` (phase-4-execution.md Step 4B, defined alongside # `classify_predecessor_state()`) confirms it. This block is the wake/compaction-time mirror of # phase-4-execution.md item 6.5's live-session check — it MUST call the identical function, not a # re-derived equivalent, to avoid the drift class forge#1812/#1837 already had to fix once for # classification and regex tooling respectively. This is what handles the case where a GATED # predecessor with no PR (or a PR that never touched the guessed shared file) resolves the # `needs-human`/`workflow:invalid` state AFTER the orchestrator session that dispatched it has # already ended — the realistic trigger point named in the "Why this matters" note above. READY_ISSUES=() NEWLY_BLOCKED=() # dependents whose gating predecessor is GATED with a still-live edge — need blocked-on-human-merge tracking for NUM in "${ACTIVE_ISSUES[@]}"; do ALL_PREDS_DONE=true GATING_PRED="" for PRED in {predecessors_of_NUM}; do case "${ISSUE_CLASS[$PRED]:-IN_PROGRESS}" in # DONE needs no re-verification call HERE because it already satisfies the edge, and # this reconstruction builds no SAME_FILE_BRIEF. Unlike the live DONE arm, this path # does not retain EDGE_KIND/EDGE_FILES or re-run Layer 1 extraction; consequently it # does not re-derive stale DONE-path edges after wake. That live-only re-derivation is # in phase-4-execution.md lines 1208-1268. Adding the helper call here would therefore # add API cost without affecting wake-time dispatch readiness. DONE) ;; GATED|FAILED) EDGE_VERDICT=$(verify_file_overlap_edge "$PRED" "$NUM") if [ "$EDGE_VERDICT" = "DROP" ]; then echo "Edge re-verification: #${PRED} → #${NUM} dropped (GATED/FAILED predecessor never opened a PR, or its actual diff never touched the guessed shared file). Treating this predecessor as resolved for #${NUM}." # Do NOT set ALL_PREDS_DONE=false for this predecessor — the guessed edge never # materialized into a real conflict, so it does not gate #${NUM}. elif [ "${ISSUE_CLASS[$PRED]}" = "GATED" ]; then ALL_PREDS_DONE=false GATING_PRED="$PRED" else ALL_PREDS_DONE=false # FAILED with a confirmed real edge — dependent stays blocked/skipped, handled by existing FAILED-cascade logic elsewhere fi ;; *) ALL_PREDS_DONE=false ;; esac done if [ "$ALL_PREDS_DONE" = "true" ]; then READY_ISSUES+=("$NUM") elif [ -n "$GATING_PRED" ]; then NEWLY_BLOCKED+=("$NUM|$GATING_PRED") fi done # 2.5. Track newly-blocked dependents (mirrors phase-4-execution.md Step 4B item 6.5) # Self-heal the label if not yet bootstrapped (same pattern as review-pr.md 6C / phase-4-execution.md item 6.5). gh label create "blocked-on-human-merge" --color "006B75" --description "Dependent of a gated (needs-human/awaiting-merge) predecessor. Managed by ForgeDock." --force -R {GH_REPO} 2>/dev/null for ENTRY in "${NEWLY_BLOCKED[@]:-}"; do [ -z "$ENTRY" ] && continue DEP="${ENTRY%%|*}" PRED="${ENTRY##*|}" # Anchor on the exact "**Gating predecessor**: #N" label with a word boundary — # a bare contains("#N") substring would false-match #50/#500 for predecessor #5. ALREADY_TRACKED=$(gh api repos/{GH_REPO}/issues/${DEP}/comments \ --jq --arg prednum "${PRED}" '[.[] | select(.body | contains("FORGE:BLOCKED_ON_HUMAN_MERGE") and test("Gating predecessor\\*\\*: #" + $prednum + "\\b"))] | length' 2>/dev/null || echo "0") if [ "$ALREADY_TRACKED" -eq 0 ]; then GATING_PR=$(gh pr list -R {GH_REPO} --state open --search "\"Closes #${PRED}\" in:body" \ --json number --jq '.[0].number // empty' 2>/dev/null || echo "") gh issue comment "$DEP" -R {GH_REPO} --body " **Gating predecessor**: #${PRED} (state: \`${ISSUE_CLASS[$PRED]}\`${GATING_PR:+, open PR #${GATING_PR}}) **Status**: Detected on orchestrator wake/compaction reconstruction. Ready to dispatch as soon as #${PRED} reaches \`workflow:merged\`." gh issue edit "$DEP" -R {GH_REPO} --add-label "blocked-on-human-merge" 2>/dev/null || true fi done # 3. Merge-triggered wake: any issue tracked as blocked-on-human-merge whose gating predecessor # is now DONE gets un-blocked and added to the ready set. This is the wake-time equivalent of # phase-4-execution.md Step 4B item 6.6 — it is what makes "auto-dispatch on merge, no manual # /orchestrate re-run" hold true even when the merge happened after the session ended. BLOCKED_NOW=$(gh issue list -R {GH_REPO} --state open --label "blocked-on-human-merge" --json number \ --jq '.[].number' 2>/dev/null || echo "") for DEP in $BLOCKED_NOW; do # Read which predecessor(s) this DEP is tracked against GATING_PREDS_RAW=$(gh api repos/{GH_REPO}/issues/${DEP}/comments \ --jq '[.[] | select(.body | contains("FORGE:BLOCKED_ON_HUMAN_MERGE")) | (.body | capture("Gating predecessor\\*\\*: #(?

[0-9]+)").p)]' 2>/dev/null || echo '[]') STILL_GATED=false for GPRED in $(echo "$GATING_PREDS_RAW" | jq -r '.[]' 2>/dev/null); do GPRED_CLASS=$(classify_predecessor_state "$GPRED") [ "$GPRED_CLASS" != "DONE" ] && STILL_GATED=true done if [ "$STILL_GATED" = "false" ]; then gh issue edit "$DEP" -R {GH_REPO} --remove-label "blocked-on-human-merge" 2>/dev/null || true gh issue comment "$DEP" -R {GH_REPO} --body " All gating predecessor(s) reached \`workflow:merged\` (detected on orchestrator wake) — dispatching now." READY_ISSUES+=("$DEP") fi done # 4. Dispatch the reconstructed ready set (DONE_ISSUES-unblocked + merge-triggered-woken + # edge-dropped-into-READY_ISSUES from step 2 above) via the standard Step 4A.pre.0 → 4A.pre → # 4A flow. FAILED_ISSUES' transitive dependents remain marked "skipped — dependency failed" per # phase-4-execution.md Step 4B item 6 — do not re-add them here — UNLESS step 2's # `verify_file_overlap_edge()` check already placed them in READY_ISSUES because the FAILED # predecessor's edge dropped (forge#1904); that case is legitimately ready, not a re-add. ``` **Why this keeps context small**: Each `Agent()` call returns an agent ID stored only in `AGENT_ISSUE_MAP`, and engine/OpenCode dispatches use equivalent ephemeral `ENGINE_DISPATCH_MAP` and `OPENCODE_DISPATCH_MAP` caches. After compaction, those maps are gone — but the DAG state, including `blocked-on-human-merge` tracking (a durable `FORGE:BLOCKED_ON_HUMAN_MERGE` comment plus label, not an in-context variable) and OpenCode child correlation (`FORGE:DISPATCH`), is fully on GitHub. The reconstruction above re-derives the ready set, the gated set, and the blocked-on-human-merge set from labels and comments alone, so the orchestrator context never needs to hold cumulative dispatch history. ---