# Epic: Competitive-gap enhancements — reach ecosystem parity without sacrificing infra depth

> **Parent of 5 child issues.** This epic groups the five bounded enhancements
> produced by the `pi_workflow_extensions_competitive_analysis` review against
> our top peer (`pi-maestro-flow`). It is the follow-up work to the #137 bounded
> workflow-engine ecosystem review (`docs/workflow-engine-review.md`).
>
> Every child issue follows the #137 policy: **adapt the concept through a
> bounded GitHub issue; never vendor or merge wholesale.** Each lands as a
> bounded PR through the Issue Delivery closed-loop + Codex review gate.

---

## Origin (the competitive analysis; top peer = pi-maestro-flow; the 5 gaps; the half-right verdict)

The #137 review (`docs/workflow-engine-review.md`, peer-watchlist, reviewed
2026-08-13) classified candidate mechanisms from the Pi workflow-engine peer
catalog. The package carrying the most distinct, source-verifiable mechanisms
relevant to this architecture is **`pi-maestro-flow`** — a declarative
ecosystem featuring:

- 63 skills / 32 roles (broad declarative ecosystem),
- a SQLite/BM25F knowledge graph (structured `spec`/`knowhow`/`domain` taxonomy,
  cross-workspace linking, auto-deposit hooks, lifecycle management),
- Goal loops (open-ended objective + independent verifier + token budget),
- a Durable Plan mode (read-only Markdown draft, approve-before-act, dedicated
  Plan model, `~/.pi/workspaces/.../plans/` tree + approval history),
- a provider circuit breaker + `/model-failover` automatic failover down a
  precedence chain, and
- `pi-cockpit` footer/observability (context gauge, cost, retry countdown).

**Where we stand** (`pi-dynamic-workflows`, this fork): the **most complete
programmable workflow + delivery engine**. We have infra depth that the peer
lacks — a deterministic VM sandbox, crash-safe disk journal, first-class
`checkpoint()` with AFK-safe default replay, herdr/tmux live monitor panes, the
closed-loop Issue Delivery pipeline (Scout → Thinker → Worker → LocalChecks →
Verifier → PR → finalization), per-subagent context/budget/compaction policy,
and role-based model routing with security isolation (`enforceRoutingPolicy`,
API-billing opt-in, Foundation fail-closed).

**The user hypothesis "we have way more features" is half right:**

- **True on infra depth** — deterministic VM, journal, `checkpoint()`, herdr,
  closed-loop delivery, per-subagent budgets, role routing w/ security
  isolation. The peer has none of these.
- **False on ecosystem breadth** — the peer ships five distinct mechanisms we
  do not have today: a workflow-integrated knowledge system, a provider
  circuit breaker + failover, a goal loop primitive, a durable plan mode, and
  richer monitor-pane observability gauges.

The competitive analysis (`pi_workflow_extensions_competitive_analysis`)
isolated exactly those five gaps and produced a per-gap design doc in
`docs/plans/competitive-gap-{1..5}-*.md`. Each design verdicts the gap as
**adapt-don't-vendor** per #137. This epic is the parent that sequences them.

---

## Strategy (adapt-don't-merge per #137; reuse Hindsight natively for gap #1; reference pi-maestro-flow for the other 4; keep our infra-depth lead)

1. **Adapt concepts, never vendor code.** Per #137, every gap is a read-only
   idea source adapted through a bounded GitHub issue. No `import` from
   `pi-maestro-flow`, no copy-paste of `EndpointCircuitBreaker`, their
   `plan.json`/`waves.json` schema, their `~/.pi/workspaces/.../plans/` tree,
   their TUI plan-mode carousel, or their `setFooter` mechanism. We build our
   own against our deterministic VM / disk journal / `checkpoint()` / herdr /
   role-based routing / Foundation fail-closed architecture.

2. **Reuse what we already have before building.** Gap #1 is the clearest
   case: the design audit found that **Hindsight** (already running on this VM,
   50+ banks, TEMpr recall, auto-consolidating observations, entity labels,
   mental models, `memory_gardener`) is a *superset* of the peer's SQLite/BM25F
   knowledge graph. The gap is **integration, not capability** — so #1 reuses
   Hindsight natively instead of standing up a parallel store. Gaps #3 and #4
   likewise reuse existing primitives (`agent()`, `parallel()`, `retry()`,
   `gate()`, `checkpoint()`, `budget`, the disk journal) — the new code is thin
   sugar over what already exists.

3. **Reference `pi-maestro-flow` for the other four** (#2 circuit breaker,
   #3 goal loop, #4 plan mode, #5 monitor gauges) as a concept source only.
   Each child issue's plan doc cites the peer README/commit and records the
   #137 attribution (credit in CHANGELOG + `docs/workflow-engine-review.md`
   peer-watchlist). We read their mechanism, then implement it inside our
   architecture with our determinism/fail-closed/security-isolation constraints
   preserved.

4. **Keep our infra-depth lead.** None of the five gaps trades away a
   determinism, journal-replay, AFK-safe, or security-isolation property.
   Specifically:
   - #2's circuit breaker uses **lazy** recovery (checked at call time), no
     `setTimeout` background sweep, no cross-process persistent state — the VM
     stays deterministic. Failover chains are **operator-declared per role**,
     never auto-invented; an empty chain fails closed (= today's behavior).
   - #3's `goal()` is pure sugar over `agent()` + `checkpoint()`; no new
     runtime state, no new journal shape, no ambient `turn_end` verification.
   - #4's `planApprove` is layered on `checkpoint()` with `default: false` for
     AFK safety and a plan-checksum in the journaled identity so a plan change
     invalidates the replay.
   - #5's footer is a **pure** projection of already-collected snapshot data —
     no `Date.now()`, no I/O, default-off on the live task panel (byte-identical
     regression test is a hard gate).
   - #1 adds no new store; Hindsight already provides the capability.

5. **Do not chase ecosystem breadth.** The peer's 63-skill/32-role declarative
   catalog, intent-to-chain NL classifier, and MOA/ACO swarm are explicitly
   **not worth filing** (see Non-goals). We close the five *mechanism* gaps, not
   the *catalog* gap.

---

## The 5 child issues

| #  | Title | Complexity | Can-do-natively | Plan doc |
|----|-------|-----------|-----------------|----------|
| 1  | Integrate Hindsight as workflow knowledge system (no parallel store) | medium | **yes** — research confirmed Hindsight is a superset of the peer's SQLite/BM25F KG; gap is integration, not capability | [`competitive-gap-1-knowledge-graph-hindsight.md`](competitive-gap-1-knowledge-graph-hindsight.md) |
| 2  | Provider circuit breaker + automatic model failover down a precedence chain | medium | partial — concept adapted from peer; implemented against our role-routing + Foundation fail-closed (lazy recovery, no background sweep) | [`competitive-gap-2-circuit-breaker.md`](competitive-gap-2-circuit-breaker.md) |
| 3  | Token-budgeted Goal loop primitive with independent verifier | medium | yes (as sugar) — composes `agent()` + `checkpoint()` + `budget` + disk journal we already have; no new runtime state | [`competitive-gap-3-goal-loop.md`](competitive-gap-3-goal-loop.md) |
| 4  | Durable Plan mode on top of `checkpoint()` | medium | yes (as sugar) — `checkpoint()` is the harder half we already have; adds plan checksum + approval-history artifact | [`competitive-gap-4-plan-mode.md`](competitive-gap-4-plan-mode.md) |
| 5  | Enrich monitor pane with context/token footer gauges + retry progress | medium | yes — additive to #161's Option D tree; pure projection of existing snapshot fields (plus one new `attemptsUsed`/`maxAttempts` pair) | [`competitive-gap-5-monitor-pane-gauges.md`](competitive-gap-5-monitor-pane-gauges.md) |

All five are scoped **medium** complexity: each is a bounded, single-PR change
with a known file-touch list (3–9 files), no new dependencies, and no lockfile
edit. None requires a new saved workflow or a catalog/lock contract change
except where explicitly noted in the child plan doc.

---

## Sequencing recommendation (cheapest first)

Build the five issues in this order so each lands on a stable base and the
riskiest, most open-ended work is last:

1. **#5 — monitor pane gauges.** Cheapest and most isolated. Additive to #161's
   Option D tree; default-off footer behind a flag, byte-identical regression
   test as a hard gate. No runtime semantics change. Ships confidence in the
   snapshot/render path before the riskier gaps.
2. **#4 — plan mode.** Builds directly on `checkpoint()` (the harder half we
   already have). The `planApprove` primitive is thin sugar; the riskiest part
   is the plan-checksum-in-journal-identity extension, which is well-bounded.
   Landing it before #3 lets a future `goal()` or Issue Delivery lane opt into
   plan-then-approve-then-execute.
3. **#1 — Hindsight integration.** Cheapest *capability* win and the only one
   that reuses an already-running system, but gated on confirming the research
   (recall tool enablement, bank mission/entity_labels config, auto-deposit
   wiring). Land it once #5 and #4 have de-risked the render and checkpoint
   paths, so the integration PR is the only novel surface.
4. **#2 — circuit breaker.** Touches the agent() retry loop, model
   resolution, role config, and the monitor pane — the widest blast radius of
   the five. Land it after #5 has stabilized the monitor render path it will
   also touch. Lazy recovery keeps it deterministic, but it deserves the
   cleanest base.
5. **#3 — goal loop.** Most open-ended: a new primitive, a verifier contract,
   budget-exhaustion semantics, and the first primitive that composes
   `checkpoint()` for an inconclusive-pause path. It benefits from #4's
   `planApprove` (a goal loop may plan-then-approve before executing rounds)
   and from #1's knowledge recall (a goal round can pull prior findings), so
   it lands last.

**Sequencing array (for tooling):** `[5, 4, 1, 2, 3]`

Issues may be opened in parallel (each has its own plan doc), but **merge
order** should follow this sequence unless a later issue explicitly depends on
an earlier one (e.g. #3's optional `planApprove` reuse depends on #4).

---

## Non-goals (do NOT chase — rejected as not-worth-filing)

Per #137, the competitive analysis rejected the following peer capabilities as
**not worth filing** because they conflict with this fork's harness
broker/catalog contract, duplicate an existing mechanism, or trade away a
property we will not give up:

- **63-skill / 32-role declarative catalog.** Our workflows are authored
  JavaScript with an explicit quality-stdlib (`agent()`, `parallel()`,
  `pipeline()`, `dag()`, `verify()`, `judgePanel()`, `loopUntilDry()`,
  `completenessCheck()`, `checkpoint()`, and now `goal()`/`planApprove()`).
  A declarative skill/role catalog is a different authoring model, not a
  mechanism gap. Chasing it would duplicate the stdlib under a second surface.
- **Intent-to-chain NL classifier** (peer's natural-language → skill/role
  routing). We route by explicit `opts.tier`/`opts.model`/`agentType` for
  determinism and security isolation; an NL classifier would re-introduce
  nondeterminism into routing and weaken `enforceRoutingPolicy`.
- **MOA / ACO swarm** (mixture-of-agents / ant-colony swarm coordination). Out
  of scope: we already have `parallel()` + `dag()` + `judgePanel()` for
  structured fan-out/fan-in, and a swarm conflicts with the deterministic VM
  contract (no `Math.random`, no unbounded nondeterminism).
- **Vendoring any peer source.** #137 is absolute on this. No `import` from
  `pi-maestro-flow`, no copy-paste of `EndpointCircuitBreaker`, `plan.json`,
  `waves.json`, the `~/.pi/workspaces/.../plans/` tree, the TUI plan-mode
  carousel, `setFooter`, or `pi-cockpit`'s renderer-wrapping mechanism.
- **A parallel SQLite/BM25F knowledge store.** Gap #1 explicitly rejects this:
  Hindsight already provides the capability; a second store would split the
  knowledge graph and duplicate effort.
- **Cross-process / persistent circuit-breaker state.** Gap #2 rejects it: a
  restart starts with all circuits closed, preserving VM determinism.
- **A host-level Plan *mode*** (full-screen editor, `Alt+Shift+P`,
  `Shift+Tab` rebinding). Gap #4 ships a workflow-script primitive, not a host
  editor mode.
- **A live wall-clock retry countdown timer.** Gap #5 shows *attempts used /
  max* as state, not a ticking timer — our retry loop is synchronous with no
  backoff delay, and a sub-second timer would fight the ~1s render throttle.

---

## Success criteria

The epic is **done** when all five child issues have landed as bounded PRs
through the Issue Delivery closed-loop and every one of the following holds:

1. **Each gap is a bounded PR through the closed loop.** Every child issue
   ships via the Issue Delivery pipeline (Scout → Thinker → Worker →
   LocalChecks → Verifier → PR → finalization) with the closed-loop Verifier
   stage passed (the "Adversarial review (wave A)" comment posted locally
   before push) and CI green (`test`, `pi-compatibility` floor+latest,
   `package-smoke`).
2. **Codex review gate satisfied per child.** Each PR passes the Codex recheck
   workflow (4-min-cycle poll up to 12 min, early-exit on 👍 or a findings-free
   review posted after the PR's latest-push cutoff) before merge. No PR merges
   on CI-green alone.
3. **No new dependencies, no lockfile rewrite.** All five are source/test/docs
   changes only (Gap #1 touches an *extension config*, not this repo's
   lockfile; Gap #2 adds one new module, no dep). If a gap genuinely needs a
   dependency, it is surfaced and re-scoped, not silently added.
4. **#137 attribution recorded.** Each gap that adapts a peer concept (#2, #3,
   #4, #5) credits `pi-maestro-flow` in `CHANGELOG.md` and the
   `docs/workflow-engine-review.md` peer-watchlist ("adapted, not vendored").
   Gap #1 credits Hindsight as the reused native system.
5. **Infra-depth lead preserved.** No PR weakens a determinism, journal-replay,
   AFK-safe-default, or security-isolation property. Specifically:
   - #2's breaker is lazy-recovery + operator-declared failover + fail-closed
     empty chain;
   - #3's `goal()` adds no new runtime state/journal shape;
   - #4's `planApprove` is `default: false` AFK-safe with a plan-checksum in the
     journaled identity;
   - #5's footer is pure over the snapshot and default-off on the live task
     panel (byte-identical regression test green);
   - #1 adds no parallel store.
6. **Tautology-free tests.** Every child PR's tests derive expected values
   from an independent oracle (hand-computed constants, precomputed checksums,
   golden snapshots, explicit event sequences) — never recomputed by the same
   code path under test.
7. **Epic doc + child plan docs stay accurate.** This epic doc and the five
   `docs/plans/competitive-gap-*.md` docs reflect the shipped design (verdict,
   scope, non-goals, file-touch list). Any in-flight deviation is recorded in
   the child issue, not silently diverged.

When all five are merged and the success criteria above hold, the epic closes
with a final note in `docs/workflow-engine-review.md` recording that the
2026-08-13 peer-watchlist gap set has been dispositioned (adopted via bounded
issues, attribution recorded) — closing the #137 loop for this cycle.