# CAI Strategy — Internal Tool Pivot **Date:** 2026-06-25 **Decision owner:** Kai **Status:** Proposed (paper decision — no code changes yet) --- ## 1. The decision in one line CAI stops being a product and becomes **a sharp internal tool: the deterministic, zero-token drift gate for AI project context, shaped to our own multi-agent workflow.** We drop the token-savings pitch, stop competing on "memory that loads itself," and keep only the part nothing else can do for us. ## 2. Why this, and why now The competitive research (9 agents, 6 rivals) reached a blunt verdict: *as a product*, CAI is dead — `mex` is a strict superset, Claude Code now ships native memory + CLAUDE.md drift flagging, and CAI's headline token claim ships with zero measurement. **But "internal tool" dissolves the verdict.** Every damaging finding was about winning users: | Research finding | Why it stops mattering internally | |---|---| | mex has 11 checkers, 1.1k stars, more features | Stars don't help us. CAI is *ours* — shaped to our repos, our agents, our CI. We can take mex's good ideas without adopting mex. | | Platform absorbed the scaffold/memory story | We were never going to out-engineer Anthropic. We don't need to — we need the one thing they structurally won't build. | | No public traction, last entrant | Irrelevant. Audience = us. | | Token claim unmeasured | Internally this is a *gift*: we can finally measure whether the scaffold pays for itself on our actual work, with no marketing pressure to inflate. | What remains is real and **uniquely un-absorbable**: native drift handling is LLM *judgment* against transcripts/CLAUDE.md text. It never checks docs against the **live codebase**, it's per-user/uncommitted, and it's Claude-only. CAI's checkers verify scaffold-vs-actual-code deterministically, in CI, across every agent, committed to git. That gap is the whole reason to keep CAI alive. ## 3. What CAI is FOR (internal mission) > Keep our AI agents' shared, committed project context honest — automatically, deterministically, across every tool we use — and tell us the truth about whether it's worth it. Three jobs, in priority order: ### Job 1 — Cross-tool config drift gate *(highest leverage)* We run multiple agents (Claude Code today; Cursor/Copilot/Codex in the mix). `CLAUDE.md`, `.cursorrules`, `AGENTS.md`, `copilot-instructions.md` silently diverge — each tool reads its own file, nobody keeps them in sync. CAI's `tool-config` + `command`/`dependency`/`path` checkers already catch "this doc references a script/dep/path that no longer exists." Make that the marketed core: **one source of truth, drift-checked across all agents, failing CI when it lies.** Native is Claude-only; this is the daily-annoyance killer. ### Job 2 — Team-shared committed memory Native auto-memory is per-developer and uncommitted. For shared repos (ReqPilot, the LHIND projects) we want project memory that lives in git, is diffable in PRs, and is the same for everyone and every agent. CAI's scaffold already *is* this — the value is the committed + drift-checked combination, not the scaffold for its own sake. ### Job 3 — Honest self-measurement (`cai bench`) Settle the question the product never answered: **does the scaffold actually save tokens on our workloads, or is it ceremony?** Build a paired-session A/B harness (see §6). Lead with the real number even if it's modest or negative. Internally, an honest "−5% on short tasks, +40% on knowledge-heavy ones, crossover at ~X" is worth more than any marketing multiplier. ## 4. What we CUT An internal tool should be *smaller*, not feature-complete. Dead weight to remove or stop investing in: - **The token-savings README pitch** — delete or downgrade to an honest, measured statement once `cai bench` exists. Keeping the unmeasured claim is dishonest with ourselves. - **Product-shaped surface** — multi-tool install ceremony, the "5-minute setup," marketing framing in README/SETUP. Internal tools don't onboard strangers. - **Anything that races mex on breadth** — TUI, event log/timeline, heartbeat mode, agent-memory mode for non-code agents. These are mex's scope-creep; we don't need them and shouldn't copy them. *(If one proves useful later, add it deliberately — not to match a feature list.)* - **The "navigable memory saves tokens" thesis as a selling point** — the platform does navigable memory now. We keep the scaffold only as the *substrate the drift checkers verify*, not as the headline. **Keep and harden:** the drift engine (`src/drift/`), the checkers, the scoring, the `ScaffoldLayout` + single-manifest refactors we just did, the JSON output path (it's what CI consumes), `cai check`. ## 5. The thin-but-real moat (internal version) CAI is worth maintaining for us *only* because it does what platform + mex don't do *for our setup*: 1. **Deterministic, zero-token, runs-in-CI** — a hard gate, not an LLM's opinion, with no token cost and no flakiness. 2. **Codebase-grounded** — verifies docs against real files/scripts/deps/versions, catching docs that drifted from code (not just memory that drifted from CLAUDE.md). 3. **Multi-tool, one source of truth** — Cursor/Windsurf/Copilot/Codex/Claude kept consistent; not Claude-only. 4. **Committed + team-shared** — version-controlled, PR-reviewable, identical for everyone. If a path forward doesn't strengthen one of these four, it's out of scope. ## 6. `cai bench` — the measurement design (so we're not guessing) Mirror `rtk gain`'s rigor. The honest method: 1. Fixed task suite, 6–10 representative tasks from our real work (add endpoint, fix bug, explain a module, extend a pattern). 2. Two arms: **A** = flat baseline (one fat CLAUDE.md with the same knowledge inline, or no memory); **B** = CAI scaffold (CLAUDE.md → ROUTER → context → pattern). 3. Identical prompts, identical model, **cold sessions**, **N ≥ 5 repeats per cell** to control output variance. 4. Capture **ground-truth tokens from the usage API / transcript** — input vs output vs cache-read — not estimates. 5. Report **median + variance**, per-task, and **explicitly net out the always-loaded scaffold input tax** paid every turn. 6. State the **crossover point**: below what output volume does the scaffold cost more than it saves? 7. Hold **task success/quality constant** — cheaper-but-worse is not a win. Expected honest result: modest savings concentrated on knowledge-heavy/long-output tasks, possibly a loss on short ones. Disclosing that is the point. ## 7. Sequencing (proposed — not started) 1. **This doc** → agree the direction. ← *we are here* 2. **Reposition README/AGENTS** to the drift-gate mission; strip the token pitch. 3. **Prototype the CI gate** — `cai check --json` in a GitHub Action / OpenShift pipeline, fail on drift-score threshold. Prove Job 1 end-to-end on one real repo (ReqPilot?). 4. **Build `cai bench`** — get our own numbers; let data confirm or kill the scaffold-keeps-its-keep assumption. 5. **Harden cross-tool sync** (Job 1) based on what the CI gate surfaces in practice. Each step is independently useful and stoppable. We never commit to the whole arc up front. ## 8. The one risk to stay honest about Even internally, the closest competitor to "use CAI" is **"just use mex"** (it does all this, more maturely) or **"just use native + a 5-line CI grep."** CAI earns its keep only if (a) it's shaped to our workflow in ways a generic tool isn't, and (b) the maintenance cost stays low. If we find ourselves rebuilding mex feature-by-feature, we've lost the plot — at that point, adopt mex and delete CAI. **The internal-tool bet is only rational while CAI stays small, specific, and ours.** --- ### Decision requested Agree the mission (§3), the cut list (§4), and the sequencing (§7)? If yes, next concrete step is §7.2 + §7.3 (reposition + prototype the CI gate on a real repo).