# Release Validation Procedure (Per Provider)

**Audience**: Maintainer driving release-readiness validation across all supported agentic stacks. Run this once per provider when validating a stable release (e.g. v2026.5.13).

**Assumption**: You hold a lowest-tier paid account on each provider. The in-session agent is expected to **report** outcomes accurately but is **not** expected to self-correct or debug. Your job is to capture verbatim what the session does, not to coach it into passing.

**Scope per session**: ~30 minutes. Each step has a Pass/Fail/Unsure verdict and an evidence slot. If a step Fails or is Unsure, **note it and continue** — don't block the rest of the run on one regression.

**Where to record**: The provider-specific child issue under the validation epic (e.g. `provider-validation(<name>): v2026.5.13 release validation`). Paste transcript excerpts directly in issue comments.

---

## 0. Pre-flight (Do once, on the host machine)

Run these checks **before** opening any provider session. They verify your local install is healthy and the version under test is what you expect.

```bash
# 0.1 — Confirm install is current
aiwg version
# Expected: 2026.5.13 (or whichever stable you're validating)

# 0.2 — Confirm install health
aiwg doctor
# Expected: all checks green; if any red, fix or note the failure before continuing.

# 0.3 — Pick a clean target project (fresh dir or empty repo)
mkdir -p /tmp/aiwg-rv-<provider-slug> && cd /tmp/aiwg-rv-<provider-slug>
# (Substitute <provider-slug>: claude, codex, copilot, etc.)
```

Record outcomes in the issue under **§ 0 Pre-flight**.

---

## 1. Deploy AIWG to the provider (Regression check #1)

```bash
# 1.1 — Deploy a small framework. Use sdlc-complete unless the provider's quickref calls for something else.
aiwg use sdlc --provider <provider-slug>
# OpenClaw is user-scope-only — it deploys to ~/.openclaw/ regardless of cwd;
# omit --scope (it now defaults to user) or pass --scope user.

# 1.2 — Probe the workspace state.
# Note: migration lives at .workspace.migration and the scoped flag at .workspace.workspace.isFrameworkScoped
# (top-level .migration is null — see #1523/#1524 validation).
aiwg status --probe --json | jq '{migration: .workspace.migration, isFrameworkScoped: .workspace.workspace.isFrameworkScoped}'
```

**Pass criteria — provider-dependent.** The migration/scoped probe is a *project-local* signal; it does not apply uniformly across providers.

*Project-local providers* (claude-code, codex, copilot, cursor, factory, opencode, warp, windsurf) deploy into the project and migrate a framework-scoped `.aiwg/` workspace:
- `aiwg use` exits 0.
- `migration.status` is `"completed"` (not `"partial"`) on a fresh workspace (regression check for #1516).
- `isFrameworkScoped` is `true`.

*Home-dir / user-scope providers* (**openclaw**, **openhuman**) deploy to the home dir and **do not create a project-local framework-scoped workspace** — so `migration.status` is `"none"` and `isFrameworkScoped` is `false` **by design. That is a PASS, not a #1516 regression.** The probe above is N/A for these providers; verify the home-dir deploy instead:

```bash
# OpenClaw — deploys to ~/.openclaw/ (home-dir, user-scope)
aiwg use sdlc --provider openclaw                              # exit 0
ls ~/.openclaw/agents ~/.openclaw/rules ~/.openclaw/behaviors  # populated
grep -c "AIWG SDLC Framework" ~/.openclaw/workspace/AGENTS.md  # >= 1 — Discover-First bridge (#1562)

# OpenHuman — deploys to .agents/ + .openhuman/ + AGENTS.md bridge
ls .agents/agents .openhuman/skills                            # populated
grep -c "AIWG SDLC Framework" AGENTS.md                        # >= 1
```
- `aiwg use` exits 0.
- The provider's deploy directories are populated.
- The context bridge is present (workspace `AGENTS.md` for openclaw; project `AGENTS.md` for openhuman).

**Evidence to paste**: the step 1.2 output, plus — for home-dir providers — the deploy-directory listing and bridge grep.

---

## 2. Deploy AIWG hooks (Regression check #2 — Claude Code only)

This step only applies if the provider is Claude Code. Skip for all other providers.

If you're on a workstation that **had AIWG installed before May 10, 2026** (pre-`be3ee551`):

```bash
# 2.1 — Show the current SessionStart hook reference in settings.json
jq '.hooks.SessionStart' .claude/settings.json
# Expected: command ends in "aiwg-session.cjs". If it ends in ".js", the v2026.5.13 hook fix has not yet
# been applied to this settings.json.

# 2.2 — If you saw ".js" above, refresh to trigger the heal
aiwg refresh
# Re-run 2.1 — command should now end in ".cjs".
```

**Pass criteria**: `.cjs` path in the hook command after refresh. If the path was already `.cjs` (new install), step 2.2 was unnecessary — record that as Pass.

---

## 3. Open a fresh provider session

Open the provider's UI / CLI / IDE — whatever is the canonical entry point for that provider — in the project directory from step 0.3. Make sure the deployed AIWG files are visible to the agent (no startup error referring to a missing file).

**Pass criteria**: Session opens. Agent acknowledges. No error banners about missing skills, missing config, missing hooks, or `MODULE_NOT_FOUND`.

**Evidence**: Copy any startup banner or error verbatim.

If the session fails to open or shows hook errors: STOP this provider's pass and report the error in the issue. Don't attempt to fix.

---

## 4. Discovery hookup — does the agent know how to find AIWG things?

Type **exactly this** into the session, verbatim:

> Find an AIWG skill that handles intake forms.

Watch what the agent does. Record which of the following it uses (one or more):

| Hook | Looks like |
|---|---|
| **rule** | Mentions the discover-first rule, or runs `aiwg discover` directly. |
| **config / context file** | Cites `AGENTS.md`, `AIWG.md`, `WARP.md`, `.github/copilot-instructions.md`, or the provider's equivalent context file. |
| **quickref** | Uses a loaded AIWG quickref skill (mentions `sdlc-quickref`, `aiwg-utils-quickref`, etc.). |
| **discovery-agent** | Delegates to `aiwg-finder` or the provider's curator / subagent equivalent. |
| **none** | Improvises from training data; never cites AIWG-specific surface. |

**Pass**: at least one hook fires AND the agent surfaces a real intake-related skill (`intake-wizard`, `intake-start`, or `intake-from-codebase`).

**Fail**: agent declines, claims AIWG has no such skill, or invents a skill name not in the index.

**Evidence**: paste the agent's reply verbatim (truncate to ~20 lines if very long).

---

## 5. Skill body fetch — can the agent read a deployed skill?

Type **exactly this** into the session:

> Read the AIWG quickref for AIWG utility discovery and summarize the fallback when a skill is not loaded.

Record the read-access behavior:

| Path | Looks like |
|---|---|
| **`aiwg show`** | Agent runs `aiwg show skill aiwg-utils-quickref` (or equivalent) and quotes the content. |
| **Direct file read** | Agent reads a `.md` file via its native Read tool. Acceptable for providers where `aiwg show` isn't reachable. |
| **MCP `rule-show` / `skill-show`** | Hermes only — agent uses MCP tool. |
| **Failure** | Agent guesses, declines, or references a non-existent path. |

**Pass criteria**: any of the first three (real content surfaces). **Fail**: guesswork or refusal.

**Evidence**: paste the agent's summary verbatim, especially the fallback description.

---

## 6. Skill invocation — does a representative skill actually run?

Type **exactly this**:

> Run aiwg discover for "deploy production" and tell me the top three results.

**Pass criteria**: agent either runs the `aiwg discover` command (CLI dispatch) or reports the top results via the index. The expected #1 result is `flow-deploy-to-production` (score ~0.51).

**Fail**: agent invents results, declines without trying, or surfaces names not in the deployed corpus.

**Evidence**: paste the discover output verbatim.

---

## 7. Provider-specific normalization verification

Each provider has gaps in what it natively offers; AIWG fills those gaps so behavior across the 11 stacks is comparable. Step 7 verifies that AIWG's normalization is actually doing its job for the provider under test. Skip the rows tagged `n/a` for providers you're not validating.

Each row has a **gap** (what the vendor doesn't do natively), an **AIWG normalization** (what AIWG ships to cover it), and a **verification** (what you check to prove the normalization landed). Pass = all rows green. Fail any row → record + continue.

### Universal pre-check (all providers)

Before opening the per-provider matrix, confirm the universal artifact-presence baseline:

```bash
# Verify AIWG knows where it deployed and what
aiwg list                                              # framework presence
aiwg discover "intake wizard" --limit 3 --json         # discovery surface live
aiwg show skill aiwg-utils-quickref | head -10         # quickref reachable
```

---

### 7-A. Claude Code

| Gap (vendor side) | AIWG normalization | Verification |
|---|---|---|
| No declarative agent definitions — they live in markdown files only | Ships `.claude/agents/*.md` with frontmatter Claude Code parses | `ls .claude/agents/ \| wc -l` ≥ 1; open one in the IDE, confirm it shows in the agent picker |
| No project-level rule auto-load contract — only system prompt | Ships `.claude/rules/*.md` discovered by Claude Code's rules-directory loader; `CLAUDE.md` references AIWG context via `@AIWG.md` which transitively pulls the rules in | `ls .claude/rules/ \| wc -l` ≥ 50 (real deployed corpus); `grep -E "^@AIWG\.md" CLAUDE.md` returns 1; rules load on session start (verify by asking the agent which rules it's following). Note: `.claude/rules/` is the load path, not `@`-mentions in CLAUDE.md directly — that was the original verification's mistake. |
| Hook scripts must be `.cjs` because `"type": "module"` makes `.js` load as ESM and `require()` fails | Hooks installer (#1308) deploys `aiwg-{session,permissions,trace}.cjs` and re-writes stale `.js` paths in `settings.json` (v2026.5.13 fix) | `jq '.hooks.SessionStart' .claude/settings.json` shows a command ending in `.cjs`. Open a session — no `SessionStart:startup hook error` / `cjs/loader:1459` banner. |
| ~455 of ~480 skills exceed the listed-skills budget (25% of context window) | Kernel skill set (~20) is always-loaded; remainder reached via `aiwg discover` / `aiwg show` per the `skill-discovery` rule | Ask the session "How many AIWG skills are available?" — agent should answer "~480 reachable via aiwg discover, ~20 in the kernel set" or invoke discover instead of enumerating |
| No marketplace plugin install workflow without a marketplace.json | Ships `.claude-plugin/marketplace.json` with 13 plugins published | `/plugin marketplace list` in Claude Code shows AIWG; `/plugin install sdlc@aiwg` succeeds |

### 7-B. OpenAI Codex

| Gap (vendor side) | AIWG normalization | Verification |
|---|---|---|
| Commands are a static built-in enum in `codex-rs` — file-based commands are NOT scanned by the loader | `.codex/commands/` is deployed for operator visibility but the discovery bridge is `AGENTS.md` (linked aggressively per the 32KB cap) | Open `AGENTS.md` at repo root — it links to `.aiwg/AIWG.md`, frameworks, and rules. Size is well under 32KB (`wc -c AGENTS.md`). Inside session: agent uses AGENTS.md context, not the `.codex/commands/` files |
| `~/.codex/` is home-scope (cross-project) by Codex design — project artifacts not discovered automatically | Deploys to BOTH `~/.codex/{prompts,skills}/` (home) AND `.codex/` (project mirror); AGENTS.md is the project-scope bridge | `ls ~/.codex/skills/ \| wc -l` ≥ 1; `ls ~/.codex/prompts/ \| wc -l` ≥ 1 |
| No native Skill tool dispatch — agent has no way to invoke a skill body | Agents must use `aiwg show skill <name>` via shell tool — discovery returns paths the agent fetches via Bash | Step 5 already proves this; in this session verify the agent uses `aiwg show` (not a direct path read) and the CLI returns content |
| OIDC trusted publishing not native to Codex workflow tooling | Documented in `docs/contributing/versioning.md`, automated via GitHub Actions OIDC step in `.github/workflows/npm-publish.yml` | N/A for in-session validation — release-pipeline concern |
| 32KB AGENTS.md cap (per Codex doc) means inlining everything overflows | `aiwg use` and `aiwg regenerate` emit a link-heavy AGENTS.md, not a content dump | `wc -c AGENTS.md` < 32000 |

### 7-C. GitHub Copilot

| Gap (vendor side) | AIWG normalization | Verification |
|---|---|---|
| Agents must use Copilot's `.agent.md` format (markdown + YAML frontmatter) with specific keys | Agents installer writes `.github/agents/*.agent.md` with correct frontmatter | `head -10 .github/agents/$(ls .github/agents/ \| head -1)` shows `---` frontmatter with `name`, `description`, `tools` |
| Rules are not free-form — they deploy as path-scoped `.github/instructions/*.md` with `applyTo:` frontmatter glob | Rules installer rewrites AIWG rules into instruction files with the right `applyTo` matchers | `head -10 .github/instructions/$(ls .github/instructions/ \| head -1)` shows `applyTo: "**/*.md"` or similar; activating on a matching file fires the rule |
| Commands deploy as `.github/prompts/*.prompt.md`, not loose markdown | Prompt files generated with `.prompt.md` extension | `ls .github/prompts/*.prompt.md \| wc -l` ≥ 1; invokable in Copilot UI |
| No discrete skills concept — only prompt files | Skills deploy to `.github/skills/` AND `.agents/skills/` (cross-agent mirror) | Both directories populated; agent in session reaches them via either path |

### 7-D. Cursor

| Gap (vendor side) | AIWG normalization | Verification |
|---|---|---|
| Rules auto-activate via glob frontmatter in `.cursor/rules/*.mdc` — not free-form `.md` | Rules installer generates `.mdc` files with correct `globs:` frontmatter | `head -8 .cursor/rules/$(ls .cursor/rules/*.mdc \| head -1)` shows `globs:` field; rule fires when matching file is open |
| Agents directory naming differs from Claude Code (`.cursor/agents/` vs `.claude/agents/`) | Provider adapter deploys to `.cursor/agents/` | `ls .cursor/agents/ \| wc -l` ≥ 1 |
| No Skill tool — skills are just files | `aiwg show` is the dispatch path; `.cursor/skills/` deployed for visibility | Step 5 already proves the agent uses `aiwg show` |
| Cursor context window varies by tier; agent should defer-to-discover, not eagerly load | Discovery-first rule deployed; quickref kernel set kept small | Ask session "list all AIWG skills" — agent should run discover, not attempt to load all |

### 7-E. Warp Terminal

| Gap (vendor side) | AIWG normalization | Verification |
|---|---|---|
| Warp reads single-file `WARP.md` natively (no rules-directory contract) | `aiwg use --provider warp` emits a **lean managed `WARP.md` bridge** — Discover-First Protocol + Context Finalization + Parallelism Cap near the top — not a full rules aggregate; discrete rules/commands/behaviors deploy under `.warp/` | `grep -c 'aiwg-managed' WARP.md` == 1 (lean managed bridge present) and `WARP.md` leads with the Discover-First Protocol. Do NOT expect a large `^### `-rule aggregate — that's the obsolete shape (`grep -c "^### " WARP.md` ≥ 10 no longer applies) |
| Warp has no native agents picker/directory contract for AIWG agents | Agents reachable via `aiwg discover` / `aiwg show`; the lean `WARP.md` bridge points at discovery rather than inlining an agent index (any `.warp/agents/` is operator-browsing only, not required) | Agent in session reaches AIWG agents via `aiwg discover` / `aiwg show` (not an inlined WARP.md agent dump) |
| Commands deploy to `.warp/commands/` (discrete) AND aggregated section in WARP.md | Both surfaces populated | `ls .warp/commands/ \| wc -l` ≥ 1; WARP.md has a "Commands" section |
| Behaviors not natively supported — aggregated into rules | `.warp/rules/behaviors/` directory contains behavior rules | `ls .warp/rules/behaviors/ \| wc -l` ≥ 1 |

### 7-F. Factory AI

| Gap (vendor side) | AIWG normalization | Verification |
|---|---|---|
| Agents live in `.factory/droids/` — different naming convention (droid, not agent) | Provider adapter renames on deploy and applies Factory's droid frontmatter | `ls .factory/droids/ \| wc -l` ≥ 1; droid frontmatter present |
| Commands and skills use `.factory/{commands,skills}/` | Standard deployment | Both populated |
| MCP-via-Factory binding is provider-specific | Configured via `.factory/` MCP config when relevant | If MCP servers configured: `aiwg mcp list` shows Factory-bound entries |

### 7-G. OpenCode

| Gap (vendor side) | AIWG normalization | Verification |
|---|---|---|
| Directory names are **singular** (`agent/`, `command/`, `skill/`, `rule/`) — not plural | Provider adapter writes to the singular path names | `ls .opencode/agent/ .opencode/command/ .opencode/skill/ .opencode/rule/` all exist and populated |
| Skills also need a cross-agent mirror at `.agents/skills/` | Both locations populated by deploy | `ls .agents/skills/ \| wc -l` ≥ 1 |
| OpenCode does not enforce frontmatter strictly; AIWG ships it anyway for portability | Files retain frontmatter even when not required | `head -5 .opencode/agent/$(ls .opencode/agent/ \| head -1)` shows `---` frontmatter |

### 7-H. Windsurf

| Gap (vendor side) | AIWG normalization | Verification |
|---|---|---|
| Windsurf reads `AGENTS.md` + `.windsurf/rules/` natively; there's no native agents picker, so agent reachability must be normalized | `aiwg use --provider windsurf` deploys agents to `.windsurf/agents/*.soul.md` (+ `.aiwg-manifest.json`) and emits a **lean discover-first `AGENTS.md` bridge** (not an inlined agent dump) | `ls .windsurf/agents/*.soul.md \| wc -l` ≥ 1 **AND** `grep -c 'aiwg-managed' AGENTS.md` == 1 (regenerated lean bridge present). Agents are reachable via `.windsurf/agents/` + the `.agents/skills/` mirror + `aiwg discover`/`aiwg show` — NOT via an obsolete inlined `^## Agent: ` block |
| Commands deploy to `.windsurf/workflows/` (not `commands/`) | Provider adapter uses correct path name | `ls .windsurf/workflows/ \| wc -l` ≥ 1 |
| Rules use `.windsurf/rules/*.md` with Windsurf-specific frontmatter | Generated with correct frontmatter | `head -8 .windsurf/rules/$(ls .windsurf/rules/ \| head -1)` shows expected frontmatter |
| Skills get `.windsurf/skills/` AND `.agents/skills/` mirror | Both populated | Both directories present |

### 7-I. OpenClaw

| Gap (vendor side) | AIWG normalization | Verification |
|---|---|---|
| All artifacts deploy to `~/.openclaw/` (home-scope) — not the project | Provider adapter writes to home; `aiwg use sdlc --provider openclaw` populates home | `ls ~/.openclaw/{agents,commands,skills,rules}/ \| wc -l` ≥ 4 |
| First provider with **native behavior** support — all other providers receive generated behavior rules | Behaviors deploy natively to `~/.openclaw/behaviors/`; rules-derived behaviors deploy elsewhere | `ls ~/.openclaw/behaviors/ \| wc -l` ≥ 1; the files are real behavior bundles, not rule-generated stubs |
| OpenClaw's skill-listing budget is hard cap 150 | Same kernel-skill strategy as Claude Code | Inside session, "list all skills" — agent uses discover, not enumeration |
| Cross-project context lives in home; project-scope context loaded on session-start | Documented in OpenClaw deploy notes | Open OpenClaw on a fresh project — AIWG content from `~/.openclaw/` is reachable without re-deploying per-project |

### 7-J. Hermes

| Gap (vendor side) | AIWG normalization | Verification |
|---|---|---|
| Hermes baseline is file deploy + CLI discovery; MCP is optional sidecar enrichment | AGENTS.md/.hermes.md generated, skills deployed under `~/.hermes/skills/`, optional commands reach AIWG via `mcp_aiwg_command_run` | `AGENTS.md` names `aiwg discover`; optional MCP session lists `mcp_aiwg_command_run` |
| 16 core MCP tools by default; 45 more via `AIWG_MCP_TOOLSETS=<csv>` | Default tools available; opt-in extras via env var | Inside session, list MCP tools — count is 16 (or 61 if `AIWG_MCP_TOOLSETS=all`, including deprecated compatibility tools) |
| Standard skills (~460) recursively discovered under `~/.hermes/skills/.aiwg/`; kernel (~20) at top level | Curator (v0.12.0+) protects kernel via `.bundled_manifest` | `ls ~/.hermes/skills/ \| grep -v "^\.aiwg$" \| wc -l` ≥ 20 (kernel); `find ~/.hermes/skills/.aiwg/ -name SKILL.md \| wc -l` ≥ 300 (standard) |
| `AGENTS.md` + `.hermes.md` are the discovery bridges (no provider-native context file) | Both files generated by `aiwg use --provider hermes` | Both files present at project root |
| Rules inlined in AGENTS.md + reachable via CLI `aiwg show rule`; MCP `rule-list`/`rule-show` optional | Non-MCP and MCP surfaces populated | `grep -c "^### Rule:" AGENTS.md` ≥ 7; `AGENTS.md` contains `aiwg show rule <name>`; optional MCP `rule-list` returns rule names |

### 7-K. Omnius

| Gap (vendor side) | AIWG normalization | Verification |
|---|---|---|
| Omnius is a **first-party AIWG integrator** — no separate `aiwg use` step; AIWG ships embedded in the Omnius runtime | `npm i -g omnius` installs both; `aiwg discover` reachable inside Omnius sessions without project setup | Inside Omnius session: `aiwg discover "intake wizard"` returns ranked results without prior `aiwg use` |
| Omnius exposes AIWG via REST (`/v1/aiwg/*`) and MCP bridges | Bridges live in Omnius runtime, not in this repo | Inside session, agent can fetch a skill body via REST or MCP — paste agent's invocation verbatim |
| All AIWG-aware rules apply (`skill-discovery`, `delivery-policy`, `human-authorization`, `anti-laziness`) even though no provider-specific deploy ran | Embedded distribution carries the rules | Ask session "Which AIWG rules are active?" — agent should name at minimum the four above |
| Bundled runtime versioning may lag — Omnius releases on its own cadence | AIWG release ↔ Omnius bundled version is tracked separately | Check the Omnius release notes for the AIWG version it ships (compare to current `aiwg version` host install) |

---

Run the verification column commands. Paste any unexpected output. **Pass**: all rows green for the provider under test. **Fail**: at least one row shows the normalization didn't land — record which one in the issue.

---

## 8. Final verdict per provider

In the issue, at the top of your last comment, state one of:

```
VERDICT: PASS — all 7 checks green.
VERDICT: PASS WITH NOTES — N failures captured below, none release-blocking.
VERDICT: FAIL — release-blocking issue; details below.
```

Then list each step's outcome:

```
0. Pre-flight: PASS / FAIL / UNSURE
1. Deploy: PASS / FAIL / UNSURE
2. Hook heal (Claude Code only): PASS / FAIL / UNSURE / N/A
3. Session opens: PASS / FAIL / UNSURE
4. Discovery hookup: PASS / FAIL / UNSURE
5. Skill body fetch: PASS / FAIL / UNSURE
6. Skill invocation: PASS / FAIL / UNSURE
7. Provider regression spot check: PASS / FAIL / UNSURE
```

For every Fail or Unsure: a one-line note in the comment, AND a follow-up issue with the provider name in the title (so the matrix epic links to it).

---

## What a release-blocking failure looks like

These warrant immediate operator notice; everything else is captured as a known issue and rolled forward:

- `aiwg use` exits non-zero on a clean target.
- `aiwg status --probe` reports `migration: partial` on a fresh deploy (regression of #1516).
- Claude Code session shows `SessionStart:startup hook error → MODULE_NOT_FOUND` (regression of v2026.5.13 hook fix).
- Provider session refuses to open citing a missing AIWG file.
- Discovery hookup returns invented (hallucinated) skill names instead of real ones.

Other failures (one provider's discover doesn't fire, one provider's `aiwg show` isn't reachable, etc.) are captured as the per-provider follow-ups they always have been.

---

## When all child issues have a verdict

Close the epic with a summary comment of the form:

```
| Provider | Verdict | Notes |
|---|---|---|
| Claude Code | PASS | — |
| Codex | PASS WITH NOTES | discovery uses prompt-file path, not aiwg show |
…
```

This is the artifact future releases will reference when deciding "did we ship a regression on provider X."