--- name: audit-dev-tests description: Audit acceptance-test coverage of BA Acceptance Criteria (DEV-TEST-001..004 + 008 + 011, BLOCKING — incl. stale-trait AC churn, stub assertions and parse-LOST ACs) and business-rule test parity (DEV-TEST-009, BLOCKING — every enforceable err rule of règles-métier.md carries a MODULE-SCOPED [Trait("BR", …)] test), plus advisory checks (DEV-TEST-005..007 anti-patterns; DEV-TEST-010 — the traced [Fact] body carries the status/error tokens its AC pins verbatim, warn). Deterministic, runs after /ba-develop Phase 4 (Acceptance Tests). group: D phase: devTests kind: audit audit_only: true section_label: 'AUDIT-DEV-TESTS (rules to apply against generated AC tests vs the BA use-case.md ACs)' allowed-tools: [Read, Glob, Grep, Bash] # Bash: CLI invocation --- # audit-dev-tests — Acceptance Tests Coverage Audit ## Context You are auditing the output of **Phase 4 of `/ba-develop`** (Acceptance Tests). Compare the BA-declared Acceptance Criteria (`#AC-NN` pairs in every `
/use-case.md`) against the generated xUnit tests (`[Trait("AC","…")]` on each `[Fact]`), and ensure: 1. Every BA AC has a generated `[Fact]`. 2. No generated `[Fact]` references a stale (non-existent) BA AC. 3. No `// TODO[AC-` marker remains in the generated test bodies (Phase 4 must have filled them all). 4. Every UC declares at least one Acceptance Criterion — a UC with none is untestable (the scaffolder emits no `[Fact]`, so rule 1 has nothing to check). This is the **dev-readiness gate for vibecoded code** — it makes the BA acceptance criteria into a deterministic, executable contract. ## Rules ### DEV-TEST-001 — Every BA AC has a generated [Fact] For every `#AC-NN` declared in any `/
/use-case.md` (under the UC's `**Acceptance Criteria**` field), the generated test code under `/Tests/**/*.cs` contains a `[Fact]` carrying `[Trait("AC", "#AC-NN")]`. A missing trait means an AC has no test — the BA contract is not enforceable. - **Severity**: **err** (BLOCKING). - **Fix**: re-run `/ba-develop` Phase 4, or invoke `scaffold-tests-from-ac --spec '{"moduleDir":...,"projectPath":...,"appCode":...,"module":...}'` manually, then commit. ### DEV-TEST-002 — No `// TODO[AC-` marker remains Every generated AC test file MUST have its body filled — no `// TODO[AC-NN]:` marker may remain. The presence of even one means Phase 4's TODO-filling pass was incomplete or skipped, leaving an `Assert.Fail(...)` sentinel that does NOT actually assert the AC. - **Severity**: **err** (BLOCKING). - **Fix**: re-run Phase 4's TODO-filling subagent for the listed files, or edit the offending test bodies by hand using the AC text in the file comment. ### DEV-TEST-003 — Every test [Trait("AC",…)] resolves to a real BA AC The reverse direction of DEV-TEST-001: every `[Trait("AC", "")]` in generated code MUST point at a `#AC-NN` pair that still exists in the BA. Catches stale tests left over after an AC was deleted or renumbered upstream — a green `[Fact]` claiming a contract that no longer exists is a **coverage lie**, and the AC that replaced it may be exactly the one with no test (the AC-churn shape: UC-014 warn upstream + warn here used to let it ship with every gate green). - **Severity**: **err** (BLOCKING — was warn until 2026-08; UC-014 id-hygiene signals — gaps, duplicates — stay warn upstream, this rule blocks only the stale-claim shape). - **Fix**: delete the orphan `[Fact]`, restore the BA AC (re-run `/ba-create-use-case`), or — after a renumbering — re-run `scaffold-tests-from-ac` so the traits follow. ### DEV-TEST-008 — Every UC declares ≥ 1 Acceptance Criterion The coverage floor. `scaffold-tests-from-ac` emits one `[Fact]` **per AC**, so a UC that declares **no** `**Acceptance Criteria**` field produces zero tests and DEV-TEST-001 has nothing to compare — the "every UC is tested" guarantee is silently vacuous. This rule fails the gate on any parsed UC with zero ACs. - **Severity**: **err** (BLOCKING). - **Fix**: add at least one `- [ ] AC-NN — …` bullet under an `**Acceptance Criteria**` field for each listed UC via `/ba-create-use-case`, then re-run `/ba-develop` Phase 4. - **Relation to `/ba-audit-use-cases` UC-012**: UC-012 is the *BA-document* audit (exempts subfunction/summary UCs, verdict written pre-dev to `/_audit/use-case.md`). DEV-TEST-008 is the *dev gate* on the generated test suite — defense in depth. It carries no Cockburn level, so it floors **every** UC the scaffolder sees, and also catches a hand-edited `use-case.md` or a skipped BA audit that UC-012 never ran on. ### DEV-TEST-009 — Business-rule test parity (BLOCKING) - Every **enforceable `err` rule** of the module's `règles-métier.md` (parsed by `lib/ba-rules-rows.ts` — module + section docs, deepest scope) carries at least one test tagged `[Trait("BR", "BR-NNN")]` under **`Tests/{Module}/`** — BR codes are DOC-scoped, so traits are counted MODULE-SCOPED (path-segment match, same strict posture as DEV-API-008's trace leg): another module's `BR-001` never satisfies, and is never flagged stale against, this module's `BR-001`. When NO test file carries the module segment at all (legacy flat `Tests/` layout), the check falls back to the project-wide scan and **says so** (warn; the orphan-stale leg is then skipped). A rules doc with ZERO required rules reports the true zero as an `ok` (never silence); a module with no `règles-métier.md` at all emits nothing — that absence belongs to the BA phase (DEV-API-008 `no-rules-declared` reads the same source). The shared exemption doctrine applies (`ruleExemption`, same as PRD-129): `access` → the RBAC matrix, `numbering` → codePattern (DEV-API-022), `info` severity → observation, `- **Enforcement** : plateforme|manuel` → authored opt-out — each exemption names its REAL channel, never a silent skip. - **Severity**: **err** per missing rule (BLOCKING); **warn** when a `[Trait("BR", …)]` cites a code matching NO rule of the module (stale claim after a renumber/removal — false comfort). - **Fix**: write one test per listed rule from its `Cas valides` / `Cas invalides` (a `[Theory]` over the examples where they are data-like), tagged with the trait — see `/test-conventions`. `scaffold-tests` emits the tagged floor test for every rule riding the spec's `businessRules[]`. - History: the ba-develop gate carried this line as PROSE (« count of Category=Business tests ≥ rules ») with no implementation — nothing in the repo parsed `règles-métier.md`, so the vacuous green was structural. This rule is the real floor; vacuity is impossible by construction (zero rules → the true zero is reported, never asserted green). ### DEV-TEST-011 — No AC lost at parse (BLOCKING) The audit-side twin of the shared parser's `lost` channel (V0): a malformed or duplicate AC bullet is an assertion the contract does NOT carry — no `[Fact]` will ever cover it, and DEV-TEST-001/008 cannot even see it (the AC is simply absent from the expected set). The scaffolder exits 1 on these, but `/ba-develop` is never-halt and can swallow that exit — THIS finding is the gate that cannot be swallowed. - **Severity**: **err** (BLOCKING). - **Fix**: fix each listed bullet in its `use-case.md` (shape: `- [ ] AC-NN — text`, ids unique within the UC), then re-run `scaffold-tests-from-ac`. ### DEV-TEST-004 — No stub assertion (BLOCKING) `Assert.True(true)` (C#) / `expect(true).toBe(true)` (Vitest) is not a test — it is **false coverage wearing a green badge**: the `[Fact]` satisfies DEV-TEST-001 through its trait while asserting nothing. The generators never emit the shape; only hand-written drift trips it, and that drift blocks. - **Severity**: **err** (BLOCKING — was warn until 2026-08). - **Fix**: replace each stub with a real assertion exercising the behaviour under test (the AC text in the `// AC:` comment is the spec). ### DEV-TEST-005..007 — Test-convention anti-patterns (advisory) Distilled from the SmartStack.app `test-conventions` corpus. These are **`warn` only — they never block the gate** (consistent with keeping the vibecoding flow unobstructed); the generators are already corrected so freshly-scaffolded tests never trip them. They catch hand-written drift across `Tests/**/*.cs` and `tests/**/*.test.{ts,tsx}`. | Rule | Flags | Use instead | |------|-------|-------------| | **DEV-TEST-005** | a DateTime compared with `BeGreaterThan` / `BeLessThan` | `BeOnOrAfter` / `BeBefore` / `BeCloseTo(…, TimeSpan.FromSeconds(n))`. | | **DEV-TEST-006** | a hardcoded GUID literal (`new Guid("…")` / `Guid.Parse("…")`) | `Guid.NewGuid()` (anti-enumeration; no cross-test coupling). | | **DEV-TEST-007** | `CancellationToken.None` in a test | `TestContext.Current.CancellationToken` (xUnit v3). | - **Severity**: **warn** (advisory — does not affect exit code). - **Fix**: edit the listed files per the “use instead” column. ### DEV-TEST-010 — The [Fact] body confronts the AC's verbatim tokens (advisory) The deterministic slice of « does the test really test its AC »: when the AC text pins an HTTP status (`400`) or a dotted error code (`crm.opportunity.amount-positive`) **verbatim**, the traced `[Fact]`'s body must carry that token — digits **or** the `HttpStatusCode` name for statuses (`400` ⇔ `BadRequest`). Comment lines are stripped first (the generated body quotes the AC in a `// AC:` comment, which would cover every token trivially); TODO bodies are skipped (DEV-TEST-002 owns them); a body the extractor cannot segment makes **no claim**. - **Severity**: **warn** — assumed limit: semantic fidelity stays judgment; this catches the most frequent class (an assertion contradicting the AC's pinned figure), it does not pretend to close the whole does-the-body-match-the-AC question. - **Fix**: assert the pinned token in the test body — or fix the AC if the figure changed. ## Invocation ```bash npx --prefer-offline tsx skills/development/audit-dev-tests/cli/audit-dev-tests/index.ts \ --spec '{"moduleDir":".smartstack/ba/CRM/PIPELINE","projectPath":"./generated/crm"}' \ --json ``` The CLI returns an `AuditReport` with `findings[]` (severity + rule code + the offending IDs + fix suggestion). Exit code is non-zero on `err`. ## Notes - Parser reuse: this skill imports `parseAcFromUseCaseMd` from `templates/skills/development/testing/cli/scaffold-tests-from-ac/parse-ac.ts` (same canonical AC grammar — see `business-analyse/_workflow/doc-templates.md`). - Globs `Tests/**/*.cs` (not only `Tests/**/Acceptance/*.cs`) to also flag tests written manually under other folders that carry `[Trait("AC",...)]`. - This audit is **read-only** — no `Edit`, no `Write`, no test re-generation. Findings route back to `/ba-develop` Phase 4 (or `scaffold-tests-from-ac` directly) for remediation.