# Recipe 25 — QA and content readiness

**Profile**: any, in single-book or corpus mode.

**TL;DR**: `book-scaffold qa` turns the scaffold's content contract and stable
MDX facts into one CI-safe readiness verdict. Human output is the default;
`--format json` emits deterministic schema-v1 JSON. Use
`book-scaffold init-qa` only when a portfolio QA engine needs a network-free
`guide_qa.yaml` registry.

## Run the readiness check

Run the locally installed binary from the book root:

```bash
npm exec -- book-scaffold qa
```

This checks the implicit book in a single-book project. In a corpus, omission
of a selector checks every manifest book in manifest order. Select one exact
registered id or spell out the all-books default with:

```bash
npm exec -- book-scaffold qa --book evaluation
npm exec -- book-scaffold qa --all
```

`--book` is invalid in single-book mode. `--book` and `--all` are mutually
exclusive; an unknown book id is an invocation error rather than an empty
report.

## What QA means

QA reuses the same content-contract library as `book-scaffold validate`, then
adds deterministic readiness facts:

| Check | Blocking condition |
|---|---|
| `content_contract` | Any validation error |
| `chapters` | No non-draft chapter |
| `links` | Any broken internal target or fragment |
| `learning_objectives` | Less than 100% anchor coverage when objectives apply |
| `components` | Never blocking; counts scaffold MDX components |
| `demo_fixtures` | Invalid non-generated JSON or a failing referenced schema |

An unavailable metric is `not_applicable`, not a fabricated zero. Component
counts are inventory facts, not universal prose-quality scores. `qa` also does
not replace `astro build`: rendering, content-collection Zod checks, and KaTeX
remain build responsibilities.

The traffic-light states are `green`, `amber`, `red`, and `not_applicable`.
Amber advisories stay visible but do not fail CI. Exit status is:

| Exit | Meaning |
|---|---|
| `0` | No blocking failure (`green` or `amber`) |
| `1` | At least one selected book or corpus-shared check is `red` |
| `2` | Invalid invocation, unresolved configuration, or internal failure |

The link check inspects internal links authored in chapter Markdown/MDX. It
uses the resolved scaffold route toggles, Astro page conventions, and public
assets as its route oracle. Fragments that depend on a consumer-defined MDX
component or a known non-chapter route are reported as amber
`fragment_unverified` advisories instead of guessed successes or blocking
failures.

Schema-v1 fixture validation supports JSON Schema draft-07 (also the default
when `$schema` is absent), 2019-09, and 2020-12. Recursive `$ref` resources must
remain inside the project after symlink resolution; QA never fetches network
schemas. The JSON Schema `format` keyword is annotation-only in v1. Unsupported
or mixed dialects and out-of-project references make that fixture red.

## Human and JSON output

Human output is compact and terminal-oriented:

```bash
npm exec -- book-scaffold qa --format human
```

For automation, use the stable JSON form (`--json` is an alias):

```bash
npm --offline exec -- book-scaffold qa --format json > qa-result.json
```

JSON stdout contains only the schema-v1 document; progress and fatal
diagnostics use stderr. The document omits timestamps and durations, preserves
manifest/source order, and always contains `books`, `shared`, and `summary`.

`books` contains only the implicit `book` result or registered manifest ids.
`shared` is never a synthetic book. It is an always-present aggregate shaped
like a book result:

```json
{
  "verdict": "not_applicable",
  "checks": {},
  "diagnostics": []
}
```

That exact value is used in single-book mode. In corpus mode,
`shared.checks.demo_fixtures` holds the normal `{ state, metrics,
diagnosticIds }` check payload for JSON under `src/data/` outside a registered
book directory. A shared failure affects the top-level corpus verdict and
summary, never an individual book verdict. Its diagnostics use
`book: "corpus"` rather than borrowing a manifest id.

## Generate `guide_qa.yaml`

Portfolio-level QA engines can discover the scaffold check without a custom
consumer wrapper:

```bash
npm exec -- book-scaffold init-qa
```

A single-book project gets this deterministic file:

```yaml
# Generated by book-scaffold init-qa.
# Regenerate with: book-scaffold init-qa --force
version: 1
guides:
  - id: book
    check_cmd: npm --offline exec -- book-scaffold qa --format json
```

A corpus gets one entry per manifest book, in manifest order, with an exact
selector:

```yaml
# Generated by book-scaffold init-qa.
# Regenerate with: book-scaffold init-qa --force
version: 1
guides:
  - id: evaluation
    check_cmd: npm --offline exec -- book-scaffold qa --book evaluation --format json
  - id: llm-app-engineering
    check_cmd: npm --offline exec -- book-scaffold qa --book llm-app-engineering --format json
```

The generated `npm --offline exec --` commands can use only the installed
toolkit; they never fetch a package. `book-scaffold qa` does not read or
execute `guide_qa.yaml` itself.

An existing file is preserved and makes `init-qa` fail. Regenerate explicitly
with:

```bash
npm exec -- book-scaffold init-qa --force
```

`--force` replaces the whole `guide_qa.yaml` and changes no other file. A
portfolio engine may add presentation fields, but forced regeneration removes
those edits.

## CI wiring

After the dependency install, offline execution prevents a typo or missing
local binary from becoming an implicit registry download:

```yaml
- run: npm ci
- run: npm --offline exec -- book-scaffold qa --format json > qa-result.json
- run: npm run build
```

Keep the build step. QA supplies a content-health verdict; the production
build remains the render/deployment gate.

## Common gotchas

- **Treating N/A as zero** — unavailable profile/content metrics are explicitly
  `not_applicable`.
- **Expecting `components` to grade prose** — counts are informational and
  never impose cross-preset quotas.
- **Attributing shared corpus JSON to a book** — only files beneath
  `src/data/<book>/` belong to that book. Other non-generated JSON is checked
  once through the top-level `shared` result.
- **Counting generated indexes as fixtures** — scaffold outputs including
  `labels.json`, `references.json`, `tips.json`, and `exercises.json` are
  excluded.
- **Hand-editing generated commands** — use engine-owned presentation fields
  if needed; `init-qa --force` intentionally restores canonical commands.
- **Using `--book` for a one-book project** — the implicit id is represented as
  `book` in results and `guide_qa.yaml`, but it is not a selectable corpus id.

## Canonical files

- `scripts/qa-core.mjs` — deterministic check and aggregate engine
- `scripts/init-qa.mjs` — deterministic, overwrite-safe registry generator
- `scripts/validate-core.mjs` — shared validation library used by QA's
  content contract
- `recipes/09-validation.md` — individual diagnostics and prebuild wiring
- `../../docs/plans/active/qa-contract.md` — accepted schema and selector contract