/** * Binding a targeted `test-run:` report to the change it claims to test * (T12965). * * What a bound atom guarantees, exactly: * 1. **Freshness.** The report's own run time (vitest `startTime`, Playwright * `stats.startTime`, else the file's mtime) is not older than any of: * - the committer time of HEAD and of every commit the branch adds over * origin's default branch — unless the commits made after the run only * recorded what was on disk when it ran: HEAD as of the run is in HEAD's * reflog (that of the linked worktree holding the report, when it is * not the bound checkout; T13579), and no path that differs between that commit and the working * tree was modified, deleted or moved after the run (edit, run the * tests, commit, then verify). Any other commit after the run is * refused — a rebase onto a newer base included, since it rewrites the * files it brings in — so bind the report before committing, or re-run. * One exception (T13784, `test-run:` only): on the default branch, when * HEAD is on origin's default branch and HEAD as of the run is its * ancestor, the commits after the run are main moving on, not the * change; the atom records HEAD as of the run (`runHeadSha`), and the * post-merge stand-in refuses it when those commits touched the * change's files or the test files it ran; * - the newest working-tree modification of any path the branch changed * against origin's default branch (committed, uncommitted or untracked; * the report itself excluded) — or, with no origin to diff against, any * uncommitted tracked edit — except a path that a checkout or merge * after the run rewrote with identical bytes (HEAD moved and the file * equals its blob at both HEADs; T13567, T13569). A bare touch still * stales: without a run-time fingerprint it looks like a revert; * - for each uncommitted deletion (a rename's old path included, since a * move keeps the file's own mtime), the modification time of the * nearest existing directory it was removed from. * Commit times, reflog times and mtimes are recorded facts, not proof of * content: this refuses stale reports, it does not attest that the report * is honest. * 2. **Relevance.** When the change touches workspace packages only, the * report covers a test file of every affected package that has one — * each directly changed package and each package that depends on one, * the set `tool:test-affected` runs. An affected package with no test * file (tracked, or added by the change) cannot be covered and is * recorded in `untestedPackages`, as `tool:test-affected` records a * dependent with no test project; when no affected package has a test * file, the report is refused. A test file covers its package when one * of its tests passed (a file a `-t` filter skipped entirely covers * nothing); one such file is enough: a targeted report speaks for the * packages it ran, not for every file in them — merged CI does that. A * workspace-wide change (a path outside every package, such as a root * config or the lockfile) is refused outright: only a full-suite run * speaks for it, and a report cannot show that it ran the whole suite * (test configs exclude tracked test files, so per-package file counts * prove nothing) — `tool:test` runs it. A path outside every package * in a top-level directory with test files (a `tooling/` beside the * workspace) is not workspace-wide: the report must pass every test file * outside the packages under that top-level directory (T13754). A * report with no passed test at all is refused before any of this. * A standalone project (no workspace declared, T13403) is one package, * its root: a change inside the root is that package's, never * workspace-wide, and since that package is the whole project the report * must also have passed every test file the change adds or edits. * Docs-only changes, and checkouts with no origin to diff against, are * not judged. The `test-run:` reports of one evidence write are judged * together: a vitest report and a Playwright report ({@link playwrightTestRun}) * cover a change's unit and e2e files between them (T14031). * 3. **Identity.** HEAD and the tool cache's tree hash (T12958) at verify time are * recorded, and `cleo complete` refuses the atom once that tree moved * (unless merged CI or a full run carries the gate). * * It does NOT prove the report was produced by this tree's code: a report is * a file the caller supplies. Merged CI (`ci:`) or `tool:test` are the * evidence that does. * * @task T12965 */ import type { EvidenceAtom } from '@cleocode/contracts'; import { type ScopedChangedPaths } from './affected-packages.js'; /** The parts of a vitest (or jest) JSON report the binding reads. */ export interface TestRunReport { /** Epoch milliseconds the run started. */ startTime?: number; /** * Per-file results: `name` is the test file path, `assertionResults` its * tests (`passed`, `failed`, `skipped`/`pending`, `todo`). A file's own * `status` says nothing about whether any test ran: vitest reports a file * whose every test a `-t` filter skipped as `passed`. */ testResults?: Array<{ status?: string; name?: string; assertionResults?: Array<{ status?: string; }>; }>; } /** * Most test files a `test-run:` atom lists; `testFileCount` keeps the true * total when a full-suite report covers more. */ export declare const TEST_RUN_MAX_RECORDED_FILES = 200; /** * The report's test files that ran at least one passing test, relative to * `root` where they lie inside it, slash-separated, sorted and unique. A file * whose every test was skipped, filtered out (`-t`) or todo covers nothing, * and neither does a file that lists no `assertionResults` (T12965 review). * * @param report - Parsed report. * @param root - Execution root. * @returns The covered test files. * @task T12965 */ export declare function coveredTestFiles(report: TestRunReport, root: string): string[]; /** * A Playwright JSON report in the vitest shape the binding and the counter * checks read (T14031), from {@link playwrightTestRun}. */ export type PlaywrightTestRun = { readonly startTime?: number; readonly testResults: Array<{ status: string; name: string; assertionResults: Array<{ status: string; }>; }>; readonly numTotalTests: number; readonly numPassedTests: number; readonly numFailedTests: number; readonly numPendingTests: number; readonly numTodoTests: number; }; /** A Playwright report that cannot stand as test-run evidence, and why. */ export interface PlaywrightTestRunRefusal { readonly refusal: string; readonly codeName: 'E_EVIDENCE_INVALID' | 'E_EVIDENCE_TESTS_FAILED'; } /** * Adapt a Playwright JSON report (`--reporter=json`) to the vitest shape, so * an e2e spec can be test-run evidence (T14031); null when `report` is not * one (no `suites` array and numeric `stats.expected`). * * The files run are the suites' `file`s (each spec's own `file` when set), * resolved against `config.rootDir`, which Playwright makes them relative to. * Counts come from the spec results, one per test and project: `expected` * and `flaky` (passed on retry) pass, `unexpected` fails, `skipped` is * pending. They must equal `stats`, and the run time is `stats.startTime`. A * report listing top-level `errors` (no tests found, a config or global setup * failure) is refused; the rest — vacuous, skipped-only or failing runs — is * refused by the same checks as a vitest report. * * @param report - A parsed test-run report. * @returns The adapted report, a refusal, or null when it is not Playwright's. * @task T14031 */ export declare function playwrightTestRun(report: Readonly>): PlaywrightTestRun | PlaywrightTestRunRefusal | null; /** * The changed paths that are test files still on disk (vitest and jest * naming), sorted: the tests a change adds or edits (T13403). * * @param root - Execution root. * @param changed - Root-relative changed paths. * @returns The changed test files that exist. * @task T13403 */ export declare function changedTestFiles(root: string, changed: readonly string[]): string[]; /** What binding a report decided. */ export type TestRunBinding = { /** The report is refused. */ ok: false; /** Older than the change (stale), or not covering it (insufficient). */ codeName: 'E_EVIDENCE_STALE' | 'E_EVIDENCE_INSUFFICIENT'; /** Why, and what to do instead. */ reason: string; } | { /** The report may be bound. */ ok: true; /** Affected packages with no test file, which no report could cover (sorted). */ untestedPackages: string[]; /** HEAD as of the run, when main moved on after it ({@link ReportFreshness}, T13784). */ runHeadSha?: string; /** Changed CLEO config left to `tool:config-schema` (T13778). */ configPaths?: string[]; /** Dependents of a declared generated package left to a typecheck (T13783). */ typecheckDirs?: string[]; /** Untracked files outside the change, set aside at bind time (T14000). */ untrackedSetAside?: string[]; }; /** What the freshness check of a report or receipt decided. */ export type ReportFreshness = { /** Older than the change. */ ok: false; codeName: 'E_EVIDENCE_STALE'; reason: string; } | { ok: true; /** False when the report has no run time at all, so nothing is judged. */ judged: boolean; /** The change's scoped paths, or null outside git / with no origin. */ changes: ScopedChangedPaths | null; /** * HEAD as of the run, when the only commits after it landed on origin's * default branch (T13784, `test-run` only): the run did not see them, so * the atom records it and the post-merge stand-in refuses the report when * they touched the change's files or the tests it ran. */ runHeadSha?: string; }; /** * Whether a report (or receipt) ran after every part of the change it claims * to speak for — point 1 of the module doc, shared by `test-run:` and * `qa-run:` (T13427). * * @param startTime - The run's own start time (epoch ms), if it states one; * else the file's mtime is the run time. * @param reportPath - Absolute path of the report file. * @param root - The task's checkout. * @param atom - The atom kind, named in the remedy. * @returns The stale refusal, or the change's scoped paths for the relevance check. * @task T12965 * @task T13427 */ export declare function reportFreshness(startTime: unknown, reportPath: string, root: string, atom: 'test-run' | 'qa-run'): ReportFreshness; /** * Bind a report to the change it claims to test, or refuse it when it is * older than the change or does not cover it (see the module doc for exactly * what this guarantees). * * @param report - Parsed report. * @param reportPath - Absolute path of the report file. * @param root - The task's checkout (a git checkout, or not). * @param testFiles - {@link coveredTestFiles} of the report. * @returns The refusal, or the binding with the affected packages left untested. * @task T12965 */ export declare function bindTestRunReport(report: TestRunReport, reportPath: string, root: string, testFiles: readonly string[]): TestRunBinding; /** * Whether the test files a report ran cover a change — point 2 of the module * doc, applied to an explicit list of changed paths. {@link bindTestRunReport} * passes the branch's diff; `cleo complete` passes a merged change's own files * when it judges a run recorded after the merge (T13623). * * CLEO-owned config files are left to `tool:config-schema` (T13778): the * report is judged over the rest, and a binding names them as `configPaths`, * which the gate then requires a config-schema result for. * * @param root - The checkout the test files are relative to. * @param changed - The change's paths, relative to `root` (non-empty). * @param testFiles - Test files the report ran with a passing test. * @returns The refusal, or the binding with the affected packages left untested. * @task T12965 * @task T13623 * @task T13778 */ export declare function testRunCoversChange(root: string, changed: readonly string[], testFiles: readonly string[]): TestRunBinding; /** * What a targeted test-run report must run to speak for a change — point 2 of * the module doc as data. {@link testRunCoversChange} judges a report against * it, and `cleo tasks changeset --test-plan` reports it, so the two cannot * drift (T13795). * * - `configPaths`: CLEO config files, left to `tool:config-schema` (T13778). * - `full`: the rest is workspace-wide; no test-run report speaks for it. * - `targeted`: the report must pass every file of `outsideTests` and of * `everyChangedTest`, and at least one test file of every package in * `required`. `direct` lists the changed packages; `untestedPackages` the * affected ones with no test; `typecheckDirs` the typecheck-only dependents * left to a typecheck (T13783). `standalone` marks a single-package project. */ export type TestRunRequirement = { kind: 'full'; reason: string; configPaths: string[]; } | { kind: 'targeted'; configPaths: string[]; direct: string[]; required: string[]; untestedPackages: string[]; typecheckDirs: string[]; outsideTests: string[]; everyChangedTest: string[]; standalone: boolean; }; /** * The {@link TestRunRequirement} of a change. * * @param root - The checkout the change is relative to. * @param changed - The change's paths, relative to `root`. * @returns What a covering report must run. * @task T13795 */ export declare function testRunRequirement(root: string, changed: readonly string[]): TestRunRequirement; /** * Why the CLEO config and the rest of a change are not covered together in * one gate write (T13778), or null. A test, QA or affected run that left CLEO * config to `tool:config-schema` (`configPaths`) needs a config-schema result * that validated those files; a config-schema result on a change with other * paths (`otherPaths`) needs another verification result for them, so a * config edit never stands in for an untested source change. Merged CI * (`ci:`) covers the whole change on its own. * * @param atoms - The validated atoms of one gate write. * @returns The refusal naming the composed route, or null. * @task T13778 */ export declare function configSchemaCompositionGap(atoms: readonly EvidenceAtom[]): string | null; //# sourceMappingURL=test-run-binding.d.ts.map