recipes
0/5 steps
Tutorial V2 · 6 minutes

Reading the evidence

For reviewers and sceptics. A green run is a claim; this is how you check it.

no prerequisites reference: For Reviewers
V2

Video recording in progress

This page and the reviewer's guide

This is the tour: what each file is for, in the order you open them. is the reference — a real bundle annotated line by line, including a screenshot that passed while recording that its capture had degraded. Read this first, keep that one open during a review.

What the video covers

BeatOn screen
0:00One finished run directory, opened cold.
0:40recipe.json — the graph that executed. Each node's intent is the argument being made.
1:40trace.json — per-node verdicts, timings, outputs. Counts come from here.
2:50summary.json — the verdict, the totals, which libraries were in scope.
3:30Screenshots, plus full-run video on Extension and iOS.
4:20diagnostics.json — application warnings, quarantined from the verdict.
5:10The two rules that make the whole thing worth reading.

The two rules

Both are enforced in code, not by convention.

Placeholders are refused, and a missing adapter throws. The harness will not accept a static stand-in where a real observation belongs, and when the adapter that would perform an action is absent it raises instead of returning a plausible-looking object. A system that guessed would produce evidence indistinguishable from the real thing.

The practical consequence: a node that passed had a real adapter execute it. That is what makes "ok": true worth anything.

Capture is not symmetric

Extension and iOS record full-run video through capture-helper; Android replay video is not implemented yet. mm-harness doctor reports what the current checkout can capture on its capture: line.

Follow along

Use a bundle you produced in the walkthrough, or any run directory.

  1. Open a bundle

    last prints the path of the most recent run for this checkout.

    $ mm-harness last
$ ls first-recipe-artifacts

  2. Skim report.md

    The whole run on one page. If it disagrees with the pull request description, stop and ask.

    $ cat first-recipe-artifacts/report.md

  3. Check the verdict and the counts

    Compare total against what the author claims was proven.

    $ cat first-recipe-artifacts/summary.json
    { "status": "pass", "total": 6, "passed": 6, "failed": 0 }

  4. Read the intents in order

    The per-node intents are the argument. If they do not add up to the claim, the bundle does not support it — however green it is. 

    $ cat first-recipe-artifacts/trace.json

    Triaging a failure: find the first node with "ok": false. Everything after it is consequence.

  5. Open the images, then the diagnostics

    The trace says a PNG was registered and how it was captured. Only your eyes confirm it shows what the author says. 

    $ open first-recipe-artifacts/screenshots
$ cat first-recipe-artifacts/diagnostics.json

    diagnostics.json is bracketed to the run window, so "status": "clean" is a claim about this run rather than about the log file.

The annotated reference → Next: skills setup Back to the series