# Handoff, 2026-08-03

## 0. DECISION-GRADE RC SUMMARY (read this first)

Pinned: **`worktree-pre-push-scoped-pytest` @ `16ba5c5e`**.

| Fact | Value | Command |
|---|---|---|
| RC HEAD | `16ba5c5e` at the time of writing; this document's own commit
advances it, so ALWAYS trust the command over this cell | `git rev-parse HEAD` |
| vs `origin/main` (`dd692561`) | **27 ahead, 0 behind** | `git rev-list --count origin/main..HEAD` |
| vs RC remote (`3808f63f`) | **17 ahead, 0 behind** | `git rev-list --count origin/worktree-...` |

WHY THE SHA CELL IS ALWAYS ONE BEHIND. Committing this file moves HEAD, so
a handoff can never pin its own commit. The cell records the SHA the
receipts below were taken on; the live value is one commit later. Re-run
`git rev-parse --short HEAD` and `git rev-list --count origin/main..HEAD`
rather than trusting any number typed here -- that habit is what caught
three stale pins in this document already.

LOCAL EVIDENCE AND CI EVIDENCE ARE DIFFERENT THINGS, and this document keeps
them apart deliberately. Everything under "Local receipts" ran on this machine
against this exact SHA. NO CI has ever run on `16ba5c5e` or on any commit in this
branch: a branch push triggers no workflow (see gate 2), so there is no CI
verdict to report for the RC. The CI table below is for `origin/main` only.

**A BRANCH PUSH ALONE RUNS NO CI.** Every workflow triggers only on
push-to-main or a pull_request targeting main, and `test.yml` has no
`workflow_dispatch`. Pushing this branch produces zero runs, as observed.
A CI verdict on this SHA requires a branch push AND a PR to main.

### CI evidence, exact

`origin/main` = `dd692561`:

| Workflow | Conclusion | URL |
|---|---|---|
| Post-Release Soak Monitor | success | actions/runs/30805237104 |
| Security Audit | success | actions/runs/30796434331 |
| **Tests** | **failure** | actions/runs/30794017151 |

The Tests failure is ONE job, ONE cause: `Shell tests (shard 3/4)` ->
`ShellCheck Linting FAILED`. Shards 0, 1, 2 pass. Held commit `a4d8b839`
takes shellcheck from 7 warning sites to **361 passed / 0 failed**.

**RC branch CI: 0 runs, and it will stay 0.** Every workflow triggers only on
push-to-main or a PR targeting main, and `test.yml` has no `workflow_dispatch`.
A branch push cannot produce a CI verdict. PR #181 is MERGED 2026-07-28 on a
different diff and is NOT evidence about current main.

### Local receipts on the exact RC HEAD `16ba5c5e`

Measured on this machine. NOT CI: no workflow has run on this SHA.

| Check | Result |
|---|---|
| `bash scripts/local-ci.sh` | **exit 0**, Failed 0 |
| `bash tests/run-shellcheck.sh` | **361 passed, 0 failed** |
| `bash tests/test-plan-command.sh` | **27 passed, 0 failed** (was 25/2; two stale June cost pins fixed) |
| `python3 -m pytest tests/ -q -k "bench or schema"` | **149 passed**, 1 skipped |
| `python3 -m pytest tests/` (full) | **2970 passed, 1 failed**, 19 skipped |

THE ONE FULL-SUITE FAILURE, characterised rather than hidden:
`tests/dashboard/test_build_supervisor.py::test_confined_claude_auth_uses_exact_login_keychain_capability`.

- PRE-EXISTING, not caused by anything here: a pristine `git archive HEAD`
  extract with no working-tree residue reproduces it.
- ENVIRONMENT-SENSITIVE, not a code defect: the same file passes **37/37 in
  isolation in this worktree, twice in a row**, and the failing assertion is
  about a real login-keychain capability, which differs between this worktree
  and a bare extract.
- It is NOT in the fast tier, which is why `local-ci.sh` is green while the
  full suite is not. Both numbers are reported rather than the flattering one.

### Working-tree residue, classified. NOTHING deleted, reverted or committed.

| Path | Class | Note |
|---|---|---|
| `coverage/clover.xml` | generated residue | rewritten by test runs |
| `coverage/lcov-report/index.html` | generated residue | rewritten by test runs |
| `loki-ts/dist/loki.js.map` | generated residue | rewritten by `bun run build` |
| `f.txt` (deleted) | **user-owned** | deleted in working tree only; **still present in HEAD**, fully recoverable via `git checkout -- f.txt` |
| `benchmarks/results/prompt-ablation.jsonl` | measurement output | untracked; the 6 real ablation trials |


### Every held commit (27, unpushed to main)

| SHA | Subject |
|---|---|
| `16ba5c5e` | feat(bench): `loki bench oracles` -- reachable where the spend decision is made |
| `5cc5ffd2` | feat(bench): attest the artifact hashes so an edited answer key cannot pass |
| `6dbef513` | docs: both remaining tiers now verified against a near-miss, gate 3 precondition MET |
| `89babec7` | docs: gate 3's precondition is PARTIALLY met, with the hashes that make it checkable |
| `43e4d530` | test(bench): pin the private probe's own guarantees, and refuse to promote a tier on private evidence |
| `f9895095` | feat(bench): hash-bound private oracle probes -- the replayable version of what I retracted |
| `ed9f7a6c` | fix(bench,docs): retract the medium/high VERIFIED label -- the evidence cannot be replayed |
| `a54c190c` | feat(bench): medium and high oracles verified against a plausible wrong answer |
| `1fd65420` | docs: make HEAD decision-ready -- exact-SHA receipts, local evidence kept apart from CI |
| `5e0ed5b5` | fix(tests): two demo assertions pinned a June cost the estimator has since corrected |
| `49ebefa2` | docs: reconcile the handoff to live topology and separate the three founder gates |
| `7b25a772` | docs(bench): record that adversarially_unverified is unreachable today, and why it stays |
| `ea930bbb` | fix(bench,ci): fail on model mismatch, wire the integrity check, add adversarial oracle probes |
| `8980313e` | docs: decision-grade RC handoff reconciled to current topology |
| `2c031a14` | feat(safety): default-deny release interlock, and a detector for the recurring core.bare fault |
| `6ffa5080` | feat(bench): founder-gated spend interlock, and keep grader evidence beside the measured flag |
| `35add6d7` | fix(bench): flag acceptance commands that cannot fail -- a timeout scored as success |
| `3808f63f` | feat(bench): cross-tier oracle receipt -- verifies the graders, runs no agent |
| `1591f880` | docs: re-verify ref topology after an audit, and name the single CI blocker |
| `ef0a6d49` | docs: put the spend decision's actual numbers in the handoff |
| `12669cd3` | docs: the no-spend outcome frontier -- 3 held-out tasks, deterministic oracles, nothing run |
| `fdba7023` | docs: handoff -- reconciliation, receipts, risks, founder-gated next action |
| `19c2b5ad` | fix(tests): bound each completion probe, so a hang is a NAMED failure not rc143 |
| `90468210` | feat(advisor): point at the calibration signal, and pin that it can never rank |
| `0f4da110` | docs: freeze the T1/T2 benchmark designs, unrun and unpaid |
| `8c33d123` | feat(tools): offline calibration audit, and it discloses that it cannot measure accuracy |
| `a4d8b839` | fix(lint): the last red CI job was three pre-existing shellcheck warnings |

### THE THREE FOUNDER GATES, deliberately separate

Ordered by dependency, not by value. Gate 1 first because gates 2 and 3 both
move code toward main, and today nothing non-bypassable stands between main
and a publish.

**GATE 1 -- a durable, non-bypassable publish control.** Zero spend, zero
code. Either an `environment:` key on release.yml's publish jobs (GitHub then
requires a named reviewer before those jobs start), or a ruleset on main.
Needs repository settings access, which is why it is not done here.

Everything I built is operator-side and BYPASSABLE by not running it:
`scripts/release-approval-gate.sh` refuses by default when run, and is a
no-op when skipped. Treat a green run as "the operator checked", never as
"publishing is gated". Measured today: release.yml triggers on any
VERSION-path push to main and runs `gh release create`, `npm publish` twice
and a Docker push, with no `environment:` key, and
`gh api .../branches/main/protection` returns 404.

**GATE 2 -- branch push PLUS a PR to main, for CI on this exact SHA.** Zero
spend. These are ONE gate because either alone yields nothing: a branch push
runs no workflow, and there is no PR without the push. Only this produces a
CI verdict on `43e4d530`. It is also the only path by which held `a4d8b839`
reaches main and clears the single remaining Tests failure.

**GATE 3 -- benchmark spend, roughly $25-$180 and 4-12 hours.** Independent
of gates 1 and 2. Buys the first OUTCOME evidence rather than affordance
evidence (`docs/OUTCOME-FRONTIER.md`).

PRECONDITION ON GATE 3: **MET, with one qualifier that decides who can
check it.** Small is verified by a receipt ANY reader can reproduce.
Medium and high are verified by a PRIVATE hash-bound receipt (below):
replayable by whoever holds the artifacts, not by a reader on a fresh
clone.

| Tier | Positive | Adversarial | Receipt status |
|---|---|---|---|
| small `simple-1-contact-form` | exit 0 | exit 1 | **VERIFIED, reproducible** -- fixtures are committed, `tier_receipt.py` re-runs both probes on demand |
| medium `multifail-1-two-modules` | -- | -- | **NOT ATTEMPTED** in the shared receipt; VERIFIED privately, see below |
| high `hard-1-order-api` | -- | -- | **NOT ATTEMPTED** in the shared receipt; VERIFIED privately, see below |

NON-REPRODUCIBLE OBSERVATION, recorded as a NOTE and explicitly NOT a
receipt. On 2026-08-03 the medium and high graders were exercised by hand
against artifacts built in a temp directory and destroyed. Observed exits
were 0 for a correct implementation and 1 for a near-miss (high: omitting
the subtotal>=100 discount; medium: fixing only cluster A). The high-tier
near-miss passed all ten validation cases and both non-discount totals.

That observation must not be promoted to VERIFIED, and the earlier draft of
this section wrongly did. The artifacts are gone, no hash binds them to the
run, and nothing in this repo can replay it -- so it fails the same standard
this codebase applies to every other claim: an unreproducible observation is
not evidence. It is recorded because it is a useful prior for whoever builds
the real probes, and for no stronger purpose.

THE MECHANISM THAT MEETS THIS PRECONDITION NOW EXISTS, and one of the two
tiers is verified against it. `benchmarks/bench/private_probe.py` runs the
in-repo graders against artifacts held OUTSIDE this repository (default
`~/loki-bench-private`), content-hashes each set over its sorted
(filename, bytes) pairs, and records the hash beside the exit code. The
artifacts stay out of the repo, so the held-out design is uncontaminated;
the hash makes the claim replayable, which the earlier by-hand run was not.

Measured 2026-08-03:

| Task | Probe | Exit | Expected | Artifact sha256 (first 16) |
|---|---|---|---|---|
| `hard-1-order-api` | positive | 0 | 0 | `884d555b6e53833d` |
| `hard-1-order-api` | adversarial | 1 | 1 | `486a62137c64167a` |
| `multifail-1-two-modules` | positive | 0 | 0 | `a7134c9c1ae36ae9` |
| `multifail-1-two-modules` | adversarial | 1 | 1 | `f4daff9d346c6116` |

Both tiers read `verified`. Each adversarial artifact is the near-miss the
grader's own docstring names: the high one omits the subtotal>=100 discount
while passing all ten validation cases and both non-discount totals; the
medium one fixes cluster A and leaves cluster B absent, and the grader
rejects it naming `cluster B_roman`.

So every tier now rejects a PLAUSIBLE WRONG answer, not merely an absent
one, which is the property that makes a paid run's number mean something.

HOW TO CHECK IT YOURSELF, and what stops it rotting.

    loki bench oracles          # or: python3 benchmarks/bench/private_probe.py

Reachable from the CLI deliberately, and placed beside `bench run`: the
question in front of a paid run is whether a green cell would mean
anything, so the answer sits one command from the spend decision. Costs
nothing -- local graders against artifacts already on disk, never a
provider.

`benchmarks/bench/private_attestation.json` commits the HASHES (never the
artifacts) and the probe compares against them. Without that the probe
would verify whatever bytes it found, so an artifact edited afterwards
would be re-blessed under a new hash while the receipt still read
`verified` -- which is how an edited answer key launders itself green.

Proven on the case that matters: appending a COSMETIC comment leaves
behaviour identical, the grader still returns its expected exit code, and
the run still fails with `attestation=DRIFTED` naming both hashes. Only
the hash comparison can catch that.

An unattested probe reads `unattested`, not drift. Absence of a record is
not evidence of tampering.

TWO QUALIFIERS THAT MUST TRAVEL WITH THAT TABLE.

First, this is a PRIVATE receipt. It is checkable by whoever holds the
artifacts, and not by a reader on a fresh clone. `tier_receipt.py`
therefore still reports medium and high as `not_attempted`, deliberately:
promoting them there would make a shared receipt's verdict depend on who
ran it, which re-creates the retracted defect one level up. The two
receipts answer different questions and a human combines them.

Second, the hashes above identify one operator's artifacts on one machine.
They prove the claim is REPLAYABLE, not that anyone else can replay it
today. Making it shareable (a second repository, an attested bundle) is a
distribution decision, not a code change.

### RELEASE EXPOSURE, measured today

`.github/workflows/release.yml` triggers on `push: paths: ['VERSION'],
branches: [main]` and runs `gh release create`, `npm publish --access public`
(twice) and a Docker push. It has **no `environment:` approval gate**, and
`gh api .../branches/main/protection` returns **404 Branch not protected**.

So a single VERSION-editing commit reaching main publishes with no human in
the loop. `scripts/release-approval-gate.sh` is a default-DENY local interlock
against exactly that, verified on a throwaway 9.11.0 -> 9.99.99 bump. The
durable fix (an `environment:` gate or a ruleset) changes the publishing path
or repo settings and is **founder-gated** -- it is listed as an approval still
needed, not silently applied.


State at handoff. Everything below is verified by a command whose output was
read, or is marked UNKNOWN. Nothing has been pushed, published, merged, or
released, and nothing was spent.

## 1. Reconciliation

SUPERSEDED SNAPSHOT. The HEAD and commit count below are from an earlier
point in the session and are kept as history, NOT as current state. Section 0
is the live topology: `7b25a772`, 16 ahead of origin/main, 6 ahead of the RC
remote. The VERSION / npm / tag rows are still accurate.

| Fact | Value (as of `19c2b5ad`) | How verified |
|---|---|---|
| Local HEAD | `19c2b5ad` (SUPERSEDED, now `7b25a772`) | `git rev-parse HEAD` |
| `origin/main` | `dd692561` (unchanged) | `git rev-parse origin/main` after fetch |
| Commits ahead | 5 (SUPERSEDED, now 16) | `git rev-list --count origin/main..HEAD` |
| repo VERSION | 9.11.0 | `cat VERSION` |
| npm latest | 9.8.1 | `npm view loki-mode version` |
| newest git tag | v9.8.1 | `git tag --sort=-v:refname \| head -1` |

**npm and the tag agree at 9.8.1.** The repo being at 9.11.0 is not drift: it
is an unreleased repo under a no-publish directive. Three minor versions of
work are committed and unpublished by instruction. Reconciled, not a defect.

## 1b. Ref topology, re-verified after an independent audit (09:04Z)

An audit reported the session ref `loki/session-1785167314-9337` at
`09138e26` as 0 ahead / 176 behind, fully contained in `origin/main`, and
concluded the "commits held/unpushed" claim was stale. **Both statements are
correct, about DIFFERENT refs.** Re-measured here:

```
git rev-list --left-right --count origin/main...loki/session-1785167314-9337
  -> 176   0     (behind 176, ahead 0)
git merge-base --is-ancestor loki/session-1785167314-9337 origin/main
  -> true         origin/main CONTAINS it; nothing to push from that ref

git rev-list --left-right --count origin/main...HEAD        # worktree-pre-push-scoped-pytest
  -> 0     8     (behind 0, ahead 8)
git merge-base --is-ancestor HEAD origin/main
  -> false        NOT contained; the 8 are unique
```

Each of the 8 was additionally tested individually with
`git merge-base --is-ancestor <sha> origin/main`; none is in `origin/main`.

The audited session ref is not one I have worked on. The held work is on
`worktree-pre-push-scoped-pytest`. The push was NOT cancelled as stale,
because it does not target the audited ref -- but it remains UNPUSHED by
directive, pending explicit approval.

## 1c. origin/main is ONE JOB from CI-green

Receipts for `origin/main` = `dd692561`:

| Workflow | Conclusion | URL |
|---|---|---|
| Security Audit | success | actions/runs/30796434331 |
| SBOM (CycloneDX) | success | actions/runs/30794017044 |
| Bun Parity | success | actions/runs/30794017132 |
| Coverage (baseline) | success | actions/runs/30794017182 |
| pages build | success | actions/runs/30794016322 |
| **Tests** | **failure** | actions/runs/30794017151 |

The Tests failure is ONE job with ONE cause: `Shell tests (shard 3/4)` ->
`ShellCheck Linting FAILED`. Shards 0, 1 and 2 pass.

Measured on both sides of the fix:

- `origin/main`'s copies of the three files carry **7 shellcheck warning
  sites** (test-iteration-grace 5, test-go-cargo-gate-timeout 1,
  cleanup-test-processes 1).
- The held commit `a4d8b839` takes those same three files to **0**, and
  `tests/run-shellcheck.sh` reports **360 passed / 0 failed**.

So the single blocker to a CI-green release candidate is already fixed and
held. No new work is required to clear it; only the push gate.

## 2. Held commits (8, unpushed)

| SHA | What it is |
|---|---|
| `a4d8b839` | Last red CI job: three PRE-EXISTING shellcheck warnings (not mine; last touched Jul 31 / Aug 1). One was a real hazard -- an unguarded `cd` that could make a gate examine the wrong tree and report a pass for a repo it never looked at. |
| `8c33d123` | Offline calibration audit. Discloses, before any number, that it measures agreement with the council majority and NOT accuracy. |
| `0f4da110` | T1/T2 benchmark designs frozen with held-out discipline. Unrun, unpaid. |
| `90468210` | Calibration wired into model-advisor as a ONE-WAY caveat. Pinned by test so it can never become a ranking input. |
| `19c2b5ad` | Per-command timeout on the completion-coverage probe. Root cause of the weekly audit's rc143, with the culprit named. |
| `fdba7023` | This handoff. |
| `12669cd3` | The no-spend outcome frontier: 3 held-out tasks, deterministic non-self-grading oracles, nothing run. |
| `ef0a6d49` | Spend-decision figures moved into the handoff so the decision needs one file, not two. |

## 3. Receipts

- **shellcheck: 360 passed, 0 failed.** Was 357/3.
- **Weekly-integrity rc143: root-caused and fixed.**
  `test-completion-coverage.sh` probes ~274 command candidates as
  `loki <c> --help`. `help` is among them (verified by re-running the
  candidate extraction standalone), so it executed `loki help --help` -- the
  unbounded self-delegation fixed in `ccf8dcbb`, which spawned a process per
  level until the fork table was exhausted and the runner was SIGTERM'd.
  Fixed with a PER-COMMAND 15s bound, never a blanket skip: skipping `help`
  would have hidden the bug that needed fixing.
  Proven non-vacuous by injecting a 60s hang into `doctor`:
  `rc=1`, 34s, `PROBE HUNG: 'loki doctor --help' did not return within 15s`.
  Without the hang: 6 passed, 0 failed, 20s.
- **CI on `dd692561` (last pushed SHA), terminal:** shards 0, 1, 2 GREEN --
  the first shell shards to pass on a runner this session. Shard 3 red on
  ShellCheck only, which `a4d8b839` fixes locally.
- **Local gate:** `scripts/local-ci.sh` exit 0, Failed 0, on every commit above.
- **Fork bomb:** contained 4,172 -> 375 processes. Root cause and counterfactual
  in `ccf8dcbb` (already pushed).

## 4. The calibration audit has NEVER run on real data

Searched this machine for council transcripts:

```
find ~/git ~/loki-bench-v7 -type d -name transcripts -path "*council*"   -> empty
find ~ -maxdepth 4 -type d -name council                                 -> empty
```

**No real council transcripts exist here.** Every calibration number produced
so far is fixture-derived, and the fixtures were built to have a hand-checkable
answer rather than to resemble production. The audit is correct against
hand-derived arithmetic; it is unvalidated against reality.

This is stated because "the tool works" and "the tool has told us something
about our system" are different claims, and only the first is supported.

## 5. Risks

- **The calibration signal is not an accuracy signal, and it never becomes
  one.** `tools/calibration-audit.py` scores agreement with the council
  MAJORITY. The council outcome is derived from the votes, so a voter partly
  causes its own label. A low Brier score there means "voted with the pack",
  not "was correct". No artifact in this repo records whether the council was
  right. This must stay a caveat; `90468210` pins by test that it cannot enter
  model-advisor's machine-readable contract.
- **Local green is not CI green.** `a4d8b839` and `19c2b5ad` are verified only
  on this machine. They are unpushed by instruction, so no runner has executed
  them. Do not read the local 360/0 as a CI verdict.
- **Shard 3 has never been fully measured locally.** Its run was stopped during
  the fork-bomb containment. Shards 0/1/2 were measured, partly during a window
  when the process table was exhausted -- and in that state `grep` and `echo`
  return EMPTY WITH NO ERROR, indistinguishable from "no matches". Readings
  taken then are absent measurements, not results.
- **Three worktree agents' output was landed after review, and review found
  real defects** (an unregistered guard that would never have run; a 48-vs-36
  test-count misreport). Self-reports are not receipts.
- **Version skew is deliberate but load-bearing.** Anyone installing from npm
  gets 9.8.1 and none of this work. If a user reports a bug fixed in 9.9-9.11,
  that is why.

## 6. Founder-gated next action

Exactly one decision unblocks the rest. All five commits are gate-green
locally and held only by the standing directive.

**Option A -- release the gate.** Push the 5 commits. CI then runs them for the
first time, which is the only way to confirm shard 3 goes green and the
rc143 fix holds on a runner. No publish is implied by a push.

**Option B -- keep the gate, name the next target.** Work continues and
continues to be held. The next unblocked item is the outcome frontier design
(in flight, no-spend).

**Option C -- authorize spend.** Only this unlocks measuring OUTCOME quality
against competitors. Every axis in the scorecard that reads UNKNOWN reads that
way because no paid run has happened.

The design is ready and unrun: `docs/OUTCOME-FRONTIER.md`, three held-out
tasks with deterministic non-self-grading oracles, a frozen three-dimension
rubric, and seven dimensions deliberately left UNSCORED rather than
approximated with an LLM judge.

**Estimated cost: roughly $25 to $180 in provider spend and 4 to 12 hours of
wall clock**, for a 4-tool, 2-task matrix. That is an ESTIMATE with its
assumptions listed in the document, not a measurement, and the two runnable
tasks (brownfield, multi-file migration) are the ones priced. The scientific
task is PLAN ONLY -- it needs a paper chosen and its number transcribed before
it can run at all, and its results must never be reported as equally rigorous
to the other two.

What approving this buys: the first evidence about OUTCOME rather than CLI
affordances. What it does not buy: the seven unscored dimensions, which no
amount of spend makes deterministic.

Nothing in this handoff requires spend, and nothing in it has been published.