# Completion and steering validation — 22 October 2026

This report distinguishes CPU regression tests, real-model tests with real file effects, and real-model conversations over synthetic evidence. A blocked unsafe claim is useful negative evidence; it does not by itself prove that a legitimate task can finish.

## Hardware and scope

The live endpoint is the user's configured Ollama broker at `http://localhost:11434`, using exactly `ornith-1.5:122b` (digest `f1e64af05b7dc05153dc7f3fffc34e61fa81f672ae248b9134cc7751ac21c890`). Each model load was admitted through the broker for the ordered A100 pair, and actual runner placement plus `CUDA_VISIBLE_DEVICES` was checked before generating tokens. There is no CPU offload, alternate model, or remote fallback. The unrelated transcription lease on the third A100 was preserved.

The original `test_three` application was not resumed or modified. Actual file-effect tests use a private, neutral JSON layout/font fixture. The dialogue demonstration uses explicitly synthetic observations and receipts, and cannot establish that an external message was really sent or a test command really passed.

## Failures found by live testing

| Run (UTC) | Result | Root cause or test limitation |
| --- | --- | --- |
| 02:27 | Failed, 2 physical model requests | Router invented input references and returned an invalid continuation scope. |
| 02:30 | Failed, 3 requests | Admission produced an invalid source-review effect combination. |
| 02:34 | Failed, 7 requests | Completion omitted its user-visible reply; the actual tool schema made that field optional. |
| 02:39 | First comparison completed; full sequence failed, 9 requests | A later router decision omitted the required user-result effect. |
| 02:44 | First comparison completed; run explicitly stopped | The harness omitted the production steering reply lane. It also exposed a real admission bug: ordinary inline reporting was classified as external delivery, and source comparisons lacked verification. The incomplete run has no final report and is not counted as a pass. |
| 02:56 | Failed safely before tools, 5 requests | Both proposal attempts omitted comparison verification. The independent critic identified comparisons but emitted an inconsistent global acceptance. Host consistency checks rejected it. Distinct HTTP requests carried the correction, but the model returned identical proposals. Grammar-branch bias was suspected, not established as a decoder defect. |
| 03:05 | Failed safely before critic admission, 3 requests | A smaller decoder schema plus more warnings still produced inconsistent kind/method/effect fields. The faster failure was not a successful performance improvement. |
| 03:15 | First comparison completed; full sequence failed, 11 requests | The simplified semantic-purpose interface worked for the first task. The next proposal exposed an expressivity defect: domain requirements could not declare an inline response even though the critic correctly requested that classification. |
| 03:20 | First comparison completed; full sequence failed, 15 requests | Wrong-file inspection, steering reply and renewed source admission succeeded, and the worker honestly declared the read-only phase incomplete. The harness then required old reads to remain admissible after steering had advanced the contract revision. They correctly did not. A fresh actual registered read is required before the negative critic test. |
| 03:26 | First comparison and fresh-read continuation passed; full sequence failed, 22 requests | The actual negative critic identified wrong font, missing footer and absent mutation. One paraphrased candidate quotation made the response structurally invalid. The direct harness call had bypassed the production runner's existing single exact-quote correction. The harness must exercise that production path, not waive quote validation. |
| 03:37 | First comparison passed; steering admission held incomplete, 17 requests | An execution row omitted its newly required justification. On retry, the model selected source comparison correctly but attached an execution justification to an inline-response row. Host validation rejected both. The producer decoder schema must require the field on execution rows and forbid it on other rows, rather than represent either invalid state. |
| 03:44 | First comparison passed; update admission held, 13 requests | The critic duplicated one requirement ID and omitted the communication carrier despite a global acceptance. The retry repeated an identical source citation. Fixed keyed review fields remove model copying of the host inventory; exact valid citation duplicates have set semantics. Unknown sources, extra properties and unsupported requirements remain rejected. |
| 03:51 | Admission, steering and fresh reads passed; full sequence failed, 24 requests | The production negative critic identified the wrong font/footer but cited tool-output text instead of the candidate. Its single correction still supplied nonliteral quotes and an empty citation for a satisfied check. Inventory and current evidence IDs were correct. A request-local host span ID replaces copied candidate prose/hashes; the active schema requires a subject citation for every check without making that citation evidence of success. |
| 04:01 | Raw harness reported pass, 42 requests; post-audit rejects a full semantic pass | The producer and critic retained the explicit verify-result sentence but jointly labeled it observation, removing typed source-review verification. Actual supported comparison, negative span-based rejection and later real-file repair succeeded; they do not erase the earlier weakening. Repeated reads had distinct requests, growing histories and compact loop feedback. A contradictory discovery instruction and cache hits from the retired contract revision prevented progress. The raw report is preserved unchanged with a separate audit. |
| 04:13 | Partial validation; failed before repair, 24 requests | The stronger admission invariant passed every revision, supported comparison completed, and the real false claim was validly rejected. During runner unload/reload, the final router request received HTTP 503 `queue_admission_timeout` with `admission_retained: true`. The helper swallowed transport pending as no decision. This is a broker-admission custody failure, not a semantic rejection; no final repair happened in this run. |
| 04:29 | Complete intermediate sequence passed, 33 physical requests | Original verification obligations survived steering; current-owned reads, valid false-claim rejection, actual font/footer repair and fresh positive review all passed. This build predates the shared admission/reviewer pending-request repair and had no retained 503, so it is not final validation of that repair. Its accepted prose also miscounted the six artifact fields as seven; the requested values and actual effects passed, not flawless prose. |
| 04:40 | Explicitly stopped superseded build after seven physical requests | The full CPU suite found that propagated reviewer errors reached legacy completion normalization, and an unavailable quote correction could be issued again. The operator stopped this harness while its first result review was in flight. Its interrupted comparison is not counted as a model or semantic failure, and supplies no completed-sequence credit. |
| 04:51 | Partial validation, 33 physical calls; explicitly stopped | First comparison passed; two natural retained HTTP 503s resumed body-free under their original logical IDs. A genuine semantic mismatch required the second steering proposal; queue waits consumed no extra proposal. The read-only worker then repeated the unchanged file despite current evidence and stasis feedback. Late context reasserted mutation instructions while its invocation limit remained only in the initial message. Stop interrupted the subsequent read-only continuation; no valid negative-review or repair credit is claimed. |

| 05:05 | Read-only handoff, negative review and both real repairs passed; final verification held incomplete, 21 physical calls | The invocation-scope fix ended the read-only phase after three primary turns. Actual font/footer changes then succeeded. The plateau guard keyed read novelty only by tool arguments and counted only criterion-status changes as progress, so it refused the fresh read of the changed file. No positive completion was accepted. One natural retained router 503 resumed body-free under the same logical ID. |

| 05:17 | Failed at the 10-turn bound, 19 physical calls and one valid critic | Authentic cross-process continuation admitted fresh reads of the repaired files. The critic credited both prior mutations and current parity, but rejected the worker’s inaccurate no-change report. The operational work graph kept projecting old read-only summaries as adopted claims without originating run, time or artifact revision. The worker repeatedly reread correct artifacts to resolve that stale conflict instead of correcting its historical report. Further repair was explicitly deferred by the user. |

The repairs bind router references to exact issued IDs, validate model-authored requirement/effect populations, pass the full original request and accepted steering into completion review, and require the exact final reply before admitting visible completion. The v4 wire vocabulary also separates inspection without verification from comparing or verifying source content; both models must choose the same purpose. The fixture explicitly rejects any admission that drops the original verification obligation. Routing, task-admission proposals and critics, and required completion review now share one owned retained-request lifecycle. A typed retained broker wait preserves the exact logical request without spending semantic correction attempts. A completion-review transport failure releases its pending challenge reservation and propagates as infrastructure failure instead of becoming an empty, invalid semantic verdict. A strict backend resume validates payload, endpoint, model, GPU/CPU scope and expiry before sending a body-free resume; Stop and stale ownership cancel the caller, while custody loss remains explicit. A failed host argument check or runtime review failure is branded as a host rejection and cannot be converted into successful completion by the legacy tool-adapter compatibility path. If an issued quote correction returns no verdict, the first actual rejected output and a separate transport-only audit are retained; no second verdict is invented and finalization cannot restart that correction allowance. Cancellation cannot emit accepted completion.

Admission allows at most two complete proposal attempts, with an independent critic for each structurally valid proposal. The host never fills missing semantic effects, invents a source-comparison requirement, or substitutes its own successful verdict. Prior model interpretations remain revisable against the retained human sources; they are not additional human instructions.

The final admission interface asks the model for a semantic purpose and communication mode. A versioned host compiler maps those declarations to requested obligations; it grants no evidence credit. Exact source aliases remove model copying of opaque native IDs and hashes, while the complete original request and accepted steering remain available to both critics. Rejected outputs stay in a private audit; only bounded diagnostics re-enter the repair packet. Independent review found and closed one remaining size leak in malformed `communication.mode`, with oversized-string and nested-object regressions.

Nine primary references were inducted with acquisition hashes, quality limits and provenance. The [synthesis](../.aiwg/research/synthesis/context-engineering-live-admission-20261022.md) separates this short-control-call classification problem from the earlier accumulation of large observations. On the same 217-character request, captured critic data fell from 9,046 to 1,756 characters and retry data from 14,675 to 1,247. These are context-string measurements, not a claim of equivalent token or runtime savings.


The 04:29 final critic also exposed a separate context-growth defect: 43,803 prompt tokens for the small fixture. The model packet repeated the same 31,895-byte evidence array in two host slots, including an 11,612-byte prior-review audit, and repeated canonical requirements and current source state. The new model projection removes exact duplicate host copies and bulky payloads only from narrowly identified review bookkeeping. Every factual evidence row, current source body and native human source remains; the full raw input, audit, parser and currentness bindings remain unchanged.

A CPU replay of the already projected 04:29 packet reduced review-evidence bytes from 77,737 to 26,415 and full UTF-8 user-packet bytes from 89,405 to 38,083. That replay restored only the host snapshot schema tag; it did not reconstruct previously omitted scheduling fields or compare the original production binding hash. Reconstructed-input immutability and source-body preservation were checked separately from the production binding regressions. These byte measurements establish no post-change token or latency result.

The read-loop repair preserves the exact same-run invocation context as quoted protected context across steering and reconstruction, alongside the original native objective. Context does not become new human or effect authority, and a new invocation clears the old scope. Phase-specific edit procedures now require exposed edit methods or actual discovery capability. Focused runner tests cover the read-only scope, unchanged mutation obligations, cached rereads, honest incomplete handoff and scope reset. The actual 05:05 read-only continuation passed in three primary turns, and the independent negative critic correctly rejected the wrong-font claim.

A second source defect was isolated after both actual repairs: argument-only novelty made a read of the changed file look unchanged, while criterion-only progress left a stale plateau counter. The repair renews a read only for a changed on-disk revision after an actual owned mutation to that path. Genuine successful file mutations reset the no-delta streak after before/after byte checks; they do not satisfy a requirement or substitute for final review. Repeated reads, no-op writes, new call IDs and refused calls do not renew that allowance.

## Real terminal findings

Actual Node → Bun native-terminal tests exposed two input losses before inference: a submission arriving before startup handlers were installed, and input captured by a visually closing modal after the host had returned input custody. Startup now buffers accepted submissions until handlers are ready. Closing decorations relinquish interactive custody immediately while cleanup remains bounded and owned by the retiring modal instance.

After accepted steering, cached reads are now eligible only under their original run, epoch, objective revision and requirement-set digest. A changed owner forces real tool execution and a new receipt; the same-revision path retains its deduplication. An actual same-run CPU regression failed before the patch and passed after it. Capability guidance also stops advertising discovery when `tool_search` is unavailable, and explicitly identifies its bounded name list as a preview of complete exposed tool definitions.

A real PTY reproduction accepted input one millisecond after modal close while the decoration was still retiring. CPU lifecycle tests also cover cleanup and same-ID modal replacement. This is separate from an inference-through-the-TUI pass.

The real native TUI inference run ending at 03:33:38 UTC passed all eight checks in 103.4 seconds using 15 actual inference requests. It exercised the compiled CLI entry point, native model picker without inference, streamed chat, a registered file read returned to the model and painted in the terminal, editable input plus scrolling/resize during streaming, actual transport cancellation, and terminal restoration. Observed input latency during streaming was 25 ms; Stop restored editable input in 107 ms. These are one-run fixture timings, not a general latency benchmark. This run used build 7; the subsequent execution-obligation admission change requires its own validation.

## Verification status

**HOLD publication.** Build 19 passed the CPU checks and admitted the formerly refused live verification reads, but the final 05:17 continuation failed. Its report SHA-256 is `3a08186a1fd062f3e72401b1defbbd3d74c170456ae5b97e97ba0b5ed8e49388`; the unchanged report, trace, placement observations and resulting fixture are preserved in the private ignored acquisition cache. No final positive completion was accepted. The user directed that further fixes be deferred and this exact RCA be filed on GitHub and pushed to `main`. The latest build’s real-TUI inference run was therefore not started; the earlier build-7 pass remains separate evidence.

The current source has passed 460 orchestrator cases and 204 CLI cases across 45 test files, including the required web transport lifecycle checks. Four older native-transport failures were reproduced unchanged against commit `3a6fc147` in an isolated HEAD-source copy using the same installed dependencies (not a historic dependency rebuild). Their stale method-source and independent-intent fixtures were updated; the complete file now passes 39 cases with one intentional skip. Two further stale recovery fixtures expected optional grounding despite having no grounding consumer. The same two failures were reproduced against unchanged HEAD with current installed dependencies. Their assertions now check that the actual failure remains in the next primary packet and no unused optional grounder runs; bounded recovery assertions remain. Repeated runs and native Bun subcases are not counted twice. All eleven workspace packages were rebuilt after the v4 admission, owner-bound read-cache, shared retained-review custody, runtime-failure, review-context projection, invocation-scope and post-mutation read-guard changes; the 204-case CLI suite then passed against those rebuilt exports. The builder’s publish-artifact audit passed. Any final tarball audit is packaging evidence only and cannot clear the failed live-completion gate.

## Remaining blocker: stale historical claims in current context

Tracked in [GitHub issue #4](https://github.com/robit-man/open-agents/issues/4).

The failure is in the production context projection, rather than a missing repair or a fresh-read refusal. `_recordCompletionSummaryClaims` in `agenticRunner.ts` persists assertions from earlier completion summaries. `_refreshOperationalWorkGraph` supplies the full completion ledger to `operational-world/workGraphProjection.ts`, whose proposed-claim loop projects historical statements without checking current physical revisions or supersession. `claimAuthority` labels non-derived claims, including `task_complete_summary`, as adopted. `reasoningSlice.ts` renders those statements without their original run, time or artifact revision.

The actual late system context still said that the page had DefaultSans and an empty footer, that “the two required mutations were NOT performed,” that “page.json is unchanged,” and that no editing tool was exposed. Those statements came from the earlier read-only phase. Both real edits had subsequently succeeded; the current file reads matched the reference and `file_edit` was exposed. The worker explicitly called this a conflict and repeated artifact retrieval through its final bounded turns.

The independent critic did not miss the real effects. It stated: “two successful file edits (ev_0013, ev_0017) and current source-review reads (ev_0025, ev_0026).” It rejected the candidate’s inaccurate claim that no change had been needed. The worker then tried to resolve a stale artifact-state conflict instead of reconciling the action timeline and correcting that report. Current-byte parity is real; the critic’s bundled objection to the separate byte-identical wording does not invalidate the matching current hashes.

The deferred repair must preserve the actual historical receipts while preventing superseded summaries and old capability limits from masquerading as current facts. It needs a regression covering read-only incomplete handoff → real repair → process restart → fresh verification → truthful final report, including an unchanged-read negative control. Hiding the critic, discarding the original request, changing the requirements, or treating confidence as evidence would not repair this defect.

## Real conversations over mock evidence

The [complete dialogue](completion-deliberation-examples-20261022.md) preserves actual executor/critic responses from 15 physical requests to the configured model. Three explicitly seeded false claims were rejected with valid `artifact_failed` verdicts: unchanged wrong font, a required command exiting 2, and an external delivery attempt without acknowledgement. Executors withdrew the unsupported claims and described the required repair.

The fourth control supplied a synthetic repair receipt. The critic correctly left the mutation unassessed because a mock receipt does not establish an actual file change. That control is inconclusive and the raw demonstration report has `passed: false`; this is not presented as an overall passing execution test. The separately recorded real-file sequence is responsible for demonstrating successful actual repair.

## Limits

The initial retained-objective continuation used a new runner in the same OS process. The 05:05 continuation additionally loaded the authentic retained objective and latest ledger in a new OS process after the prior process had stopped; its original request, accepted steering and pending verification obligations survived. That run still ended incomplete at the plateau defect, so process recovery alone is not completion evidence. The neutral fixture is bounded. It is not a production soak, a browser-layout comparison, or a Telegram-delivery test. Replaying a real rejection through the host's challenge tracker proves a bounded guard; it does not prove an autonomous model will stop proposing unchanged claims under every workload. Synthetic repair evidence is a simulation transition, not an observed tool effect.
