# WO-08 — Browser, files and native application interaction

Requested: 2026-10-12. Status: controlled application integration verified;
natural model planning and installed-bot deployment remain separate open gates.

Run the same typed computer-use system across application boundaries. The
requested sequence is API documentation in a browser, a new folder, entering
that folder, a subfolder and adjacent text file, opening that file in gedit,
appending the retrieved API details, and verifying the actual saved file.

## Test contract

- Use real installed browser, file manager and gedit applications with owned
  windows/processes. Report unavailable applications rather than substituting
  a synthetic editor and calling it gedit.
- Restrict mutations to a unique test directory. Preserve existing files,
  running bot/voice services and unrelated application windows.
- Use the production semantic browser/native tools, finite action IDs and
  expected effects. A scripted/rules planner measures dispatch and application
  integration; it does not establish autonomous model planning quality.
- Retain the documentation URL, observed content and source hash/span evidence
  used in the note. Preserve initial note contents when appending.
- Verify folder topology, file-manager location, editor contents, save outcome,
  exact path and independently read saved bytes. Do not infer filesystem success
  from a click, editor title, screenshot change or guessed Downloads location.
- Record action/observation timings, progress, unresolved failures, screenshots
  and owned-process cleanup. Bound retries and stop at a real unsupported
  control; do not silently replace failed GUI actions with shell mutations.
- Add an opt-in reproducible integration test and update platform limitations
  for any discovered gap. No local model calls without current hardware gates.

Acceptance is the complete requested cross-application sequence through the
actual implementation, or an evidenced blocker plus a verified scoped repair
and rerun. Record source/runtime/deployment status separately.

## Discovered provider repair

The real installed GTK4 Nautilus exposes usable controls with AT-SPI `SENSITIVE`
without `ENABLED`; its disabled Back/Forward controls lack `SENSITIVE`. Requiring
both states made every control in the observed application ineligible. The
Linux bridge now uses `SENSITIVE` for interaction eligibility and rechecks it
immediately before dispatch, preserving visibility, active-window, revision,
generation and defunct-object checks. This follows the
[AT-SPI state contract](https://docs.gtk.org/atspi2/enum.StateType.html).
GTK4 menu toggle buttons also expose `PRESSED` rather than `CHECKED`; the bridge
maps that state for toggle buttons while retaining checkbox semantics. An
already pressed toggle is not toggled off by `set_checked: true`.
The same ordinary menu reaches depth 35 with 242 nodes and at most 11 children
per node. The bridge's former depth-32 cutoff caused a truthful truncated-tree
rejection. Traversal now permits depth 64 while retaining the 600-node and
800-child limits; deeper pathological trees remain truncated and ineligible.

Validation: 31 native/state tests, 14 real isolated GTK3 accessibility tests and
17 native save-verification tests passed; execution package build passed. The
complete cross-application rerun remains in progress.

## Mixed-tool verification repair

Guarded keyboard input could reach the no-delta threshold before `native_ui`
was allowed to inspect its effect. The tools now return private, host-authored
desktop/bridge custody metadata, preserved by the CLI adapter. The runner may
durably consume one plain native observation after an actual guarded keyboard
dispatch on that same desktop. This gives no effect or completion credit and
does not reset the frontier. Another shortcut, screenshot, observation revision,
browser navigation or bridge restart cannot replenish the allowance; a newly
applied native semantic action can.

Ownership binds the task lifetime, objective, registered tool instances and
actual provider environment. Per-turn cancellation still suppresses stale
returns. Scope changes, Stop, dry runs, failed input, failed persistence and
tool replacement fail closed. An offline scripted backend regression crosses
five ordinary model turns, admits one verification read, refuses its repeat and
leaves the terminal criterion open. This exercises the normal model loop in
addition to the host-session path without calling a model server.

Focused validation: execution 86, orchestrator 64 and CLI tool-contract 18
tests passed. The real application rerun and complete workspace build are
tracked separately below.

## Native window capture repair

The real rerun created the first folder, then stopped after entering it:
Nautilus changed its title from `workspace` to `API research`, and post-input
capture searched for the old title. Post-input capture now follows the already
observed X11 window ID, refreshes geometry, and verifies geometry and desktop
scope through capture. A renamed window cannot redirect capture to a decoy with
the old title. Closed/unreadable windows remain unresolved; independent saved
file verification retains its existing behavior when a Save dialog closes.

Validation: 74 desktop/vision tests and the execution build passed. The actual
Stop case after the first folder passed with no note or terminal success and
all owned processes stopped. Completion is still being rerun through the apps.

## Accessibility observation repair

The next real attempt created both folders and reached gedit, then correctly
rejected an edit after the provider returned a portal-only snapshot. A separate
ten-read probe retained all three registered applications, so the live loss is
not established as a deterministic application crash. It did reveal background
menus exhausting the 600-node limit. Earlier gedit reads queried hundreds of
hidden widgets and produced unrealized-widget geometry warnings.

The bridge now queries geometry only for visible controls and retains inactive
top-level window identities without traversing their descendants. Hidden layout
ancestors within the active window remain traversable. Fresh desktop enumeration
clears the root cache. Required identity/state/role/count/child reads fail closed,
including negative child counts, null children and DEFUNCT responses; libatspi
can also synthesize DEFUNCT after a failed state RPC. A failed observation cannot
replace the last handles, generations or window identities, or prove disappearance.
Optional text and geometry remain best effort. Input revision and focus checks
are unchanged; no mutation retry or longer timeout was added.

Validation: 45 provider/native tests, 14 real GTK accessibility tests and 17
saved-file tests passed, along with the execution build. The application suite
is being rerun against the compiled repair.

## Read recovery after native window closure

The repaired provider correctly rejected an incomplete read immediately after
gedit closed with exit code zero. The native action had dispatched once; its
first verification read encountered the briefly stale application registry.
Required-read failures now return a fixed, sanitized unavailable-observation
code. The controller spends its existing 2.5-second/20-observation verification
budget on further reads only for that typed error after a dispatch receipt.
It does not repeat input, retry before dispatch, accept a partial graph or retry
arbitrary protocol/operation failures. Failed reads count toward the ceiling;
Stop interrupts the wait. Exhausted recovery retains an unresolved effect.

Validation: 226 focused provider, controller, verification and evidence tests
passed, and all 11 workspace packages built. Real application reruns follow.

## Single-option selection repair

The reopen test found that AT-SPI `select_child` adds an item in Nautilus's
multi-select container. The requested note and the previously selected folder
became selected together, and exact verification correctly withheld Open.
The native single-option contract now clears the prior selection before choosing
the uniquely named option. Duplicate names are rejected before any clear.
If clearing or the subsequent selection declines or throws, the receipt retains
dispatched/uncertain input so partial changes cannot be mislabeled absent.

Validation: 81 focused tests and 14 real GTK tests passed, including replacement
of a preselected option in an actual multi-select list, plus the execution build.
GTK4 Component focus and activating a file through Nautilus's location entry
remain unsupported routes in the retained attempts; the workflow uses the
observed selection pattern and a separately recorded guarded Open shortcut.

## Hidden-leaf stability repair

The workflow then reopened the exact note with its initial bytes intact, but
append was rejected before dispatch when one hidden recent-document menu leaf
changed identity. All other 483 nodes, the foreground scope and editor textbox
were unchanged. Observation now omits hidden leaf controls after a required
child-count read; hidden ancestors remain traversable so visible descendants
are preserved. The 600-identity traversal ceiling includes omitted leaves.
Visible changes still change revision and retain the usual stale-input rejection.

Validation: 73 focused tests and 14 real GTK tests passed; execution build passed.
The saved note remained unchanged in the rejected attempt, which is retained.
The first rerun also caught a text-count/child-count variable collision introduced
while moving the required count read. It was repaired and covered by a visible
nonempty text-leaf regression before repeating application tests.

## Completed application tests

| Source and desktop | Execution | Native / guarded keyboard actions | Exact note | Result |
| --- | ---: | ---: | ---: | --- |
| Shipped docs, isolated Xvfb | 26.236 s | 13 / 7 | 256 bytes, 3 source facts | Complete |
| Public Mapsource docs, visible Xephyr | 31.340 s | 13 / 7 | 302 bytes, 4 source facts | Complete |
| Stop after first folder, isolated Xvfb | 11.478 s | 4 / 1 | No note | Cancelled |

Both completions made 49 tool calls: seven browser, 35 native and seven guarded
keyboard calls. Each used three verified browser link actions, created the two
folders through Nautilus, opened the nonexistent exact note path in gedit,
edited and saved it, closed gedit, selected and reopened the saved file from
Nautilus, preserved its initial 20-byte prefix, appended the observed API facts
and saved again. The private gedit profile disables implicit trailing newline;
that preference is read back and recorded, without changing user settings.

Exact source URL/hash/UTF-16 spans, filesystem path/bytes/hash, final editor
buffer and native Save receipts agree. Each committed terminal receipt settles
16 effect identities. All eight owned processes per completion stopped; the
final Stop case stopped six and retained no successful terminal receipt. The
visible public run included ten seconds of review, excluded from its execution
time; launcher-through-cleanup time was 42.701 seconds. Final note images were
visually inspected. These are single-run timings, not a throughput distribution.

The opt-in CLI suite passed 7/7, including application completion and Stop.
Final focused execution validation passed 73 tests plus 14 real GTK tests;
17 exact-save tests and 21 independent-verifier tests also passed. All 11
workspace packages built. The independent archive verifier confirms both
completed cases and preserves earlier failed/cancelled outcomes. The launcher
also requires normal worker exit zero for success and records worker exit;
a recorded terminal result cannot conceal a later worker failure.

Reproduction and frozen evidence:
[desktop guide](../../../computer-use/CROSS_APPLICATION.md),
[evidence archive](../../../computer-use/evidence/cross-application-20261012/README.md),
[verification report](../../../computer-use/evidence/cross-application-verification-20261012.json).

## Runtime and model boundary

Every task tool call used the built production execution tools, CLI adapter and
`AgenticRunner.runHostTask` admission/cancellation path. The host planner was
fixed fixture/rules code with zero model requests. This verifies application and
tool integration, including explanatory progress and terminal verification;
it does not verify autonomous natural-language planning, Moondream perception,
decision-model latency or free-text Telegram/TUI behavior for this desktop task.

The globally installed bot remains 1.0.837 and its existing daemon was not
restarted or replaced. The active A100 voice processes remained running. The
retained broker preflight blocked new model loads because a selected GPU was
missing; it did not report all GPUs offline. WO-06's live model and deployment
gates remain open. No model load, CUDA service change or inference was performed.
