---
created: 2026-05-06
gap: G06
order: 4
severity: P1
category: observation
estimate: M
issue: "https://github.com/lidge-jun/agbrowse/issues/60"
depends_on: ['G02']
---

# G06 — Unified multimodal observation bundle (vs Playwright MCP/VisualWebArena/Vercel)

> Severity **P1** · Category `observation` · Estimate **M** ·
> Tracking issue [#60](https://github.com/lidge-jun/agbrowse/issues/60) · Depends on **G02**

## GPT Pro evidence

Evidence (competitor side): URL: https://github.com/web-arena-x/visualwebarena — quote: “visual tasks.” VisualWebArena targets multimodal agents, while Playwright MCP snapshots can include bounding boxes and screenshots. 
GitHub
+1

Evidence (agbrowse side): README.md:446-454 exposes snapshot, screenshot, text, DOM, console, and network separately; README.md:496-503 documents vision-click, but not one combined observation bundle. 
GitHub
+1

Why this matters: Visual and multimodal benchmarks require screenshot-grounded action decisions, not just a DOM snapshot. A bundle also makes failures reproducible because refs, boxes, DPR, and screenshot are captured together.
Proposed scope, respecting forbidden list:

web-ai/observation-bundle.mjs — emit URL, title, viewport, DPR, snapshot refs, bounding boxes, screenshot path, and text summary.

skills/browser/browser.mjs — add observe --bundle --screenshot --boxes --json.

web-ai/target-resolver.mjs — accept bundle refs and boxes for safe coordinate fallback.

web-ai/trace/observation-redactor.mjs — redact sensitive text from stored bundles according to existing trace policy.

structure/commands.md — define ObservationBundleV1.

structure/release_gates.md — add gate:observation-bundle-fixtures.
Test surface: test/unit + test/integration + test/eval.
cli-jaw mirror impact: parity optional unless cli-jaw claims observation-bundle parity.
Acceptance gate: keep trace redaction gates; add gate:observation-bundle-fixtures.
Estimate: M, 2–3 days.

## Diff-level work breakdown

> Fill in concrete diffs (NEW / MODIFY / DELETE with file:line) once this gap
> reaches the active sprint. Until then, the bullets in **Proposed scope**
> above are the agreed shape; do not implement before the depending gaps
> (G02) ship and `gate:all` stays green.

### NEW files
- _to be filled before implementation_

### MODIFY
- _to be filled before implementation_

### DELETE
- _to be filled before implementation_

## Tests
- `test/unit/...` — _list test files once written_

## Truth-table update
- `structure/CAPABILITY_TRUTH_TABLE.md` — add row or update status when this
  gap reaches `ready` in agbrowse.
- `cli-jaw/structure/CAPABILITY_TRUTH_TABLE.md` — mirror entry per the
  `cli-jaw mirror impact` line above.

## Release gates touched
- Existing: `gate:typecheck`, `gate:tests`, `gate:truth-table-fresh`,
  `gate:mcp-scope-frozen`, `gate:no-experimental-in-readme-ready-section`.
- Added by this gap: see **Acceptance gate** in the GPT Pro evidence block.