# v0.8.0 — Screenshots at the size an agent actually reads

**2026-08-08**

## TL;DR

Every screenshot this server has ever sent was 1920x1080, because that is the size the
display happens to be. Nobody reads them with human eyes. Measured against the smallest
text on the open web, an agent reads everything on a page at a 1024px long edge and reads
the titles and domains at 800px — so that is what gets sent now. A picture that cost
~2,500 image tokens costs ~440, and a run that takes thirty of them gets about 38,000
tokens of its context window back.

One new argument, `detail`, lets a call ask for more or less. Two new flags move the
defaults.

## Why

Image cost is charged by area, not by file size: roughly `width × height / 750` tokens.
A 1920x994 capture is ~2,540 tokens — except Anthropic clients silently resize anything
over 1568px on the long edge before charging, so the real bill was ~1,700 tokens and the
extra 400KB of base64 crossing three proxy hops bought nothing at all.

That bill is fine once. The screenshot bundled with *every* mutation is not once. A long
visual-heavy run — click, look, click, look — was spending more context on pictures of
pages than on the work.

The question was never "how small can a picture be", it was "how small can a picture be
**and still be read**". That has an empirical answer, and it turns out to be far below
where the server was operating.

## What was measured

Hacker News, at 1920x994, is close to the worst case on the open web: 11px story titles
over 7px metadata. A page of 27 fresh headlines was captured once, downscaled to each
rung of the ladder, and transcribed blind — lowest resolution first, so no rung could be
helped by having seen a sharper one. The transcriptions were then scored against the
DOM text.

| Long edge | Image tokens | Result |
|---|---|---|
| 1568px | ~1,700 | Everything. The ceiling worth paying for. |
| 1024px | ~720 | Every glyph, including the 7px `points by … | hide | comments` line. |
| 800px | ~440 | 27/27 titles exact, domains readable, metadata line marginal. |
| 640px | ~280 | 24/27 titles. All three misses were proper nouns or digits: `Dsv4` read as `OpenAI`, `M3` read as `M5`, `Partmode` read as `Partroute`. Gist intact, details unsafe. |
| 512px | ~180 | 18/27. Layout and shape survive; text does not. |

The failure mode at 640px is the interesting one: it is not that text goes blurry, it is
that unfamiliar strings get *confidently guessed wrong* from context. That makes 640px
fine for "where is the button" and wrong for "what does this say".

Most applications set body text at 14–16px rather than Hacker News' 11px, so these
numbers are a floor, not an average.

## Highlights

| Change | Effect |
|---|---|
| Captures are downscaled before they leave the server | ~2,540 nominal / ~1,700 charged tokens per picture → ~440 |
| Two budgets, not one | A picture you asked for is 1024px; one attached for you is 800px |
| `detail` argument on every context-capable tool | `low` \| `medium` \| `high` \| `full` — per call, no server restart |
| `saveTo` untouched | A screenshot written to a file costs no tokens, so it keeps full resolution |
| Area-average minification | Correct filter for downscaling; sharper kernels ring around glyph edges at these ratios |
| No new dependencies | `pngjs` was already in the tree |

## How it works

Downscaling happens at the response boundary, not inside the capture path. Every reply
leaving the tool dispatcher passes through one transform that rewrites any `image/png`
block in it. One rule therefore covers all four ways a picture can reach a reader:
`screenshot_page`, `screenshot_by_uid`, the bundle attached to a mutation, and the
picture attached to an ownership refusal.

The budget for a given reply is chosen once, before the tool runs:

- an explicit `detail` on the call always wins;
- otherwise `screenshot_page` and `screenshot_by_uid` — pictures asked for by name — get
  the explicit budget, default 1024px;
- otherwise the picture was attached unasked, and gets the auto budget, default 800px.

The resampler averages each destination pixel over the source pixels it covers. Measured
cost is ~180ms for 1920x994 → 1024x530, against a capture path that already waits up to
8s for the page to settle and 500ms for the compositor — it does not show up.

Every failure path returns the original bytes. A picture that arrived is worth more than
a picture at the right size, so a corrupt decode, an unexpected format or a capture that
is already small enough all pass straight through.

Setting either budget to `0` disables downscaling on that path entirely.

## Config

| Flag | Env | Default | Meaning |
|---|---|---|---|
| `--screenshot-max-edge` | `FIREFOX_SCREENSHOT_MAX_EDGE` | `1024` | Long edge for a picture an agent asked for by name. `0` disables. |
| `--screenshot-auto-max-edge` | `FIREFOX_SCREENSHOT_AUTO_MAX_EDGE` | `800` | Long edge for the picture bundled with a mutation or a refusal. `0` disables. |

Per call:

```jsonc
{ "tab": "a5126214", "detail": "low" }    // 640px  — layout and headings
{ "tab": "a5126214", "detail": "medium" }  // 1024px — every glyph (default for explicit)
{ "tab": "a5126214", "detail": "high" }    // 1568px — the ceiling
{ "tab": "a5126214", "detail": "full" }    // no downscale
```

`detail` also accepts a raw pixel number if none of the rungs fit.

## Files changed

- `src/utils/image-scale.ts` *(new)* — detail ladder, configurable budgets, area-average
  PNG minification, response-boundary rewrite
- `src/index.ts` — budget resolved per call; applied to the dispatcher's return paths and
  to the refusal picture
- `src/context-bundle.ts` — `detail` advertised on every context-capable tool and stripped
  before handlers see it
- `src/cli.ts` — `--screenshot-max-edge`, `--screenshot-auto-max-edge`

No breaking changes. Bridge package untouched.