# OCR worker

The Python half of `ocr`. The tool spawns it once per call, hands it a JSON job
spec on stdin, and reads newline-delimited JSON events back from stdout. It
recognizes and merges; the tool writes every file.

## Files

| File | Role |
|---|---|
| `ocr_worker.py` | entry: reads the spec, owns the engine lifetime, reports each page, runs the merge |
| `source.py` | opens a PDF or a still image and renders one page on demand |
| `layout.py` | turns engine boxes into reading-order lines, finds columns, infers outline levels |
| `clean.py` | identifies folios and running heads, and joins the lines a wrap split |
| `assemble.py` | the merge pass: page records in, one Markdown document out |

## Environment

The tool looks for a virtual environment at `python/.venv` beside this
directory and falls back to `python` on `PATH`. Override it with the
`pythonPath` configuration field.

Create the environment once. Installing into it needs permission to create
directories with a private mode, so run this outside a restricted shell:

```sh
python -m venv python/.venv
python/.venv/Scripts/python -m pip install -r python/requirements.txt
```

Confirm it with the worker's own check, which is what the tool runs before every
call:

```sh
python/.venv/Scripts/python python/ocr_worker.py --self-test
```

## Protocol

stdin is one JSON object:

```json
{
  "engineLifetime": "perDocument",
  "jobs": [
    { "mode": "recognize", "id": "1", "input": "D:/docs/a.pdf", "pages": "1-5,8",
      "dpi": 200, "maxPixels": 12000000, "textScore": 0.5, "detectColumns": true,
      "maxPages": 2000, "merge": { "detectHeadings": true } },
    { "mode": "assemble", "id": "2", "input": "D:/docs/a.pdf",
      "record": { "pages": [ { "index": 5, "width": 595.0, "height": 842.0,
                               "lines": [ { "text": "...", "x0": 1, "y0": 2, "x1": 3, "y1": 4, "height": 12.0 } ] } ] },
      "merge": { "levelOverrides": { "h2": 3 } } }
  ]
}
```

`pages` is a comma-separated list of single pages and inclusive ranges, counted
from one, as a person would write them in a print dialog; a range left open
after its dash runs to the last page. It resolves to the explicit ascending list
of page numbers to process, so `"1-5,8"` on a twelve-page document is
`[1, 2, 3, 4, 5, 8]`. A malformed piece, a page below one, an end before its
start, and a selection that names no page of the document are errors for that
job.

A `recognize` job renders and recognizes the document it names. An `assemble`
job opens no document and builds no engine: it rebuilds the pages from the page
records the caller kept and runs the merge alone.

`merge` carries the merge pass's keys, all optional and in camelCase:
`removePageNumbers`, `removeRunningHeads`, `runningHeadRatio`,
`runningHeadMinPages`, `detectHeadings`, `headingMinRatio`, `indentRatio`,
`paragraphGapRatio`, `outlineCandidateRatio`, `maxOutlineCandidates`, and
`levelOverrides`.

stdout is one JSON object per line:

| Event | Fields |
|---|---|
| `start` | `id`, `input`, `pages` (the explicit list), `totalPages` |
| `page` | `id`, `page`, `totalPages`, `lines`, `chars`, `text`, and `geometry` — the page record an assemble job rebuilds from |
| `released` | `id`, `reason` |
| `done` | `id`, `input`, `pages`, `totalPages`, `lines`, `headings`, `chars`, `droppedPageNumbers`, `droppedRunningHeads`, `joinedAcrossPages`, `recognized`, `seconds`, `markdown`, `outline`, `outlineTruncated` |
| `error` | `id`, `input`, `message` |
| `end` | `documents`, `failures` |

An assemble job emits no `page` events: its `pages` and `recognized` are the
number of pages in its record, and its `seconds` is the merge's own duration.

The worker reads only the document each recognize job names. It writes no file:
every artifact is written by the caller through the mounted filesystem
capability, so a deployment's file policy sees the write. Diagnostics go to
stderr.

## Outline pass

`done` closes with `outline`, the lines whose outline level the heuristic may
have read wrongly, and with `outlineTruncated` when `maxOutlineCandidates` (400
by default) cut that list short. A caller that wants a model to check the
outline sends these candidates to the model and comes back with one `assemble`
job per document whose `merge` carries `levelOverrides`; nothing is recognized
twice.

A kept line becomes a candidate when any of these holds: the heuristic gives it
a level, it opens with a section number, it is at least `outlineCandidateRatio`
(1.05 by default) times the body glyph height, or it opens with a named section
such as an abstract or a reference list. A list item that carries no number
never does. Ids are `h1`, `h2`, … in document order, and each candidate reports
its `page`, its normalized `text`, the heuristic `level` (or `null`), its
glyph-height `ratio`, and its section `numbered` when it opens with one.

`levelOverrides` maps a candidate id to the heading level to use in place of the
heuristic one: 1 to 6, with anything above 6 clamped to 6. A value below 1, like
`null`, makes the line body text, which is how a candidate the heuristic wrongly
called a heading is demoted. A candidate the map does not mention keeps the
heuristic's own decision, and a value that is not an integer is an error for
that job.

## Tests

The unit tests cover the pure layout, cleaning, merge, and job-protocol rules
and need no third-party package:

```sh
python -m unittest discover -s python/tests -t python
```