# Exploration: Associating Context-Window Entries with Todo-Scoped Run Periods

**Goal (from user):** Understand how granular context-window entries (messages / tool
results) are associated with periods of an agentic run, and specifically whether they are
tied to a *todo item* so that, when that todo completes, its context content can be
compressed.

**Status:** Exploration complete. No code changes made — this is a design/feasibility
write-up. The conclusion is that **todo-scoped compression does not exist today**; the
mechanism must be added.

---

## 1. Current architecture (evidence-grounded)

### 1.1 The context window is a single flat message stream
- `packages/orchestrator/src/agenticRunner.ts` holds the live context as a flat
  `ChatMessage[]` array (the `messages` passed to the model). Tool results, system
  directives, and assistant turns are all appended to this one array
  (`messages.push(...)` at many sites, e.g. L4518, L4954, L13307, L13333, L13812…).
- `ChatMessage` is defined in `context-compressor.ts` (L23-31):
  ```ts
  export interface ChatMessage {
    role: "system" | "user" | "assistant" | "tool";
    content: string | null;
    tool_calls?: Array<{ id: string; type: "function"; function: { name: string; arguments: string }; }>;
    // … no todoId / spanId field exists
  }
  ```
  **There is no `todoId` / `spanId` on a message.** Messages are not tagged with which
  todo was active when they were produced.

### 1.2 Compression is PHASE-based, not TODO-based
- `packages/orchestrator/src/contextTree.ts` — `ContextTree` is a "hierarchical
  task-phase-aware context tree". It groups the message stream into **phases**:
  `explore`, `plan`, `implement`, `verify`, `mixed` (contextTree.ts L10-15).
- It tracks `_phaseMessageStartIdx` (agenticRunner L2441-2445): on a phase transition,
  the slice `messages[_phaseMessageStartIdx..now]` is captured as the OUTGOING phase's
  owned slice via `tree.observePhaseMessages`, then the cursor advances.
- `tree.contractInactive(...)` (agenticRunner L10384) summarizes inactive phases;
  `tree.archive(phaseName, archPath)` (L10413) writes them to disk.
- **So compression today is keyed to phase transitions, not todo completion.**

### 1.3 The compressor
- `packages/orchestrator/src/context-compressor.ts` —
  `StructuredContextCompressor.generateSummary(compMessages)` (agenticRunner
  L27465-27492) produces a `CompactionSummary` (goal, constraints, progress,
  keyDecisions, relevantFiles, nextSteps, criticalContext).
- It is invoked from a budget/compaction path; **not** from any todo-completion hook.

### 1.4 Persistent task boundaries (`messageLog.ts`)
- `packages/orchestrator/src/messageLog.ts` — append-only JSONL
  `.omnius/sessions/{id}.jsonl` with `task_boundary` markers + `task_summary` user
  messages (analogue of Hannover's `SystemCompactBoundaryMessage`).
  `loadBoundarySlice` returns the post-last-boundary slice.
- This is a *top-level task/goal* boundary, **not** a per-todo-item boundary.

### 1.5 Todo tracking
- `TodoReminderTodo` (agenticRunner L1771-1777):
  ```ts
  export interface TodoReminderTodo {
    id?: string;
    content: string;
    status: "pending" | "in_progress" | "completed" | "blocked";
    parentId?: string;
    blocker?: string;
  }
  ```
- Todo state is maintained for the **reminder-gating** function
  (`shouldInjectTodoReminder`, L1802-1843) — it checks "turns since last todo_write" to
  nudge planning. It does **not** record start/end turns and does **not** associate
  messages with todos.
- There is **no span map** linking a todo id → `[startTurn, endTurn]` → message indices.
- The dedicated context-intake module `context-fabric.ts` (473 lines — a typed "Context
  Fabric" that emits one bounded frame before each model call) contains **0** references to
  `todo` (`grep -c 'todo' = 0`). This confirms that even the purpose-built context layer
  does **not** associate context entries with todos.
- `todoTruth.ts` exists as a separate todo-truth/verification module, but it is not wired
  into the message stream or the compressor, so it does not provide todo→context span
  tracking either. It is a candidate integration point for future work.

---

## 2. Gap analysis (the direct answer to the user's question)

**Today, granular context entries are NOT associated with a todo item.** They are
associated with:
- a flat global stream (no per-message tagging), and
- a *phase* (via `ContextTree`), which is a coarse activity classification
  (`explore`/`plan`/`implement`/`verify`), **not** a todo.

Therefore *"when the todo is complete, compress that context"* is **not currently
possible** without first adding todo-scoped span tracking. The building blocks
(compressor, archiver, boundary markers) already exist — they are just wired to phases
and top-level tasks, not to todos.

---

## 3. Proposed design: todo-scoped compression-on-completion

Three additive pieces are needed:

### 3.1 Tag messages with the active todo id
Add `todoId?: string` to `ChatMessage` (or maintain a parallel
`Map<messageIndex, todoId>`). Set it whenever a `todo_write` marks a leaf `in_progress`,
and roll it to the next leaf when the active todo changes.

### 3.2 Track todo active spans
Maintain `Map<todoId, { startTurn, endTurn, startMsgIdx, endMsgIdx }>`. On `todo_write`:
- leaf → `in_progress`: open a span (record current turn + `messages.length`).
- leaf → `completed` / `blocked`: close the span (record end turn + `messages.length`).

### 3.3 Compress on completion
When a todo span closes, take `messages[span.startMsgIdx .. span.endMsgIdx]`, run
`StructuredContextCompressor.generateSummary(...)` (reuse the existing compressor), and
replace that slice with a single compact `task_summary` user message (mirroring
`messageLog.ts`'s boundary pattern). This reclaims tokens for finished work while keeping
a retrievable summary.

### 3.4 Reusable integration points (already in the codebase)
- `ContextTree.contractInactive` / `archive` — reuse the summarize + persist pattern.
- `StructuredContextCompressor.generateSummary` — reuse for the summary text.
- `messageLog.ts` boundary markers — reuse the `task_summary` convention for the
  replacement message.
- `shouldInjectTodoReminder` (L1802) — natural hook to detect todo status transitions and
  drive span open/close.

---

## 4. Open questions / risks
- **Nested todos (`parentId`):** compress only leaf spans, or roll up to the parent on
  parent completion?
- **Re-reads after compression:** the recently-fixed file_read de-dupe removal means the
  model can re-read files for live state. Compressed todo summaries must remain available
  via `surfaceAnchors` / `messageLog` retrieval, not silently dropped.
- **Token accounting:** ensure the replacement summary is smaller than the compressed
  slice (budget guard) so compression is net-positive.
- **Ordering:** spans can overlap if the model interleaves todos; the span map must handle
  non-contiguous message indices (a todo's messages may be interleaved with another's).

---

## 5. Evidence index
| Fact | Location |
|------|----------|
| Flat `ChatMessage[]` context stream | agenticRunner.ts (many `messages.push` sites; L4518, L4954, L13307…) |
| `ChatMessage` has no todoId | context-compressor.ts L23-31 |
| Phase-based grouping | contextTree.ts L10-15, L260-313 |
| Phase message cursor | agenticRunner.ts L2441-2445 |
| Phase contract/archive | agenticRunner.ts L10384, L10413 |
| Compressor invocation | agenticRunner.ts L27465-27492 |
| Persistent task boundaries | messageLog.ts (JSONL + `task_summary`) |
| Todo interface | agenticRunner.ts L1771-1777 |
| Todo reminder gating (no span tracking) | agenticRunner.ts L1802-1843 |
| Dedicated context layer has 0 todo refs | context-fabric.ts (`grep -c 'todo' = 0`) |
| Separate todo-truth module, not wired to context | todoTruth.ts |