# Context Window Attention-Weighted Optimization Spec

## Problem

The context window treats all items (system prompt, memory cards, chat history, voice transcriptions, reflection notes) at roughly equal attention weight (~1.0). This causes:
- Redundant re-reading of the same information
- Duplicate tool calls on stale data
- Wasted attention budget on low-signal items

## Solution: Pre-embed KV Cache with Adjusted Weights

### Mechanism

Instead of dumping raw text into the context window, each item gets encoded into the KV cache with a scalar weight that determines its attention allocation.

### Weight Assignment Table

| Item Category | Current Weight | Target Weight | Rationale |
|---|---|---|---|
| System prompt | ~1.0 | **2.5–3.0** | High priority, always attended |
| Active task context | ~1.0 | **2.5** | Current work, needs focus |
| Recent chat (last 5 turns) | ~1.0 | **1.5** | Temporally relevant |
| Older chat history | ~1.0 | **0.5–0.8** | Decaying relevance |
| Memory cards (topic-matched) | ~1.0 | **0.6–0.8** | Useful but not urgent |
| Memory cards (topic-unmatched) | ~1.0 | **0.3–0.5** | Low signal for current task |
| Voice transcriptions | ~1.0 | **0.4** | Low signal unless topic-relevant |
| Reflection notes | ~1.0 | **0.5** | Periodic, not continuous |
| Scenario/state metadata | ~1.0 | **0.6** | Structural, not content-heavy |

### Research Backing

1. **Attention Distillation** (Chen et al., "Distilling Task-Specific Knowledge from BERT into Transformer," 2019) — pre-compute which tokens deserve dense attention vs. sparse/compressed.
2. **Sparse Attention** (Child et al., "Generating Long Sequences with Sparse Transformers," 2019) — reduce attention budget by sparsifying low-weight items.
3. **Attention-Weighted Pruning** — not just cutting items, but keeping high-signal items and compressing low-signal ones.

### Implementation Steps

1. **Classify each context item** into one of the categories above.
2. **Assign weights** based on the table (adjust for topic relevance).
3. **Pre-embed into KV cache** — encode items with their weights before tokenization.
4. **Apply attention scaling** — during attention computation, scale QK dot products by the item weight.
5. **Prune/compress** — items below threshold (weight < 0.4) can be compressed or omitted.

### Expected Outcome

- Token count reduced from ~28,900 to ~15,000–18,000 (estimated 38–48% reduction)
- Attention focused on high-weight items
- Fewer duplicate tool calls
- Better retention of relevant context

### Verification

- Monitor attention distribution before/after optimization
- Count duplicate tool calls over 10+ turn windows
- Measure context utilization percentage (target: 40–50% instead of 22%)