# Context Management for Medium Models (30-40B) — Implementation Proposal

**Date:** 2026-04-25
**Problem:** Medium-tier models (~35B params) exhibit excessive repetition at ~35% context fill despite 256K context windows
**Root Cause:** Monolithic context structure + attention degradation + tool schema bloat

---

## 1. Literature Summary

### Key Findings

| Paper | Key Insight | Application |
|-------|-------------|-------------|
| **Lost in the Middle** (Liu et al., 2024) | U-shaped attention: strong at start/end, 40% accuracy drop in middle 50% | Position critical info at edges |
| **RECOMP** (ICLR 2024) | Context compressed to 6% with minimal quality loss via observation masking | Aggressive tool output masking |
| **AgentFold** (arXiv:2510.24699) | Multi-scale folding prevents exponential fact decay (0.99^100 = 36.6%) | Progressive summarization with locked blocks |
| **ARC** (arXiv:2601.12030) | Active revision + reflection = 11% accuracy gain | Structural preservation through compaction |
| **NATURAL PLAN** (arXiv:2406.04520) | GPT-4 only 31% on planning even with full context | Efficient use > naive expansion |
| **ToolLLM DFSDT** (arXiv:2307.16789) | Error preservation + backtracking = +35pp success | Error-preserving compaction strategy |
| **SPRINT** (arXiv:2506.05745) | Parallel sub-calls distribute reasoning | Sub-agent delegation for independent tasks |
| **Recursive LMs** (arXiv:2512.24601) | Externalize to REPL, recursive chunk analysis | RLM context OS layer |

### Medium Model Specific Issues

1. **Earlier attention degradation** - Starts at 35% vs 50% for large models
2. **Tool schema bloat** - 64+ tools = ~15K tokens (6% of 256K)
3. **Repetition loops** - Model re-reads files it already has in context
4. **Completed task persistence** - Finished todos still occupy context

---

## 2. Current Omnius Implementation Analysis

### What We Have (Strengths)

```
┌─────────────────────────────────────────────────────────────────┐
│                    CURRENT CONTEXT ARCHITECTURE                  │
├─────────────────────────────────────────────────────────────────┤
│                                                                  │
│  [System Prompt] ─┬─> [Head: preserved]                         │
│                   │                                              │
│  [User Task] ─────┘                                              │
│                                                                  │
│  [Middle Messages] ──> Compaction ──> [Summary Block]           │
│         │                    │                                   │
│         │              ┌─────┴─────┐                             │
│         │              │ Strategies│                             │
│         │              │ - default │                             │
│         │              │ - aggressive                             │
│         │              │ - decisions                             │
│         │              │ - errors   │                             │
│         │              │ - summary  │                             │
│         │              │ - structured                             │
│         │              └───────────┘                             │
│         │                    │                                   │
│         └────────────────────┴──> [Memex Archive]               │
│                                                                  │
│  [Recent Messages] ──> Preserved verbatim (4-12 msgs)           │
│                                                                  │
└─────────────────────────────────────────────────────────────────┘
```

**Compaction Thresholds (tier-aware):**
- Small (≤7B): 65% of context window
- Medium (8-29B): 70% of context window
- Large (≥30B): 75% of context window
- Deep mode: 85% of context window

**Features:**
- ✅ Progressive summarization (AgentFold-inspired)
- ✅ Observation masking (RECOMP-inspired)
- ✅ Goal re-injection after compaction
- ✅ Anti-repetition reminders for small/medium
- ✅ Tool calling reminders for small models
- ✅ Memex archive for large tool outputs
- ✅ SNR (Signal-to-Noise Ratio) tracking
- ✅ Task state preservation through compaction
- ✅ File registry with access counts
- ✅ Sub-agent types with tool restrictions

### What We Lack (Gaps)

```
┌─────────────────────────────────────────────────────────────────┐
│                         IDENTIFIED GAPS                          │
├─────────────────────────────────────────────────────────────────┤
│                                                                  │
│  1. MONOLITHIC CONTEXT                                           │
│     └─> Flat message array, no hierarchical structure            │
│     └─> All context equally weighted                             │
│                                                                  │
│  2. NO TASK-PHASE AWARENESS                                      │
│     └─> Completed todos still in context                         │
│     └─> No dynamic expansion/contraction based on phase          │
│                                                                  │
│  3. TOOL SCHEMA BLOAT                                            │
│     └─> All 64+ tools loaded even when not needed                │
│     └─> ~15K tokens for tool schemas                             │
│                                                                  │
│  4. SUB-AGENT CONTEXT LEAKAGE                                    │
│     └─> Sub-agents inherit parent context unnecessarily          │
│     └─> No clean context boundary for delegation                 │
│                                                                  │
│  5. REACTIVE ONLY                                                │
│     └─> Compaction only at threshold                             │
│     └─> No proactive pruning of completed work                   │
│                                                                  │
│  6. NO ANCHOR SURFACING                                          │
│     └─> Previous task context always present                     │
│     └─> Should surface only when needed                          │
│                                                                  │
└─────────────────────────────────────────────────────────────────┘
```

---

## 3. Proposed Implementation Roadmap

### Phase 1: Task-Phase-Aware Context Tree (PRIORITY: HIGH)

**Problem:** Context is monolithic; completed work persists unnecessarily

**Solution:** Hierarchical context tree with phase-based expansion/contraction

```typescript
interface ContextTree {
  // Root: always present
  root: {
    systemPrompt: string;      // Condensed for medium models
    activeGoal: string;        // Current task
    toolSubset: string[];      // Only relevant tools
  };
  
  // Phase nodes: expand when active, contract when complete
  phases: {
    explore: ContextNode;      // Active during exploration
    plan: ContextNode;         // Active during planning
    implement: ContextNode;    // Active during implementation
    verify: ContextNode;       // Active during verification
  };
  
  // Archive: accessible via memex_retrieve
  archive: {
    completedPhases: string[]; // Hash IDs
    priorTasks: string[];      // Hash IDs
  };
}

interface ContextNode {
  status: "active" | "contracted" | "archived";
  messages: ChatMessage[];
  summary?: string;           // Generated when contracted
  anchors: string[];          // Key facts to preserve
}
```

**Implementation:**

1. **Phase detection** - Analyze recent messages to determine current phase
2. **Automatic contraction** - When phase completes, summarize + archive
3. **Anchor extraction** - Preserve only critical facts from completed phases
4. **Dynamic expansion** - When returning to a phase, restore from archive

**Code locations:**
- `packages/orchestrator/src/agenticRunner.ts:5357` (compactMessages)
- `packages/orchestrator/src/agenticRunner.ts:1143` (contextLimits)

### Phase 2: Tool Schema Lazy Loading (PRIORITY: HIGH)

**Problem:** 64+ tools = ~15K tokens loaded at start

**Solution:** Tier-aware tool subsets with on-demand expansion

```typescript
// Medium model tool subsets
const MEDIUM_CORE_TOOLS = [
  "file_read", "file_write", "file_edit",
  "shell", "grep_search", "find_files",
  "task_complete", "memory_read", "memory_write"
];

const MEDIUM_EXPANDABLE_TOOLS = {
  web: ["web_search", "web_fetch", "web_crawl"],
  code: ["file_patch", "file_explore", "batch_edit"],
  agent: ["agent", "sub_agent", "background_run"],
  memory: ["memory_search", "working_notes"],
  // ... more subsets
};

// On-demand loading via explore_tools
function expandToolSubset(subset: string): void {
  // Add tools to active set
  // Update tool schema in context
}
```

**Implementation:**

1. **Core tool set** - Only 9-12 tools loaded initially for medium models
2. **Subset expansion** - `explore_tools("web")` loads web tools
3. **Schema caching** - Tool schemas cached, not re-sent
4. **Usage tracking** - Auto-unload unused tools after N turns

**Code locations:**
- `packages/execution/src/index.ts` (tool exports)
- `packages/orchestrator/src/agenticRunner.ts:870` (tool schema injection)

### Phase 3: Completed Todo Pruning (PRIORITY: MEDIUM)

**Problem:** Completed todos persist in context, wasting tokens

**Solution:** Automatic pruning with anchor extraction

```typescript
interface TodoState {
  active: Todo[];           // In context
  completed: TodoAnchor[];  // Summarized
  archived: string[];       // Hash IDs in Memex
}

interface TodoAnchor {
  id: string;
  summary: string;          // One-line summary
  keyFiles: string[];       // Files touched
  outcome: "success" | "blocked" | "delegated";
}

// Pruning logic
function pruneCompletedTodos(): void {
  const completed = todos.filter(t => t.status === "completed");
  const anchors = completed.map(t => extractAnchor(t));
  
  // Add anchors to context (compact)
  // Archive full todos to Memex
  // Update todo list
}
```

**Implementation:**

1. **Completion detection** - When todo marked completed
2. **Anchor extraction** - Generate one-line summary + key files
3. **Context update** - Replace full todo with anchor
4. **Memex archive** - Store full todo for retrieval

**Code locations:**
- `packages/execution/src/tools/todo.ts`
- `packages/orchestrator/src/agenticRunner.ts:5554` (formatTaskState)

### Phase 4: Sub-Agent Context Isolation (PRIORITY: MEDIUM)

**Problem:** Sub-agents inherit parent context, causing bloat

**Solution:** Clean context boundary with explicit handoff

```typescript
interface SubAgentContext {
  // Minimal context for sub-agent
  task: string;              // Delegated task only
  relevantFiles: string[];   // Only files needed
  toolSubset: string[];      // Only tools needed
  
  // NOT included
  // - Parent conversation history
  // - Completed todos
  // - Other sub-agent results
}

interface HandoffProtocol {
  // Parent → Sub-agent
  handoff: {
    task: string;
    files: FileContent[];    // Pre-loaded files
    constraints: string[];   // Rules to follow
  };
  
  // Sub-agent → Parent
  return: {
    summary: string;         // What was done
    filesModified: string[]; // Files changed
    artifacts: string[];     // Memex IDs for results
  };
}
```

**Implementation:**

1. **Context stripping** - Remove parent context before spawn
2. **File pre-loading** - Only relevant files passed
3. **Result summarization** - Sub-agent returns summary, not full context
4. **Parent integration** - Summary added to parent context

**Code locations:**
- `packages/execution/src/tools/agent-tool.ts`
- `packages/orchestrator/src/agent-types.ts`

### Phase 5: Proactive Context Pruning (PRIORITY: LOW)

**Problem:** Compaction only at threshold, no proactive cleanup

**Solution:** Background pruning of low-value context

```typescript
interface PruningRules {
  // Auto-prune after N turns
  duplicateToolCalls: { maxOccurrences: 2 };
  oldFileReads: { maxAge: 10 };  // Turns
  successfulTests: { keepSummary: true };
  
  // Never prune
  errors: { preserve: true };
  decisions: { preserve: true };
  activeFiles: { preserve: true };
}

// Background pruning (runs every 5 turns)
function proactivePrune(): void {
  // Remove duplicate tool calls
  // Summarize old file reads
  // Archive successful test runs
  // Update SNR
}
```

**Implementation:**

1. **Turn counter** - Track message age
2. **Value scoring** - Score each message for relevance
3. **Background pruning** - Run every N turns
4. **SNR update** - Recalculate after pruning

**Code locations:**
- `packages/orchestrator/src/agenticRunner.ts:5357` (compactMessages)

### Phase 6: Anchor Surfacing (PRIORITY: LOW)

**Problem:** Previous task context always present

**Solution:** Demand-driven anchor retrieval

```typescript
interface AnchorStore {
  // Lightweight anchors always present
  anchors: Map<string, Anchor>;
  
  // Full context retrieved on demand
  archive: Map<string, string>;  // Memex IDs
}

interface Anchor {
  id: string;
  type: "file" | "decision" | "error" | "task";
  summary: string;           // One line
  keywords: string[];        // For retrieval
  memexId?: string;          // Full context
}

// Retrieval triggered by:
// - Keyword match in current task
// - File path reference
// - Explicit memex_retrieve call
```

**Implementation:**

1. **Anchor extraction** - During compaction, extract anchors
2. **Keyword indexing** - Index anchors by keywords
3. **Demand retrieval** - Surface when keywords match
4. **Context injection** - Add retrieved context to recent messages

**Code locations:**
- `packages/orchestrator/src/agenticRunner.ts:5561` (formatFileRegistry)
- `packages/memory/src/episodeStore.ts` (semantic search)

---

## 4. Implementation Priority Matrix

| Phase | Impact | Effort | Priority | Dependencies |
|-------|--------|--------|----------|--------------|
| Phase 1: Context Tree | HIGH | HIGH | P0 | None |
| Phase 2: Tool Lazy Loading | HIGH | MEDIUM | P0 | None |
| Phase 3: Todo Pruning | MEDIUM | LOW | P1 | Phase 1 |
| Phase 4: Sub-Agent Isolation | MEDIUM | MEDIUM | P1 | Phase 1 |
| Phase 5: Proactive Pruning | LOW | MEDIUM | P2 | Phase 1 |
| Phase 6: Anchor Surfacing | LOW | HIGH | P2 | Phase 1, Phase 3 |

**Recommended Order:**
1. **Phase 2** (Tool Lazy Loading) - Quick win, immediate token savings
2. **Phase 1** (Context Tree) - Foundation for all other phases
3. **Phase 3** (Todo Pruning) - Builds on Phase 1
4. **Phase 4** (Sub-Agent Isolation) - Independent, medium effort
5. **Phase 5** (Proactive Pruning) - Enhancement
6. **Phase 6** (Anchor Surfacing) - Enhancement

---

## 5. Expected Outcomes

### Token Savings (Estimated)

| Component | Before | After | Savings |
|-----------|--------|-------|---------|
| Tool schemas | ~15K | ~3K | 80% |
| Completed todos | ~5K | ~500 | 90% |
| Old file reads | ~10K | ~2K | 80% |
| Sub-agent context | ~8K | ~2K | 75% |
| **Total** | ~38K | ~7.5K | **80%** |

### Repetition Loop Reduction

- **Before:** 35% repetition rate at 35% context fill
- **After:** <10% repetition rate at 70% context fill

### Context Utilization

- **Before:** Monolithic, degrades at 35%
- **After:** Hierarchical, maintains quality to 70%

---

## 6. Research References

1. Lost in the Middle: https://arxiv.org/abs/2307.03172
2. RECOMP: https://arxiv.org/abs/2310.04408
3. AgentFold: https://arxiv.org/abs/2510.24699
4. ARC: https://arxiv.org/abs/2601.12030
5. NATURAL PLAN: https://arxiv.org/abs/2406.04520
6. ToolLLM DFSDT: https://arxiv.org/abs/2307.16789
7. SPRINT: https://arxiv.org/abs/2506.05745
8. Recursive LMs: https://arxiv.org/abs/2512.24601
9. MASS (multi-agent topology): https://arxiv.org/abs/2502.11578
10. ExpeL (experience learning): https://arxiv.org/abs/2308.10144

---

## 7. Next Steps

1. **Review this proposal** with team
2. **Prototype Phase 2** (Tool Lazy Loading) - 2-3 days
3. **Design Phase 1** (Context Tree) - 1 week
4. **Implement Phase 1** - 2-3 weeks
5. **Iterate on remaining phases** based on learnings

---

*Generated by Omnius — Context Management Deep Dive (2026-04-25)*