# Memory & Context Compaction

VeilCLI provides two complementary systems for managing long-running agent knowledge: **persistent memory files** for cross-session recall, and **context compaction** for keeping LLM context windows healthy during long tasks.

---

## Persistent Memory

Memory files are plain Markdown files that persist across sessions. They are injected into the agent's system prompt at the start of each conversation.

### Memory scopes

| Scope | File path | Who writes it | Who reads it |
|-------|-----------|--------------|--------------|
| `agent` | `.veil/memory/agents/<name>/MEMORY.md` | The agent itself | Only that agent |
| `global` | `.veil/memory/MEMORY.md` | Any agent | All agents with memory enabled |

### Writing memory

Agents use the `memory_write` tool:

```
"Remember that the project uses port 5051"
→ memory_write({ content: "Project uses port 5051.", scope: "agent" })
```

Each entry is appended with a date comment:
```markdown
<!-- 2025-03-02 -->
Project uses port 5051.
```

**Cleanup guards.** The `memory_write` tool returns one of three statuses:

| Status | Trigger |
|--------|---------|
| `written` (default) | New entry appended successfully. |
| `skipped_empty` | `content` was null, an empty string, or whitespace-only — nothing was written. |
| `skipped_duplicate` | After trimming, the new entry is byte-identical to the **last** existing entry in the file. The append is dropped to avoid noisy duplication. Substring overlaps and partial matches are NOT considered duplicates — only an exact re-append. |

These guards live on the tool path. The HTTP endpoints `PUT /memory/:file` and `PUT /agents/:name/memory/:file` perform a literal overwrite without applying these guards — see [Memory API → tool/HTTP asymmetry](../api/07-memory.md#http-put-vs-the-memory_write-tool).

### Reading memory

- **Automatic injection**: If `memory.enabled` is `true` in the agent config (or settings), the memory file is read and included in the system prompt at session start.
- **Manual read**: Use `memory_read` to explicitly read memory mid-task.
- **Search**: Use `memory_search` to find relevant entries by keyword.

### Configuring memory

In `agent.json`:
```json
{
  "memory": {
    "enabled": true,
    "maxLines": 300
  }
}
```

In `settings.json` (applies globally to all agents):
```json
{
  "memory": {
    "enabled": true,
    "maxLines": 500
  }
}
```

`maxLines` caps the length of `MEMORY.md` on disk. When a write pushes the file past the cap, the **oldest whole entries** are moved to a dated `archive-YYYY-MM.md` file in the same directory (entries are never split mid-body). With `enabled: false`, `memory_write` and the pre-compaction memory extractor are both disabled.

### Memory best practices

- Write focused, factual entries — avoid vague notes like "talked about stuff"
- Use global memory for facts shared across agents (e.g. project conventions, API endpoints)
- Use agent memory for agent-specific state (e.g. "already processed file X")
- Periodically use `memory_search` before writing to avoid duplicates

---

## Context Compaction

LLM context windows are finite. During long tasks with many tool calls, the conversation history can exceed the model's context limit. VeilCLI manages this automatically through **context compaction**.

### How it works

1. **Observation masking**: Tool result messages older than `observationMaskingTurns` turns are replaced with `[observation masked]`. The tool call itself remains, but the full output is hidden. This is the first and cheapest form of compression.

2. **Full compaction**: If the estimated token count exceeds `threshold × model_context_limit`, the runtime:
   - Sends the current conversation to an LLM (using the `compact` model role if configured, otherwise `main`) with a special prompt asking for a structured summary
   - Replaces the full message history with the summary plus the most recent turns
   - Resumes the turn with the compressed context

### Configuration

In `settings.json`:
```json
{
  "compaction": {
    "threshold": 0.8,
    "observationMaskingTurns": 10
  }
}
```

| Field | Default | Description |
|-------|---------|-------------|
| `threshold` | `0.85` | Fraction of context window that triggers full compaction (0.1–1.0) |
| `observationMaskingTurns` | `10` | Keep tool results visible for N most recent turns; mask older ones |

### Using a dedicated compact model

For cost efficiency, use a cheaper/faster model for compaction. Set the compact model role in settings:

```json
{
  "models": {
    "main": { "model": "anthropic/claude-4-6-sonnet" },
    "compact": { "model": "google/gemini-flash-1.5-8b" }
  }
}
```

The `compact` role falls back to `main` if not set. Both use the provider resolved from `routing` config. Provider credentials go in `auth.json`:

```json
{
  "providers": {
    "openrouter": {
      "type": "openai",
      "base_url": "https://openrouter.ai/api/v1",
      "api_key": "sk-or-v1-..."
    }
  },
  "routing": {
    "default": "openrouter",
    "fallback": []
  }
}
```

### Memory extraction during compaction

Before compacting, VeilCLI extracts important facts from the conversation and writes them to the agent's memory file (if memory is enabled). This means the agent "remembers" key decisions even after the context is compressed.

---

## Default Compaction (Manual / On-Demand)

In addition to the automatic compaction above, VeilCLI includes a simpler **defaultCompaction** method that you invoke explicitly. It is designed for chat sessions where you want full control over when the context is compressed.

### How it works

1. The session tracks two pieces of state: a running `compactSummary` text and a `compactSize` counter (how many non-system messages have been summarized so far).
2. When you call `POST /sessions/:id/compact`, the runtime collects the next batch of messages — starting after `compactSize` — until their combined character length reaches `compactionCount`% of the total uncompacted context.
3. Those messages (plus the existing summary) are sent to the LLM, which produces an updated summary.
4. The session stores the new summary and advances `compactSize` to cover the newly compacted messages.
5. On every subsequent LLM turn, the summary is automatically injected right after the system prompt so the model always has full context. Only the messages from `compactSize` onwards are sent — the rest are replaced by the summary.

### Example

Session has 100 equal-length messages, `compactionCount = 50` (the default):

- **First `POST /sessions/:id/compact`**: collects the first 50 messages (50% of total), summarizes them → `compactSize = 50`, `compactSummary = "..."`.
- **Second call**: starting from message 50, calculates 50% of `(summaryLength + remaining 50 messages)` and collects that batch → `compactSize` advances further.
- Messages from position 0 to `compactSize - 1` are never sent to the LLM again; only the summary stands in for them.

### Configuration in `agent.json`

```json
{
  "defaultCompaction": {
    "enabled": true,
    "compactionCount": 50,
    "autoThreshold": 80,
    "model": "default",
    "customInstructions": "Always preserve the active task list and any pending decisions."
  }
}
```

| Field | Type | Default | Description |
|-------|------|---------|-------------|
| `enabled` | boolean | `true` | Enable or disable all compaction for this agent. When `false`, both auto-compact and `POST /sessions/:id/compact` are disabled. |
| `compactionCount` | number (1–99) | `50` | Percentage of uncompacted context to summarize per compaction run |
| `autoThreshold` | number (1–99) | — | Auto-compact when `context_size / context_size_limit × 100` reaches this percentage. Omit to disable auto-compaction. |
| `model` | string | `"default"` | Model to use for the compaction LLM call. `"default"` (or omitting) uses the same model as the session. Set to a cheaper model (e.g. `"google/gemini-flash-1.5-8b"`) to reduce cost. |
| `customInstructions` | string | — | Additional instructions appended to the compaction system prompt. Use this to preserve domain-specific context (e.g. active task lists, decision logs, running state). |

All values are snapshotted into the session at creation time — changing agent config does not affect existing sessions.

### Auto-compaction

If `autoThreshold` is set, the runtime checks after every LLM loop iteration whether the session's context usage (`context_size / context_size_limit × 100`) has reached the threshold. When it has, compaction runs automatically — no manual call needed.

```json
{
  "defaultCompaction": {
    "autoThreshold": 75
  }
}
```

This is the recommended setup for long-running chat sessions.

### API

```
POST /sessions/:id/compact
```

**Response:**
```json
{
  "sessionId": "sess_...",
  "compactedCount": 47,
  "newSize": 47,
  "alreadyUpToDate": false,
  "session": { ... }
}
```

| Field | Description |
|-------|-------------|
| `compactedCount` | Number of messages included in this compaction run |
| `newSize` | Total messages summarized so far (the new `compact_size`) |
| `alreadyUpToDate` | `true` if there were no new messages to compact |

**When `alreadyUpToDate` is `true`**, the session is already fully up-to-date — calling again does nothing until new messages arrive.

**Error responses**

| Code | Condition |
|------|-----------|
| `400 COMPACTION_DISABLED` | `defaultCompaction.enabled` is `false` for this agent |
| `400 SESSION_CLOSED` | Session is already closed |
| `404 SESSION_NOT_FOUND` | No session with that ID |

### Model resolution

The model used for the compaction LLM call is resolved in this order:

1. `defaultCompaction.model` in `agent.json` — if set and not `"default"`
2. The session's own model (snapshotted at creation from `agent.json`)
3. The global `main` model from `auth.json` / `settings.json`

The global `compact` model role from `auth.json` is **not** used for default compaction — it is only used by the auto compaction system. To use a cheaper model for default compaction, set `defaultCompaction.model` explicitly in `agent.json`.

---

## Memory vs Compaction

| Feature | Memory | Auto Compaction | Default Compaction |
|---------|--------|-----------------|-------------------|
| **Scope** | Cross-session | Within a session | Within a session |
| **Trigger** | Agent calls `memory_write` | Automatic at context threshold | Manual (`POST /sessions/:id/compact`) or automatic (via `autoThreshold`) |
| **Storage** | Markdown files on disk | Replaced messages in DB | `compact_summary` + `compact_size` on session |
| **Purpose** | Long-term recall across runs | Keep long tasks running | Chat sessions, with optional auto-trigger |
| **Controllable** | Yes (agent-side) | No | Yes (caller-side or per-agent config) |

Use **memory** to remember things between runs. Rely on **auto compaction** to handle long autonomous tasks gracefully. Use **default compaction** for chat sessions — with `autoThreshold` it can also run automatically, giving you the same hands-off experience with more predictable summarization behaviour.