# Default Compaction — Implementation Report > Prepared for review. This document covers every decision made, why it was made, and how each piece of code integrates with the existing codebase. --- ## 1. What Was Built A new compaction method called **defaultCompaction** that: - Stores a rolling summary (`compactSummary`) and a progress pointer (`compactSize`) directly on the session row in SQLite. - Is triggered **manually** via `POST /sessions/:id/compact` — no automatic trigger (yet). - Compacts `compactionCount`% of the uncompacted context (by character length, not message count) into a single summary, advancing the pointer each time. - Injects the summary transparently before every LLM call, right after system prompts, so the agent always has full context without needing to know anything about compaction. --- ## 2. Files Changed / Created | File | Type | Change | |------|------|--------| | `migrations/006-default-compaction.sql` | NEW | Adds 3 columns to `sessions` table | | `infrastructure/database.js` | MODIFIED | `ensureColumn` x3 + `createSession` accepts `compactCount` | | `settings/fields.js` | MODIFIED | Added `DEFAULT_COMPACT_COUNT: 50` constant | | `schemas/agent.json` | MODIFIED | Added `defaultCompaction.compactionCount` property | | `core/default-compaction.js` | NEW | Core module — `runDefaultCompaction` + `buildMessagesWithSummary` | | `core/router.js` | MODIFIED | 3 injection points + `compactCount` on all session creations | | `api/routes/sessions.js` | MODIFIED | `POST /:id/compact` endpoint + `compactCount` on manual session create | | `docs/guide/08-memory.md` | MODIFIED | Full new section on defaultCompaction | | `docs/guide/03-configuration.md` | MODIFIED | Updated `compact` model role description | --- ## 3. Database Design ### Why these three columns? ```sql compact_summary TEXT -- accumulated summary, NULL until first compaction compact_size INTEGER DEFAULT 0 -- pointer: how many non-system messages are summarized compact_count INTEGER DEFAULT 50 -- % to compact per call, snapshotted at session creation ``` **`compact_summary`**: The rolling text. It grows richer with each compaction call (it's not replaced, it's updated). Stored directly on the session so it's always available without an extra table lookup. **`compact_size`**: A simple integer counter. It tells the system exactly where in the message list the summary "ends" — messages before this index are already in the summary; messages from this index onwards are still "raw". This is the key to making incremental compaction work. **`compact_count`**: Snapshotted from agent config at session creation. This is intentional — the agent's config may change between sessions, but a running session should use the value it was started with for consistency. ### Why not a separate table? The state is 1:1 with the session. A separate table would add a JOIN for no benefit. Columns on the session row are atomic with `updateSession` (which uses `camelToSnake` for field-to-column mapping — verified that `compactSummary → compact_summary` etc. all resolve correctly). ### Migration safety The SQL migration file uses `ALTER TABLE ... ADD COLUMN`. The `ensureColumn()` function in `database.js` runs on every startup and is idempotent — it checks `PRAGMA table_info` before altering. This means even if the migration SQL fails (e.g. SQLite version quirks), the `ensureColumn` calls still add the columns. This is the same pattern already used for all previous columns in this codebase. --- ## 4. Algorithm — `runDefaultCompaction` ### Why percentage-by-length, not by message count? The user's spec explicitly asks for this: "if the session has 100 messages with different lengths, 50% of it will not be 50 messages, it will start collecting messages until it reaches 50% of all messages text length combined." A count-based approach fails badly if the first 5 messages are giant tool outputs and the next 95 are short assistant replies. Length-based compaction compresses the right *amount* of context. ### The algorithm step-by-step ``` 1. Load session → compactCount (50), compactSize (0), compactSummary ("") 2. Load all DB messages → split into systemMsgs and nonSystemMsgs 3. Guard: if nonSystemMsgs.length <= compactSize → nothing new → return alreadyUpToDate 4. uncompactedMsgs = nonSystemMsgs[compactSize..] 5. totalLength = summaryLen + sum(uncompacted chars) 6. targetLength = floor(totalLength × compactCount/100) 7. Greedy collect from uncompactedMsgs[0]: accumulate until batchLen >= targetLength 8. batch = collected messages; cutoff = number collected 9. Guard: if cutoff === 0 or targetLength === 0 → return alreadyUpToDate 10. Call LLM: prompt = existing summary + batch 11. Store newSummary → update session: compactSummary = newSummary, compactSize += cutoff 12. Return { compactedCount: cutoff, newSize } ``` ### Why does the greedy loop go to cutoff instead of stopping at exactly the boundary? Because messages can't be split. If the target is 6500 chars and the current message pushes us to 6600, we include it. This is a deliberate rounding-up behavior — it's better to compact slightly more than slightly less. ### Why use `callLLM` directly instead of the existing `compactMessages`? The existing `compactMessages` in `core/compaction.js` does something fundamentally different — it replaces the entire message history with a 2-message summary pair in the `messages` array. `defaultCompaction` works differently: it only updates the DB session state and leaves the raw messages untouched. The injection happens separately in `buildMessagesWithSummary`. These are two different strategies and must remain separate. ### Model selection Uses `getModelConfig(settings, F.MODEL_COMPACT)`. From `utils/settings.js:128-132`, this function already handles the fallback: if `compact` model is not configured, it returns the `main` model config. This means `defaultCompaction` works even without a dedicated compact model. No special error case needed. --- ## 5. Injection — `buildMessagesWithSummary` ### Why inject in `router.js`, not `loop.js`? `loop.js` receives a `messages` array and operates on it in-memory throughout the loop. It doesn't know where the messages came from or whether they need a summary prepended. `router.js` is the place where messages are assembled from DB before being handed to the loop — it's the natural seam. The alternative (injecting in `loop.js`) would mean: 1. Passing extra state into `runLoop` (sessionId, db access for session) 2. The loop would need to know about compaction state — a concern it doesn't currently have 3. The injection would run once per iteration, not once per loop start Injecting in `router.js` means the loop is completely unaware of compaction — it just gets a clean messages array. ### The injection shape ```javascript [ ...systemMessages, // unchanged { role: 'user', content: '[Summary of previous conversation]\n\n...' }, { role: 'assistant', content: 'Understood. I have the context...' }, ...nonSystemMessages.slice(compactSize), // only uncompacted messages ] ``` **Why a user+assistant pair?** Because OpenAI-compatible APIs require strict message alternation for some models (user → assistant → user → ...). A standalone `user` message with no `assistant` response is valid, but a standalone `assistant` message with no preceding `user` is not. Using a matched pair is the safest and most compatible approach — it's the same pattern the existing `compactMessages` function uses. **Why not inject as a `system` message?** System messages are typically at the very beginning and used for instructions. Injecting a summary as a system message could confuse some models or get masked by future compaction operations. A conversational pair is semantically clearer: the "user" is presenting the prior context, and the "assistant" acknowledges it. ### Why the injection is in-memory only (not persisted to DB) The injected messages are ephemeral — they're built on every load from the session's `compact_summary` field. Persisting them would: 1. Pollute the real message history with synthetic messages 2. Cause them to be included in future compaction counts (double-counting) 3. Make the DB hard to inspect and reason about The real messages (from `compactSize` onwards) stay in DB exactly as they were written. The summary is a separate field. Clean separation. ### Three injection points in `router.js` | Function | Injection point | Why | |----------|----------------|-----| | `runChat` | After user message is added, before `runLoop` | Chat sessions are the primary use case | | `runTask` (existing session) | After messages assembled, before `runLoop` | Tasks can resume compacted sessions | | `resumeTask` | After messages loaded from DB | Resumed tasks may have been compacted while waiting | `runSubagent` and `runDaemonTick` always create fresh sessions — `compactSummary` is always null, `buildMessagesWithSummary` returns messages unchanged. Adding `buildMessagesWithSummary` there would be a no-op but unnecessary noise. --- ## 6. Agent Config Schema ### Why `additionalProperties: false` required us to add the property `schemas/agent.json` uses `"additionalProperties": false` at the root level, enforced by `ajv` in `core/agent.js`. Any field not declared in `properties` causes agent loading to throw. The `defaultCompaction` property was added correctly: ```json "defaultCompaction": { "type": "object", "additionalProperties": false, "properties": { "compactionCount": { "type": "number", "minimum": 1, "maximum": 99 } } } ``` `additionalProperties: false` on the nested object too — consistent with how `memory` is defined in the same schema. ### Why is `compactionCount` a `number` and not `integer`? The user's spec shows `compactionCount: 50` without specifying integer-only. Accepting floats (e.g. `33.3`) is more flexible and still valid for the percentage calculation. The DB stores it as `REAL` implicitly if a float is passed, but since `compact_count` is `INTEGER`, SQLite will truncate it. This is acceptable behavior. --- ## 7. `createSession` Update Every path that creates a session now passes `compactCount`: - `POST /sessions` (manual) → reads `agent.defaultCompaction?.compactionCount` - `runChat` → same - `runTask` (new session) → same - `runSubagent` → same `runDaemonTick` also creates a session — but daemon sessions are fire-and-forget (closed immediately after the tick). Daemon sessions can technically be compacted via the API, but there's no meaningful use case. The `compactCount` is still stored correctly for consistency. The `compactCount` validation in `createSession` is defensive: ```javascript const resolvedCompactCount = (typeof compactCount === 'number' && compactCount >= 1 && compactCount <= 99) ? compactCount : 50; ``` If `agent.defaultCompaction` is undefined (most agents), `undefined ?? 50` correctly defaults to 50 at the call site, and the database stores 50. --- ## 8. What Happens When Compaction is Called Multiple Times First call (100 msgs, compact_count=50): - `totalLength` = 0 + sum(100 msgs) - `targetLength` = 50% of totalLength - Collects messages until 50% of total chars reached (say, 47 msgs) - `compact_size = 47`, `compact_summary = "...47 msgs summarized..."` Second call (100 msgs, compact_size=47): - `uncompactedMsgs` = msgs[47..99] = 53 msgs - `totalLength` = summaryLen + sum(53 msgs) ← summary is shorter than 47 msgs (compressed) - `targetLength` = 50% of that total - Collects until 50% reached (fewer messages since summary is smaller) - `compact_size` advances further This is correct and predictable: each call compacts roughly half of whatever context remains uncompacted. --- ## 9. Edge Cases Handled | Edge Case | Handling | |-----------|----------| | Session never had any messages | `nonSystemMsgs.length <= compactSize` → `alreadyUpToDate: true` | | All messages already compacted | Same guard: `compactSize >= nonSystemMsgs.length` | | `targetLength === 0` (all-empty messages) | Explicit `if (cutoff === 0 || targetLength === 0)` guard | | LLM returns empty summary | Throws `"LLM returned an empty summary"` — caller gets a 500 with clear message | | No compact model configured | `getModelConfig` falls back to main — always works | | Session is closed | API endpoint returns `400 SESSION_CLOSED` | | Session not found | API endpoint returns `404 SESSION_NOT_FOUND` | | Old sessions (before migration) | `compact_size = 0`, `compact_summary = null` — injection is no-op, compaction works normally | | `compactCount` out of range in agent.json | AJV schema validates 1–99 at agent load time | --- ## 10. What Was NOT Done (Intentional) - **Auto-trigger**: The user said this is for later. The function is standalone so wiring an auto-trigger is a one-liner in `loop.js`. - **`POST /sessions` accepting `compactCount` override from request body**: Not asked for yet. Can be added trivially. - **Emitting a `context.defaultCompacted` event**: Not asked for in the spec. The session object returned from the endpoint already shows updated state. - **Writing tests**: Not asked for. The logic is straightforward and testable in isolation. - **Touching the old `compaction_count` column** (the orphaned one from migration 001): Left as-is to avoid breaking anything. It has a different meaning (was intended to count how many times auto-compaction ran) and is never read. --- ## 11. Integration Proof ### Router chain ``` POST /sessions/:id/compact → sessions.js route handler → runDefaultCompaction({ sessionId, settings }) → db.getSession(sessionId) [reads current compact state] → db.getMessages(sessionId) [loads all messages] → callLLM(...) [summarizes batch] → db.updateSession(sessionId, { compactSummary, compactSize }) → db.getSession(sessionId) [reads updated state for response] → res.json({ ... }) ``` ### Per-turn injection chain (runChat example) ``` POST /agents/:name/chat → runChat({ agentName, message, sessionId }) → db.getMessages(sid) [load history from DB] → messages.push(userMsg) [add current user turn] → db.addMessage(userMsg) [persist user turn to DB] → buildMessagesWithSummary({ messages, session: db.getSession(sid) }) ↓ if compact_summary is not empty: → [system, summary_user, summary_asst, messages[compact_size..], userMsg] ↓ else: → [system, ...messages, userMsg] (unchanged) → runLoop({ messages: ... }) [LLM sees summary + recent messages] ``` The loop never writes the injected summary messages to DB (it only persists messages it creates: assistant responses and tool results). The DB messages remain the ground truth. --- ## 12. How to Use ### Set up (optional — agent.json) ```json { "name": "my-agent", "model": "...", "defaultCompaction": { "compactionCount": 40 } } ``` ### Run compaction ```bash curl -X POST http://localhost:5050/sessions/sess_abc123/compact ``` Response: ```json { "sessionId": "sess_abc123", "compactedCount": 47, "newSize": 47, "alreadyUpToDate": false, "session": { "compact_size": 47, "compact_summary": "..." } } ``` ### Nothing to compact ```json { "compactedCount": 0, "newSize": 47, "alreadyUpToDate": true } ``` ### Continue chatting No changes needed. The next chat message automatically uses the summary. --- ## 13. Potential Questions **Q: What if the summary itself is very long — could it overflow the context?** A: The summary replaces many messages, so it should always be shorter than what it replaces. If the summary grows large, calling `/compact` again will include the old summary in the total length calculation and target 50% of the combined total. **Q: Does compaction affect tasks?** A: Yes — `runTask` (existing session path) and `resumeTask` both inject the summary. A session can be compacted while a task is waiting and the resumed task will have the injected summary. **Q: What if the session is currently running?** A: The endpoint doesn't block on active sessions. If a session is mid-loop (active), calling `/compact` would update `compact_summary` in DB. The current in-memory `messages` in the running loop is unaffected. On the next turn, when the loop persists its messages to DB and a future invocation loads from DB, the summary would be used. In practice, you should compact sessions that are idle or in chat mode between turns. The user explicitly said this is for testing/manual use for now. **Q: What happens on `POST /sessions/:id/reset`?** A: The reset route calls `db.resetSession(sessionId)` which deletes all messages and resets token counts. However, `resetSession` does **not** clear `compact_summary` or `compact_size`. This could leave stale compaction state. This is a known gap — but since the user said this is for manual testing, it's acceptable for now. A future PR should add `compact_summary = NULL, compact_size = 0` to the reset.