--- title: "Durable Resume" description: "When a hosted agent run is interrupted and resumes in the same turn, a tool-call journal derived from the durable ledger blocks exact repeats of write calls that already completed." --- # Durable Resume **Who is this for:** anyone who wants to understand how the framework's run recovery avoids duplicate side effects. This is built-in behavior — there is nothing to wire up. Hosted agent runs get interrupted: a serverless function hits its hard timeout mid-stream, a gateway drops the connection at 45s, a socket hangs up, the platform cold-starts. The framework already recovers from these by saving the conversation prefix and re-running the LLM call ("continue from where you left off"). But recovery alone has a sharp edge: if the interrupted attempt **already sent an email or created a ticket**, a naive resume could do it again. Durable resume closes that gap. On resume, the framework knows which side-effecting tool calls already completed and refuses to re-run them — at two layers. ```html
Run-event ledger
tool_start / tool_done
Tool-call journalcompleted = start+doneinterrupted = start, no done
Layer 1 · prompt note → model
Layer 2 · hard-block re-dispatched write
``` ```css .diagram-durable { display: flex; align-items: center; gap: 14px; flex-wrap: wrap; } .diagram-durable .diagram-col { display: flex; flex-direction: column; gap: 8px; } .diagram-durable .diagram-arrow { font-size: 22px; line-height: 1; } .diagram-durable .center { display: flex; flex-direction: column; align-items: center; gap: 4px; } ``` ## The tool-call journal {#journal} The journal is a **pure read over the durable run-event ledger** — there is no new recording hook in the hot path. It classifies the tool calls already recorded for the current turn: - **Completed** — a `tool_start` with a matching `tool_done`. The call ran, its side effect happened, and its result was recorded. **Do not re-run.** - **Interrupted** — a `tool_start` with **no** matching `tool_done`. The call began, its side effect may or may not have landed, and the interruption ate the result. Outcome unknown. Matching mirrors how durable turns are rebuilt elsewhere: a `tool_done` pairs with the oldest still-open `tool_start` for the same tool name (FIFO per tool). A `clear` event (discarded partial output) resets the per-turn tally so abandoned partials don't leave phantom open calls. ## Layer 1: prompt-level journal note {#prompt-note} When a run resumes (soft timeout, gateway timeout, or any resumable transport error), the framework appends a **structured journal note** to the resume prompt, right after the "continue from where you left off" nudge. The note tells the model, in plain text: - which tool calls **already completed** (with short results) so it reuses them and does **not** re-run them, and - which tool calls were **interrupted with unknown outcome** so it verifies state before assuming success or failure. When the journal is empty (a turn with no tool activity, or a clean continuation), nothing extra is appended and resume behavior is byte-for-byte what it was before. The note is best-effort: a failed ledger read never blocks a recovery that would otherwise succeed. ## Layer 2: tool-layer hard-block {#hard-block} The prompt note is advisory — a well-behaved model heeds it, but a model isn't a guarantee. So the loop also enforces it at the tool layer. Before the loop runs in a resumed chunk, it snapshots the journal once (capturing only **prior** chunks of this logical turn). When the model re-dispatches a **write** tool whose tool name **and input** match a completed journal entry, the loop short-circuits: it returns the journaled result instead of executing the action, with a note that the call already completed in an earlier interrupted attempt and was not re-run to avoid a duplicate side effect. Key properties: - **Write tools only.** Read-only (`readOnly` / GET) actions are never blocked — re-reading is safe and idempotent. - **Content-addressed.** The match is on tool name + input signature, so a resumed call sitting at a different position in the turn still matches; a _different_ call (different args) is treated as fresh and runs normally. - **Consume-once.** Each completed entry is claimed when matched, so two genuinely-distinct identical fresh calls in the same turn don't both short-circuit on one journaled completion. - **Fresh calls untouched.** A first-turn call sees an empty journal; nothing changes for normal runs. The hard-block stops only an **exact repeat**: the same write tool with the same input, after an earlier attempt in the same turn finished and recorded its result. It does not rule out every duplicate email, charge, or ticket: - **A reworded retry isn't matched.** If the model changes any argument, such as the email body, the call no longer matches a completed call, so the hard-block doesn't stop it. - **A crash mid-send can repeat.** If the process dies after the side effect lands but before its result is recorded, the call counts as interrupted, and the hard-block covers only completed calls. A separate guard refuses an identical retry of an interrupted write, but only when the resumed run's history shows the interruption. - **A crashed run can resume in the same turn through either of two separate paths.** One is a [durable background run](/docs/durable-background-runs). The other is a later chunk of a normal foreground turn using the [foreground self-chain](/docs/durable-background-runs#foreground-self-chain). Any other crashed run ends with an error. Choosing **Continue** on it resumes the same turn, so both layers apply. An action whose side effect must never repeat needs its own idempotency check. The hard-block never applies to read-only actions. ## Automatic continuation after a time limit {#auto-continue} When the server stops a turn at its run time limit and nothing server-side picks it up, the chat continues it on its own, in the **same turn**. The resumed run gets the thread's tool calls and results plus the journal note above, so finished steps are reused, and an identical repeat of a write — including a cross-app `call-agent` delegation, whose idempotency key is derived from the turn — returns the recorded result instead of being sent again. The chat shows a _Resuming_ status, and Stop keeps working. A turn continues automatically **at most three times**. The count is stored with the turn's runs, so reloading the page does not reset it, and a second tab cannot double it. After that, or after any error, stop, cancellation, or credential or rate limit, the turn ends with **Continue** available and waits for you. Choosing **Continue** resumes the stopped run's own turn the same way, with no limit on how often, so both layers apply to it as they do to an automatic continuation. It is offered only for the conversation's newest stop. If you send a new message instead, the agent is first told what the unfinished turn already did, but neither layer blocks a repeat. ## What's next - [**Real-Time Sync**](/docs/real-time-collaboration) — how the durable run ledger streams to the client and replays on reconnect. - [**Actions**](/docs/actions-overview) — `readOnly` marks reads as safe to re-run; everything else is treated as side-effecting. - [**In-Loop Processors**](/docs/processors) — another loop-internal hardening seam. - [**Durable Background Runs**](/docs/durable-background-runs) — moving a long turn onto a separate background function.