# Durable Message Queue This server module owns a separate `message-queue.sqlite` database. The default path is `/message-queue.sqlite`; an explicit `null` disables it. Its runtime creates the attachment store beside the database at `message-queue-attachments`, an OpenCode adapter, and a paused worker. Schema v5 uses WAL, foreign keys, a busy timeout, and FULL synchronous writes. Initialization protects newer schemas with `message_queue_unsupported_schema`, protects unmarked queue databases with `message_queue_invalid_schema`, and migrates v1 through v5 inside immediate transactions. The additive migration retains v1 tables and rows, adds dispatch/reconciliation fields, activation metadata, import staging tables, and a delivery target. Every item persists `deliveryTarget`. Missing legacy targets migrate to `{ kind: 'primary' }`. v4 Assistant targets migrate valid OpenCode text/file `parts` and attachment files into `deliveryParts`, preserving part order and appending converted attachment files; malformed targets settle as `failed` with `malformed_target`. Public item DTOs expose Assistant `deliveryParts` at the top level and expose `deliveryTarget` as `{ kind: 'assistant', assistantID }`; captured binding, provider, model, agent, system, and related delivery configuration remain server-private. Assistant admission accepts strict `{ type: 'text', text }` and `{ type: 'file', mime, attachmentID }` parts. Every file ID binds uniquely to one canonical queue attachment occurrence; synthetic sidecar file indexes align with sidecar attachment IDs. v4 migration retains legacy `{ type: 'file', mime, url }` parts. The worker materializes canonical attachment IDs into OpenCode file URLs immediately before `beginAttempt`, preserving one captured durable delivery object across retries and edits. Delivery accepts up to 64 file parts and 129 total parts under a 70 MiB aggregate validation budget. An optional `syntheticParts` edit sidecar preserves non-derivable context by stable part ID and validates every owned attachment against its `['part', partID, attachmentID]` occurrence. Each sidecar entry also records validated `deliveryPartIndexes`, binding its text and files to the initial worker payload so edit/requeue cannot introduce omitted context. Public scope pages expose this sidecar so queue edit can rebuild the owning draft; worker delivery continues to consume only the compiled `deliveryParts`. The queue captures the authoritative Assistant configuration and complete delivery parts durably. Restart, retry, and worker delivery send those captured parts unchanged. Primary admission continues to accept an omitted delivery target. A changed binding settles the row as `failed` with `stale_target`; clients create a new item when they need a new binding. Assistant text delivery parts may carry the optional boolean `synthetic` marker. The server validator, public scope DTO parser, durable row, and worker preserve that marker. ## Authority modes: shadow / active / cutover Runtime authority is durable per runtime through `getQueueAuthority`/`setQueueAuthority` (`shadow`, `active`, `paused`) with generation CAS, activation epoch, timestamp, manifest hash, and protocol 4. | Authority | Role | |---|---| | `shadow` | Queue DB may stage imports and accept admissions for cutover prep; the worker does **not** dispatch. Legacy client ledger remains the user-visible path until activation. | | `active` | OpenChamber durable queue is authoritative. Worker may reserve, claim, POST, and reconcile. UI chips and mutations use server scope pages. | | `paused` | Dispatch leases are fenced; `sending` rows move to `reconciling`. Resume advances generation so stale probe/dispatch leases cannot POST. | Import staging accepts canonical payloads with scope/item ordinals, payload hashes, and all three queue identities; seal derives the stable manifest. Activation commits a sealed shadow import atomically, verifies identities and attachments, advances authority to `active`, and records its epoch. Late imports atomically append new items in manifest order while stable queue-item/operation identity pairs confirm active rows and completion tombstones after dispatch replaces the provisional admission message ID. There is no long-lived "dormant module" state in production: cutover is explicit `shadow` → `active` (with optional `paused`). Documentation that previously described the production v3 queue as authoritative while this module stayed dormant is obsolete once authority is `active`. Scopes key records by the server-derived runtime key, normalized directory, and session ID. Stateless Assistant clients use the stable synthetic session ID `assistant:` for this existing scope contract; execution still uses a fresh captured-workspace OpenCode Session per item. Client transport identity stays outside durable keys. Each scope exposes a stable `scopeID`, revision, and worktree lifecycle state. A runtime holds at most 128 scopes; creating a new scope at the cap first evicts the oldest empty scopes (LRU by `updated_at`, scope ID tiebreak — safe because `queue_item` is the only foreign key into `queue_scope`), and admission fails with `scope_limit` (HTTP 409) only when every scope still holds items. Edit reservations fence ordinary edit, removal, and manual-send mutations for their exact queue item only; they never fence reorder or unrelated rows in the scope. Reservation renewal binds queue item ID, token, and current authority generation; it accepts a bounded TTL until its durable expiry timestamp, commits only the expiry field in an immediate transaction, and leaves scope and global revisions unchanged. Reserved removal presents its token and authority generation in the same SQLite delete CAS. Releasing a reservation wakes the worker. ## Per-item sendConfig OpenChamber queue continues to store per-item `sendConfig` (`providerID`, `modelID`, optional `agent`/`variant`) on each durable row. Admission and edit validate and persist that object; the worker adapter reads `context.sendConfig` for every POST. There is no scope-level default that replaces missing row config at dispatch time—each admitted row carries the configuration the user selected when the item entered the queue. Admission accepts bounded canonical content, Composer sidecars that canonically serialize to that content, attachment references that occur in the item attachment list, optional composer mentions, and explicit attachment issues. Paste sidecars retain their compact visible token while their payload contributes the canonical content; Session sidecars contribute their stable `@session:` token; durable Skill and Command sidecars contribute `[skill:]` and `[command:]`. Composer reference and mention fields follow the durable input-draft limits and range contract. Standard admission serializes the complete request once and checks its UTF-8 size against a 70 MiB budget before JSON cloning, canonicalization, hashing, or a SQLite transaction; the same serialization supplies the clone. The HTTP parser permits the exact 72 MiB envelope budget. Standard admission requires an empty issue list; `migrationImport` retains issues and persists the item as unresolved. Each item accepts at most 64 attachments, each attachment accepts at most 25 MiB, and one item accepts at most 50 MiB. Binding calculates upload sizes from ready upload metadata and server-path sizes from the authorized realpath stat; client size declarations receive consistency validation. Queue output contains JSON-safe attachment metadata only: object hash, storage key, server-path locator, name, media type, and size. Upload tokens and staging internals remain server-only. Persisted Composer mentions remain in the Composer document sidecar; public scope item DTOs expose them as top-level `composerMentions` alongside `composerDocument`. `createAttachmentUpload`, `markAttachmentReady`, and `expireAttachmentUploads` manage runtime-scoped staging metadata. Runtime GC expires uploads across every durable runtime and protects ready objects through their expiry. Queue snapshots use canonical attachment DTOs with `attachmentID`, `occurrenceRefID`, `filename`, `mimeType`, `size`, `source`, and a locator carrying an upload ID or canonical server path. Sources are `local`, `vscode`, or `server`; occurrences use `['root', attachmentID]` or `['part', partID, attachmentID]`. Legacy attachment fields enter only through `migrationImport`, while ordinary admissions require canonical DTOs. Ready uploads bind to queue rows atomically during admission or edit. Runtime GC removes unreferenced metadata through SQLite CAS before deleting object files; filesystem GC safely retries orphan files after deletion faults. Server-path locators resolve and validate regular files inside the real worktree scope with the injected `isServerPathAllowed(path, runtimeKey)` authorization gate during admission, dispatch, and content streaming. The content endpoint receives a validated attachment stream with MIME type, filename, and byte length. `queue_attachment` rows leave `attachment_object` metadata available to `listAttachmentObjectsForGC` after item removal or completion. ## Dispatch fencing and reconciling `reserveEligibilityCandidate` acquires an internal expiring probe lease in `queue_attempt`, leaving the queued item, public due time, row version, and scope revision stable while the worker checks upstream eligibility. Probe validity includes the current authority generation, so pause/resume immediately fences stale tokens. `deferEligibilityCandidate` shortens that lease to the next probe boundary. `claimNext` requires active authority, preserves per-scope queue heads, promotes the matching probe token into the dispatch lease, and returns its fencing generation. **Same-scope POST slot:** at most one `sending` row may hold a live POST per scope. `reconciling` only tracks confirmation after an accepted or ambiguous POST and does **not** occupy the manual dispatch slot—so a second manual intent may enter one `sending`/POST while an earlier row reconciles. Automatic candidates still require eligibility `available` + `idle` + `settled`; manual dispatch proceeds after an authoritative `available` read (busy/unsettled sessions allowed). Manual and automatic heads still serialize by durable `position` among `queued`/`retrying` peers. Probe and dispatch leases fence the same row and peer probe leases so a queue row never receives a second concurrent POST; reconcile leases on `reconciling` rows do not block the next head. Claim selection and update fence a scope with every `sending` row and every valid probe lease on `queued`/`retrying` peers. Manual promotion and reorder clear queued probe leases in their scope before committing, so their explicit ordering intent preempts an eligibility check. An existing `sending` or `reconciling` attempt stays pinned in list order because its POST cannot be unsent, but only `sending` locks the concurrent POST slot. Manual send promotes its target immediately after active tracking rows, and reorder overwrites waiting-row order while preserving active slots. `renewLease`, `releaseIneligible`, `beginAttempt`, `scheduleRetry`, `markAcceptedForReconciliation`, `markAmbiguous`, `markFailed`, and completion paths all carry `{ queueItemID, leaseToken, fenceGeneration, runtimeKey }`. Successful accepted primary delivery calls `markAcceptedForReconciliation` (shared durable transition with transport ambiguity); `markAmbiguous` remains for unknown transport outcomes and stays compatible with older callers. Durable rows remain until exact message confirmation or completion tombstone. ## Worker and OpenCode adapter The worker starts paused. Start, timer, and wake calls share one launched run flight; their public wake promise settles with status, while rejected flights emit a worker diagnostic and later wakes may retry. Stop aborts controllers and waits for every active task to settle before the runtime closes SQLite. Each run captures one runtime key and adapter runtime snapshot, **starts dispatch probe tasks first** (up to concurrency), then claims and awaits due reconciliation work, and finally joins active probes. That ordering keeps a hung `findMessage` (bounded by the 10s reconcile timeout) from delaying a waiting manual intent's POST while same-scope `sending` and same-row leases still prevent a second concurrent POST; automatic candidates remain gated on `available` + `idle` + `settled`. It reserves a queued candidate with an expiring probe token, and promotes that token only after the bounded eligibility probe permits dispatch. Stateless Assistant items deliberately bypass OpenCode eligibility and automatic admission tokens because their synthetic scope is not an execution Session; same-Assistant scope claiming still serializes them, while different Assistant scopes use normal worker concurrency. A successful stateless Assistant `promptAsync` admission first persists its user message in Assistant SQLite and advances the Assistant binding revision, then completes and deletes the queue row. The resulting Assistant revision tip refreshes clients that were still displaying the previous disposable binding. An ambiguous stateless delivery becomes a client-mutable `failed` row instead of an indefinitely active reconciliation row, and terminal rows do not block later dispatchable rows in their scope. Upgrade recovery also settles legacy stateless reconciliation rows without querying their disposable Session: null-error rows represent the old accepted path and complete, while rows carrying a transport error become failed. Busy, unsettled, timed-out, and unavailable candidates remain queued without row-version, scope-revision, or revision-tip churn. Every lease mutation carries `{ queueItemID, leaseToken, fenceGeneration, runtimeKey }`. `claimNext` returns `{ item, leaseToken, leaseExpiresAt, fenceGeneration }`; the item includes scope ID, directory, and session ID. The service receives the durable queue runtime identity, while the adapter receives an independently injected upstream OpenCode runtime snapshot. OpenCode adapter (SDK 1.18.x) hardening: - `safeStatus` reads `result.response.status`, `result.error.status`, then `result.status`. - `send` treats only explicit 2xx (including empty 200/202/204) with no `error` as `ok`. `undefined`/malformed results are never success. - Definitive 4xx (except 408/429) is `failed`; 408/429/5xx and transport throws are `ambiguous`. - `findMessage` prefers `client.v2.session.message` when present; 404 means `found: false`. On 404/405/501 or unsupported surfaces it falls back to legacy `client.session.message` (if the SDK exposes it), then the existing bounded `session.messages` scan—without re-POSTing. ## Confirmation and reconciliation `confirmByMessage({ runtimeKey, directory, sessionID, messageID, source })` completes an attempt from a realtime or query event without a worker lease. Global event confirmation accepts current active and paused authority after validating its connection runtime identity; when the queue runtime is initialized after an existing global-stream connection, that connection late-binds its first available identity and keeps it fenced until reconnect. The service owns exact runtime, directory, session, and message matching. The completion tombstone serializes this event path with normal lease-fenced completion. Reconciliation uses an independent fenced claim, so concurrent workers and wakes issue one query for a due row. It stores start, absolute deadline, check count, and next check across restart. Unavailable checks preserve the count and use bounded exponential backoff; authoritative complete misses increment the count. The deadline or three misses resolves the row as unresolved. A client Remove may discard a sending or reconciling tracking row immediately; this is an explicit UI-authoritative cleanup and does not cancel an upstream request that already crossed the POST boundary. ## Receipts and mutations Every external mutation has a runtime-scoped durable request receipt containing its operation type, canonical payload hash, compact response JSON, and committed revision. Repeated request IDs with the same operation and canonical payload return the saved response inside a 30-day idempotency window. Receipt reads and service startup clear expired records, so an expired request leaves that window and executes as a new request. Each runtime retains at most 16,384 receipts; a mutation transaction clears expired and oldest excess records before writing its new receipt. Compact acknowledgements contain revision plus identity fields: admission/edit add scope ID, queue item ID, and row version; remove adds scope ID and removed queue item ID; reorder adds scope ID; worktree order adds project directory. Lifecycle responses retain token, state, and counts. Queue, operation, and message IDs follow the v4 globally unique identity contract. Admission replay uses the immutable queue-item/operation pair and returns the completed dispatch message ID, while worker completion continues to require the exact final identity triple. Scope mutations use revision and row-version compare-and-swap rules, with each touched scope revision set to the global committed revision. Worktree deletion freezes admissions, edits, and reorders while preserving removals and queue items for rollback or committed deletion. `markWorktreeActive` preserves a deleting lifecycle and its token; a verified recreated worktree may restore a deleted lifecycle. Worktree order is shared server state keyed by runtime and normalized project directory. Committing worktree deletion removes its path from that project's order in the same transaction and revision. Each runtime is bounded to 128 scopes, 2,048 items, 1,024 order rows, and 4,096 ordered worktree paths; one project stores up to 1,024 paths. Device-local UI layout and selection state remain local to each device. `getWorktreeLifecycle(directory, { runtimeKey })` is an internal server coordination snapshot for recovery after the Git catalog has confirmed that a worktree still exists. Trusted server callers capture the service-generated SHA-256 runtime key before an asynchronous Git flow and pass it to lifecycle read, prepare, commit, rollback, and mark-active calls. The captured key preserves one runtime boundary through the full flow. It returns the normalized runtime lifecycle state and deletion token, allowing token-CAS rollback after a crash. Message-queue HTTP snapshots expose catalog and ordering records; deletion tokens remain in the server coordination boundary. The root snapshot contains a catalog of scope identity, revision, lifecycle state, and item count plus worktree orders. Revision tips arrive as OpenChamber SSE events (`openchamber:message-queue-changed`) through the shared UI `runtimeFetch` streamed-response transport; Relay tunnels carry the response transparently. Clients lead with a snapshot GET, page scopes that diverged, then wait for the next tip only after the cache matches; tips published during paging are recovered by the following leading GET. Scope reads return metadata with one item page of at most eight items, itemCount, and nextOffset when another page exists. The first page supplies the scope revision; each later page supplies that revision as `expectedRevision`, which preserves one consistent scope view. ## Manual send and optimistic chip projection Manual send stores a one-time durable dispatch intent at any moment (waiting statuses only). Scope and item DTOs expose the non-sensitive authoritative `manualDispatchRequested` field. Authoritative tracking rows—`manualDispatchRequested === true`, `sending`, and `reconciling`—remain durable until exact confirmation/tombstone. UI chips keep pending client send, committed-ack send shadows, and authoritative `manualDispatchRequested` visible in a **"Sending…"** state (parallel to pending-admission "Queuing…"); chips hide only after authoritative `sending`/`reconciling` (or removal/confirm). `failed` / `unresolved` restore normal Send/Edit. Pending client send mutations leave the target chip visible; a definitive mutation failure clears the overlay so Send/Edit return. After the mutation leaves `pending`, a successful send keeps a **committed-ack-revision shadow**: the client result carries `committedRevision` from the action receipt even when post-commit scope reload fails, and the shadow continues selecting that exact runtime generation/scope send target for "Sending…" UI ownership while the authoritative scope revision is still behind the ack. Shadow ends when scope revision reaches the ack, the runtime generation changes, or the success mutation is cleaned from the MutationCache. Client send-pending presentation times out after 8s (`SERVER_QUEUE_SEND_PENDING_TIMEOUT_MS`) and restores clickable Send when stuck without reaching sending/reconciling/removal. The overlay never writes the revision-pinned Query cache. `beginAttempt` consumes the manual intent atomically with its message ID and attempt history; `sending` and `reconciling` status then represent active delivery tracking. Pre-attempt release and retry preserve the intent. Pause, expired-sending recovery, and ambiguity reconciliation clear the intent because the persisted attempt owns delivery certainty. Compact mutation acknowledgements retain their existing identity and revision fields. Dual-client replay of the same manual-send `requestID` returns the stored receipt once. ## Import HTTP DTOs Status responses use camelCase fields and include `activatedAt` after activation. Create, stage, seal, activate, late commit, abandon, pause, and resume each expose their own exact DTO; seal always includes `itemCount`. Import attempts key by runtime, kind, device client ID, and logical snapshot hash. `GET /imports/:importID` exposes state, manifest, count, staged ordinals and hashes, plus a persisted committed result; it excludes upload tokens and payload bodies. Committed replay returns that durable result after pause/resume generations advance. ## UI cutover recovery The UI cutover coordinator serializes refreshes per runtime. A lost activation response confirms success from the matching manifest and activation epoch. When another client has already activated or paused authority, the coordinator retains server ownership, re-prepares the current local ledger, and appends it through a late import at the current generation. Import preparation, staging, and commit failures retain frozen shadow ownership or server ownership according to authority and retry with bounded exponential backoff. Runtime changes abort the old retry lane.