# zcode-acp-server Architecture ## Overview `zcode-acp-server` bridges the headless ZCode CLI (`zcode app-server --stdio`) to ACP (Agent Client Protocol) compatible editors. It is the translation layer between ZCode's internal JSON-RPC event stream and the standard ACP protocol. ## Layered Architecture ``` application-client (Zed / JetBrains) session/update | v zcode-acp-server (stdio JSON-RPC ACP) |-- handlers/ session, extensions, dispatch, server-requests, io, slash, account |-- translators/ event-translator, projection-differ, tool-helpers |-- interaction/ adapter |-- config/ options, runtime-model, model-cache |-- backend/ client, listener, types |-- server.ts ZcodeAcpServer | v zcode app-server --stdio (line-delimited JSON) ``` ## ACP Handshake The `initialize` request (`server.ts`) negotiates the protocol version and declares the agent's shape to the editor: - **`protocolVersion`** — pinned to `PROTOCOL_VERSION` (currently 1). - **`agentInfo`** — name/title/version from `AGENT_INFO` in `utils.ts`. - **`agentCapabilities`** — `loadSession`, plus `sessionCapabilities.list / resume / fork`. The prompt capabilities (image/audio/embeddedContext) and MCP capabilities are all off. - **`authMethods`** — a single agent-type entry (`zcode-credentials`). The bridge reads the GLM API key itself from `~/.zcode/v2/config.json` and forwards it to the ZCode subprocess via `ANTHROPIC_API_KEY`; the editor never supplies credentials. Omitting the `type` field defaults to `"agent"`, which the ACP registry's auth-check accepts as "agent self-handles auth". `initialize` does **not** spawn the backend, and neither does `session/new`: the backend is lazily created on the first backend RPC — for a fresh session that is the first `session/create` at its first use (prompt / config change / extension method). This keeps the handshake succeeding even in an environment without `~/.zcode/v2/config.json` (e.g. the registry CI runs `initialize` with an isolated `HOME`). Client capabilities advertised at `initialize` are recorded on the server (`clientCapabilities`) and drive later behaviour: `supportsElicitationForm()` gates form-based elicitation, and `supportsTerminalOutput()` gates Zed's Bash terminal UI. ## Core Data Flow ### 1. Session lifecycle ``` session/new → placeholder id (backend session NOT created yet) | first use: prompt / set_config_option / extension method | session/create → register EventListener | prompt request → session/send → EventTranslator translates → dispatchEvent | | end_turn / cancelled session/update notification ``` Sessions are materialized lazily (`ensureRealSession`): an editor startup that never sends a message leaves no empty session in the backend or the App's task index. The placeholder → backend-session mapping is persisted to `~/.zcode/v2/acp-lazy-sessions.json` (`src/lazy-sessions.ts`), so a `session/ resume` / `session/load` of a placeholder from a previous bridge lifetime still resolves: with a recorded backend id the real session is resumed, without one a fresh (empty) session is materialized — never "Session not found". ### 2. Event stream subscription ``` EventStreamListener.subscribe() | session/subscribe (deliveryKind: "desktop-continuous") | ZCode pushes session/event → handleEvent() | pollEvent() consumes → EventTranslator.translate() ``` ### 3. Dual-path event handling #### Real-time path (EventTranslator) - Listens to zcode `session/event` pushes - Translates each event to an ACP `session/update` in real time - Maintains `seenToolIds` to avoid duplicates #### Snapshot path (ProjectionDiffer) - On turn completion, builds a snapshot from `session/messages` + `session/read` - Diffs two snapshots to produce new events (PlanUpdate / TextDelta / ToolCallNew, etc.) - Used for turn-completion triage and stall recovery ### 4. Dual-path deduplication ``` EventTranslator (real-time path) ├── seenToolIds: Set └── turnDone: boolean | v ProjectionDiffer (snapshot path) ├── seenToolIds: Set ├── lastToolStatus: Map └── seenMessageIds: Set | v dispatchEvent (single exit point) ``` Key: **`seenToolIds` synchronization** In `session.ts:629`, after the event path finishes processing, the state is synced to the differ: ```typescript for (const seenId of translator.seenToolIds) { differ.markToolSeen(seenId); } ``` This ensures the snapshot diff does not re-emit tools already handled by the event path, preventing Bash terminal output from being overwritten by a content-less ToolCallNew. ## Data & Privacy **No telemetry, no analytics, no third-party network calls.** The server is a local relay: prompts, code, and tool outputs pass through process memory on their way between the editor and the ZCode subprocess, but reach the GLM cloud API only because the ZCode backend itself sends them for inference. | Concern | Detail | | ----------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Network | One outbound request in the whole codebase — `src/quota/client.ts` GET to the quota API, Bearer token only, no body | | Credentials | API key from `~/.zcode/v2/config.json` (authenticates the subprocess + quota request), never logged. OAuth handled by the ZCode subprocess, not this server | | Disk | No new files. Writes only to the existing `~/.zcode/v2/tasks-index.sqlite` — syncs sessions into the ZCode app's history & search (session title + first prompt) | | Logging | `log()`/`warn()` → stderr only for troubleshooting; even with `ZCODE_ACP_DEBUG=1`, no prompts/code/keys are logged | ## Module Responsibilities ### `backend/` — ZCode process communication | File | Responsibility | | ------------- | ----------------------------------------------------------------------------------------------- | | `client.ts` | Spawn/manage the zcode subprocess, reader-loop, request/response multiplexing, process watchdog | | `listener.ts` | EventStreamListener (subscribe/consume the event stream) and TurnMonitor (snapshot polling) | | `types.ts` | ZCode JSON-RPC message type definitions | ### `translators/` — Event translation | File | Responsibility | | ---------------------- | -------------------------------------------------------------------------------------------------- | | `event-translator.ts` | Translate zcode events to InternalEvent (real-time path) | | `projection-differ.ts` | Diff two snapshots to produce InternalEvent (snapshot path) | | `tool-helpers.ts` | Tool-related pure functions: title generation, output rendering, diff parsing, location extraction | | `types.ts` | InternalEvent union type and plan entry builders | ### `handlers/` — ACP method handling | File | Responsibility | | --------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `session.ts` | session/new/list/resume/load/prompt/set_config_option/cancel | | `extensions.ts` | fork/rewind/rewindCascade/goal/compact/steer/cancelBackgroundTask/setModel/setMode/setThoughtLevel | | `dispatch.ts` | dispatchEvent single exit point: InternalEvent → ACP session/update | | `background-tasks.ts` | Session-scoped `BackgroundTaskListener` — forwards background sub-agent status (`session.updated` taskId) + completion-notification turns to the client OUTSIDE request handlers (lives across prompts) | | `session-titles.ts` | Session-scoped `SessionTitleListener` — adopts backend `session.titleUpdated` pushes (`generated`/`custom` sources; manual renames win) into sessionTitles/tasks-index/terminal tab + broadcasts `session_info_update` | | `server-requests.ts` | Handle zcode interaction/* requests (tool auth, ExitPlanMode, AskUserQuestion), protocol negotiation routing | | `io.ts` | ACP notification helpers (including `sendAvailableCommandsDeferred` deferred notification) | | `slash.ts` | Interception of `/`-prefixed commands (/compact /goal /fork /rewind /steer /model /mode /thought); non-advertised `/x` prompts are neutralized into plain text (`neutralizeSlashText`) | | `account.ts` | `account/usage_stats` — account-level plan quota for remote clients (Proposal 0002; quota pipeline + graceful error) | ### `interaction/` — Interaction bridging | File | Responsibility | | ------------ | ------------------------------------------------------------------------------------------------ | | `adapter.ts` | Conversion adapter from zcode interaction requests to ACP (requestPermission + elicitation form) | ### `config/` — Configuration management | File | Responsibility | | ------------------ | -------------------------------------------------------------- | | `options.ts` | configOptions / modes construction, set_config_option dispatch | | `runtime-model.ts` | runtimeModel overlay construction and application | | `model-cache.ts` | Model ID cache and usage initialization | ### `remote/` — Remote access (opt-in via `ZCODE_ACP_REMOTE=1`) | File | Responsibility | | --------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `broadcast.ts` | ClientRegistry + broadcast proxy: notify fans out to all clients; request is first-response-wins with loser `$/cancel_request` | | `config.ts` | ENV parsing (gate, mandatory token, hub/bridge ports) | | `endpoint.ts` | Loopback ACP endpoint (SDK AcpServer transport, port auto-increment) + hub registration/heartbeat | | `file-endpoint.ts` | Read-only `GET /fs/list` + `GET /fs/file` on the loopback server (ADR-0004): Session Root scoping, byte/line windows, streaming | | `status-endpoint.ts` | `GET /status` on the loopback server (ADR-0005): in-memory per-session running state (`pendingTurns` derivation), zero backend RPC | | `session-close-endpoint.ts` | `POST /sessions/{id}/close` on the loopback server (ADR-0006): running-guarded discovery retirement with self-healing re-appearance | | `hub-server.ts` | The hub singleton: token auth, instance discovery, byte-level WS proxying, heartbeat pruning, on-demand `?probe=1` liveness, idle exit, version self-upgrade, `GET /api/quota` direct query (ADR-0005), POST close proxy (ADR-0006) | When enabled, the same `AgentApp` serves the stdio editor and a loopback WebSocket endpoint. Every connection (editor or remote) joins the broadcast registry via `trackConnections`, so one turn's notifications reach all clients regardless of who prompted. The bridge registers itself with the machine-level the hub daemon (`zcode-acp hub`, `bin/hub.ts`), which is the only public entry point and holds no session state (see `docs/adr/0002`). The bridge's lifetime still follows the stdio client (ADR-0001); the listener is `unref()`'d so remote clients alone never keep the process alive. Hub upgrades are self-managing: each heartbeat carries the bridge's package version, and a hub that sees a NEWER bridge replies `{ok, restarting}`, exits, and is re-spawned by that bridge from its own (upgraded) `dist/` within a few seconds. Equal, older, or absent versions never trigger a restart — downgrades and mixed-version fleets are fine. Without this handshake a long-lived hub would keep running pre-upgrade code until its 10-minute idle exit. Discovery liveness has two layers: the heartbeat TTL (30s, pruned every 5s) drops bridges that stopped registering — the fallback for hard kills — and `GET /api/instances?probe=1` actively TCP-probes each registered loopback port on demand, so a client refresh gets an honest list with no background probing cost. A probe failure is not a verdict: the first one only marks the instance unhealthy, and ~8s of continuous unreachability (confirmed by a later probe) prunes it — a busy bridge's event loop can stall past the connect timeout while perfectly alive, and evicting it would kick every attached client. Session file access (ADR-0004) rides the same loopback server: the bridge serves read-only `GET /fs/list` + `GET /fs/file` scoped to each session's cwd, and the hub byte-proxies them at `/api/instances/{id}/fs/*` — no new port, and the hub still holds no path semantics. Running status and quota also have plain-HTTP channels (ADR-0005) so clients can poll without an ACP connection: the bridge's `GET /status` (pure in-memory assembly) is proxied at `/api/instances/{id}/status` and echoed coarsely in each heartbeat (`sessions[].status`), while `GET /api/quota` is queried by the hub itself — quota belongs to the machine's credentials, not to any instance, so it works with zero bridges registered. Session close (ADR-0006) is the surface's first write op: `POST /api/instances/{id}/sessions/{sid}/close` retires a conversation from discovery (running-guarded, self-healing if the editor still has it open). ## Key State Machines ### Turn state ``` subscribe | turn.started | +---------------------+ | model.streaming | | tool.updated | | session.updated | +---------------------+ | +---------------------+ | turn.completed | -> end_turn | turn.failed | -> error | turn.cancelled | -> cancelled | no protocol | | progress (120s) | -> check read watermark | watermark moved | -> forward throttled usage -> defer (alive) | active tool | -> refresh in_progress -> defer (alive) | frozen < 10 min | -> defer decision | frozen >= 10 min | -> fetch reply -> end_turn | | no reply, no output -> max_turn_requests | manual cancel | -> cancelled +---------------------+ ``` Projection polling is a recovery signal, not protocol progress. In particular, `projection.status=running` may be stale and therefore never refreshes the 120-second deadline. Neither is the prompt lock a liveness signal — verified against the Aug-28 app-server, `session/goal show` succeeds mid-turn, and a probe `session/send` is accepted (queued as steer input) while the turn runs; the lock is only held during finalisation. Instead the 15s stall-reconcile reads feed a liveness watermark (`contextUsed`/`totalTokenCount`/`turnCount`/`currentTurnId` from `session/read`): an advancing watermark proves a silently-working turn (typically a sub-agent) and defers the terminal decision indefinitely. Because an upstream ACP client can have a shorter idle deadline than this bridge, the bridge also forwards authoritative progress at most once per minute: a `usage_update` when the watermark advances, or an `in_progress` refresh for a known active tool while its watermark is temporarily quiet. These updates are state-bearing; the bridge never fabricates transcript text as a heartbeat. A watermark frozen for 10 minutes (STALE_FREEZE_MS) still marks the projection as truly stale unless a known foreground tool remains nonterminal. An active tool is direct lifecycle evidence that the turn has not completed, so the bridge keeps waiting and refreshing its existing card. Without an active tool, the turn ends gently (reply fetch first, bounded stop only when nothing was ever delivered). Already-queued events win the deadline race and are consumed first. ### Tool lifecycle ``` scheduled -> started -> progress -> result/error | | | ToolCallNew status=in_progress output (stdoutTail) | (seenToolIds.add) ``` ## Interaction Protocol Negotiation ZCode `interaction/*` requests are routed to different ACP client interaction mechanisms via protocol negotiation: ``` zcode interaction request received │ ├─ Tool auth (interaction/requestPermission) │ └─ Always uses session/request_permission (its native purpose) │ ├─ ExitPlanMode (interaction/requestUserInput + plan_approval) │ ├─ martty attached (hasMarttyClient) → elicitation/create (approve/reject form; │ │ its permission overlay draws only the title — the plan needs the form) │ └─ Otherwise → session/request_permission (editors render toolCall.content │ as full markdown; elicitation form descriptions are plain text there) │ └─ AskUserQuestion (interaction/requestUserInput) ├─ Client supports elicitation.form → elicitation/create (single form) └─ Otherwise → per-question session/request_permission (fallback) ``` **Key**: `server.supportsElicitationForm()` is detected at `initialize` time from `clientCapabilities.elicitation.form` (AskUserQuestion). ExitPlanMode keys on `server.hasMarttyClient()` instead — Zed ≥1.12 declares `elicitation.form` too, but its form descriptions are plain text while its permission popups render `toolCall.content` as markdown, so only martty benefits from the form. Tool auth always goes through request_permission, since that is its native purpose. ## Deferred Notification Mechanism The `available_commands_update` notification must be sent **after** the session response; otherwise the client's session state machine is not yet ready and drops the notification, leaving the `/` completion menu empty. ``` session/new|resume|load handler │ ├─ call newSession()/resumeSession()/loadSession() │ ├─ sendAvailableCommandsDeferred(cx, sid, SLASH_COMMANDS) │ └─ enqueue, send after 50ms (fire-and-forget) │ └─ return response │ └─ after the response is written to stdout, the 50ms timer fires sendAvailableCommands ``` `sendAvailableCommandsDeferred` (`io.ts`) encapsulates the 50ms delay logic, mirroring the Python bridge's `_pending_post_notifs` queue + `_drain_post_notifs` mechanism. ## Mode Reconciliation The session mode can change through four entry points, all of which must notify the editor UI: | Trigger | Path | Notifies UI | | -------------------------------------- | -------------------------------------------------------------- | :---------: | | `session/setMode` request | `extensions.ts:setMode` | yes | | `session/set_config_option` (mode) | `session.ts:setConfigOptionHandler` → `emitConfigOptionUpdate` | yes | | `/mode` slash command | `slash.ts` → `emitConfigOptionUpdate` | yes | | In-turn `EnterPlanMode`/`ExitPlanMode` | reconciled at turn completion | yes | The in-turn path bypasses the bridge entirely, so `prompt()` runs `emitModeIfChanged` (`session.ts`) at turn completion: it re-reads the authoritative mode via `buildModes`, compares against `server.lastMode` (the value last advertised to the client), and emits `current_mode_update` + `config_option_update` when they differ. Failures are swallowed so they cannot break the turn-completion path. ## Prompt-Lock Release on Stop `session/stop` is fire-and-forget, but ZCode has a startup delay: when stop arrives before the turn truly holds the lock the backend ignores it, the turn runs on, and the lock leaks (next `session/send` fails with "A prompt is already running"). `ensureTurnStopped` (`session.ts`) closes this gap. It mirrors the `expectLock:true` strategy from `waitForTurnIdle` (`extensions.ts`): 1. send `session/stop` 2. poll `session/goal show`; first REQUIRE seeing "prompt is running" once (proves the turn started), then wait for it to clear 3. if an 8s grace window elapses without ever seeing the lock, the turn never started (stop caught it in time) or already ended → treat as released 4. hard timeout 30s It is used at every stop site: the cancel check, the stall no-output path, the turn cancelled/failed result, the 120s no-progress timeout, and the `session/send` error path. Never throws (failures only log) so it cannot break the cancel path. ## Process Watchdog `close()` reaps the zcode process group via `process.kill(-pid)`, but only when the bridge exits cleanly enough for its signal handlers to fire (SIGTERM/SIGINT). If the bridge is SIGKILLed (Zed force-kill on reconnect, crash, OOM), the handler never runs and the zcode subprocess group is orphaned. The watchdog (`backend/client.ts:startWatchdog`) closes that gap. It is a tiny detached child that polls the bridge pid every 2s and, once the bridge is gone, sends SIGKILL to the zcode process group, then exits. It is its own process-group leader and `unref`'d, so it never holds the event loop open and is not part of the zcode group it kills. It self-terminates as soon as the zcode process exits, so a normal shutdown leaves no lingering watchdog. ## Design Decisions ### Why a dual path? | Scenario | Real-time path | Snapshot path | | ---------------------- | -------------- | ------------------------------- | | Normal streaming | Low latency | Must wait for turn end | | Lost events | Loses data | Recovers from snapshot | | Deduplication | seenToolIds | seenMessageIds + markToolSeen() | | Turn-completion triage | Not triggered | PlanUpdate / usage_update | ### Why no polling fallback? ZCode CLI 0.14.5 ~ 0.14.7 used `session/read` polling to emulate streaming. 0.14.8+ introduced `session/subscribe` event push, which has lower latency and is more reliable. This project supports only 0.14.8+ and has removed the polling fallback code. ### Why does ProjectionDiffer need to persist across turns? - `seenMessageIds`: prevents historical messages from being re-emitted after resume - `lastToolStatus` + `seenToolIds`: ensures tool state is not lost across turns - `lastUsage`: avoids duplicate usage_update pushes - `lastPlanSig`: emits only when the plan changes