# zcode-acp-server Architecture ## Overview `zcode-acp-server` bridges the headless ZCode CLI (`zcode app-server --stdio`) to ACP (Agent Client Protocol) compatible editors. It is the translation layer between ZCode's internal JSON-RPC event stream and the standard ACP protocol. ## Layered Architecture ``` application-client (Zed / JetBrains) session/update | v zcode-acp-server (stdio JSON-RPC ACP) |-- handlers/ session, extensions, dispatch, server-requests, io, slash, account |-- translators/ event-translator, projection-differ, tool-helpers |-- interaction/ adapter |-- config/ options, runtime-model, model-cache |-- backend/ client, listener, types |-- server.ts ZcodeAcpServer | v zcode app-server --stdio (line-delimited JSON) ``` ## ACP Handshake The `initialize` request (`server.ts`) negotiates the protocol version and declares the agent's shape to the editor: - **`protocolVersion`** — pinned to `PROTOCOL_VERSION` (currently 1). - **`agentInfo`** — name/title/version from `AGENT_INFO` in `utils.ts`. - **`agentCapabilities`** — `loadSession`, plus `sessionCapabilities.list / resume / fork`. The prompt capabilities (image/audio/embeddedContext) and MCP capabilities are all off. - **`authMethods`** — a single agent-type entry (`zcode-credentials`). The bridge reads the GLM API key itself from `~/.zcode/v2/config.json` and forwards it to the ZCode subprocess via `ANTHROPIC_API_KEY`; the editor never supplies credentials. Omitting the `type` field defaults to `"agent"`, which the ACP registry's auth-check accepts as "agent self-handles auth". `initialize` does **not** spawn the backend, and neither does `session/new`: the backend is lazily created on the first backend RPC — for a fresh session that is the first `session/create` at its first use (prompt / config change / extension method). This keeps the handshake succeeding even in an environment without `~/.zcode/v2/config.json` (e.g. the registry CI runs `initialize` with an isolated `HOME`). Client capabilities advertised at `initialize` are recorded on the server (`clientCapabilities`) and drive later behaviour: `supportsElicitationForm()` gates form-based elicitation, and `supportsTerminalOutput()` gates Zed's Bash terminal UI. ## Core Data Flow ### 1. Session lifecycle ``` session/new → placeholder id (backend session NOT created yet) | first use: prompt / set_config_option / extension method | session/create → register EventListener | prompt request → session/send → EventTranslator translates → dispatchEvent | | end_turn / cancelled session/update notification ``` Sessions are materialized lazily (`ensureRealSession`): an editor startup that never sends a message leaves no empty session in the backend or the App's task index. The placeholder → backend-session mapping is persisted to `~/.zcode/v2/acp-lazy-sessions.json` (`src/lazy-sessions.ts`), so a `session/ resume` / `session/load` of a placeholder from a previous bridge lifetime still resolves: with a recorded backend id the real session is resumed, without one a fresh (empty) session is materialized — never "Session not found". ### 2. Event stream subscription ``` EventStreamListener.subscribe() | session/subscribe (deliveryKind: "desktop-continuous") | ZCode pushes session/event → handleEvent() | pollEvent() consumes → EventTranslator.translate() ``` ### 3. Dual-path event handling #### Real-time path (EventTranslator) - Listens to zcode `session/event` pushes - Translates each event to an ACP `session/update` in real time - Maintains `seenToolIds` to avoid duplicates #### Snapshot path (ProjectionDiffer) - On turn completion, builds a snapshot from `session/messages` + `session/read` - Diffs two snapshots to produce new events (PlanUpdate / TextDelta / ToolCallNew, etc.) - Used for turn-completion triage and stall recovery ### 4. Dual-path deduplication ``` EventTranslator (real-time path) ├── seenToolIds: Set └── turnDone: boolean | v ProjectionDiffer (snapshot path) ├── seenToolIds: Set ├── lastToolStatus: Map └── seenMessageIds: Set | v dispatchEvent (single exit point) ``` Key: **`seenToolIds` synchronization** In `session.ts:629`, after the event path finishes processing, the state is synced to the differ: ```typescript for (const seenId of translator.seenToolIds) { differ.markToolSeen(seenId); } ``` This ensures the snapshot diff does not re-emit tools already handled by the event path, preventing Bash terminal output from being overwritten by a content-less ToolCallNew. ## Data & Privacy **No telemetry, no analytics, no third-party network calls.** The server is a local relay: prompts, code, and tool outputs pass through process memory on their way between the editor and the ZCode subprocess, but reach the GLM cloud API only because the ZCode backend itself sends them for inference. | Concern | Detail | | ----------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Network | One outbound request in the whole codebase — `src/quota/client.ts` GET to the quota API, Bearer token only, no body | | Credentials | API key from `~/.zcode/v2/config.json` (authenticates the subprocess + quota request), never logged. OAuth handled by the ZCode subprocess, not this server | | Disk | Updates `~/.zcode/v2/tasks-index.sqlite` for history/search. Embedded generic binaries are staged privately in `$ZCODE_HOME/v2/acp-attachments` and retained for native reads/resume; see [retention](../README.md#prompt-attachments). | | Logging | `log()`/`warn()` → stderr only for troubleshooting; even with `ZCODE_ACP_DEBUG=1`, no prompts/code/keys are logged | ## Module Responsibilities ### `backend/` — ZCode process communication | File | Responsibility | | ------------------ | ----------------------------------------------------------------------------------------------------------- | | `adapter.ts` | `BackendAdapter` seam (ADR-0023): `BackendKind`, the capability table (single gate source), wire vocabulary | | `jsonrpc-child.ts` | Shared stdio JSON-RPC transport: spawn, reader-loop, request multiplexing, watchdog, dialect hooks | | `client.ts` | `ZcodeBackend` — the zcode dialect over `JsonRpcChild` (bare frames, arrival responders) | | `listener.ts` | EventStreamListener (subscribe/consume the event stream) and TurnMonitor (snapshot polling) | | `types.ts` | ZCode JSON-RPC message type definitions | ### `translators/` — Event translation | File | Responsibility | | ---------------------- | -------------------------------------------------------------------------------------------------- | | `event-translator.ts` | Translate zcode events to InternalEvent (real-time path) | | `projection-differ.ts` | Diff two snapshots to produce InternalEvent (snapshot path) | | `tool-helpers.ts` | Tool-related pure functions: title generation, output rendering, diff parsing, location extraction | | `types.ts` | InternalEvent union type and plan entry builders | ### `handlers/` — ACP method handling | File | Responsibility | | --------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `session.ts` | session/new/list/resume/load/prompt/set_config_option/cancel | | `extensions.ts` | fork/rewind/rewindCascade/goal/compact/steer/cancelBackgroundTask/setModel/setMode/setThoughtLevel | | `dispatch.ts` | dispatchEvent single exit point: InternalEvent → ACP session/update | | `background-tasks.ts` | Session-scoped `BackgroundTaskListener` — forwards background sub-agent status (`session.updated` taskId) + completion-notification turns to the client OUTSIDE request handlers (lives across prompts) | | `session-titles.ts` | Session-scoped `SessionTitleListener` — adopts backend `session.titleUpdated` pushes (`generated`/`custom` sources; manual renames win) into sessionTitles/tasks-index/terminal tab + broadcasts `session_info_update` | | `server-requests.ts` | Handle zcode interaction/* requests (tool auth, ExitPlanMode, AskUserQuestion), protocol negotiation routing | | `io.ts` | ACP notification helpers (including `sendAvailableCommandsDeferred` deferred notification) | | `slash.ts` | Interception of `/`-prefixed commands (/compact /goal /fork /rewind /steer /model /mode /thought); non-advertised `/x` prompts are neutralized into plain text (`neutralizeSlashText`) | | `account.ts` | `account/usage_stats` — account-level plan quota for remote clients (Proposal 0002; quota pipeline + graceful error) | ### `interaction/` — Interaction bridging | File | Responsibility | | ------------ | ------------------------------------------------------------------------------------------------ | | `adapter.ts` | Conversion adapter from zcode interaction requests to ACP (requestPermission + elicitation form) | ### `config/` — Configuration management | File | Responsibility | | ------------------ | -------------------------------------------------------------- | | `options.ts` | configOptions / modes construction, set_config_option dispatch | | `runtime-model.ts` | runtimeModel overlay construction and application | | `model-cache.ts` | Model ID cache and usage initialization | ### `settings/` — ZCode configuration management (ADR-0025–0028) | File | Responsibility | | -------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------ | | `file-lock.ts` | ZCode's cross-process file lock (`.lock` + owner files + stale reclaim) so the bridge and a running desktop app never overwrite each other | | `atomic-write.ts` | Locked read-modify-write of a config file: unknown keys preserved, self-validation, backup, temp + rename | | `provider-config.ts` | `~/.zcode/v2/provider_config.json`: provider enable/rename, model add/remove, contextWindow / reasoningLevel | | `cli-config.ts` | `~/.zcode/cli/config.json`: skills enablement, MCP servers, hooks tree, plugin enablement | | `agents-config.ts` | `~/.zcode/v2/agents-state.json` + `~/.zcode/agents/*.md`: subagent CRUD (frontmatter only), enable, model override | | `skills.ts` | Skill discovery + enable/delete/copy-to-user with realpath escape guards | | `usage-stats.ts` | Local `~/.zcode/cli/db.sqlite` aggregation: per-model tokens + daily buckets | | `coding-plan.ts` | Credential decryption + coding-plan reset card status/opportunity/use | | `app-update.ts` | ZCode desktop app release check + verified download + bundle swap where the location allows (ADR-0028) | ### `remote/` — Remote access (opt-in via `ZCODE_ACP_REMOTE=1`) | File | Responsibility | | --------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `broadcast.ts` | ClientRegistry + broadcast proxy: notify fans out to all clients; request is first-response-wins with loser `$/cancel_request` | | `config.ts` | ENV parsing (gate, mandatory token, hub/bridge ports) | | `endpoint.ts` | Loopback ACP endpoint (SDK AcpServer transport, port auto-increment) + hub registration/heartbeat | | `file-endpoint.ts` | Read-only `GET /fs/list` + `GET /fs/file` on the loopback server (ADR-0004): Session Root scoping, byte/line windows, streaming | | `status-endpoint.ts` | `GET /status` on the loopback server (ADR-0005): in-memory per-session running state (`pendingTurns` derivation), zero backend RPC | | `session-close-endpoint.ts` | `POST /sessions/{id}/close` on the loopback server (ADR-0006): running-guarded discovery retirement with self-healing re-appearance | | `settings-endpoint.ts` | `settings/*` routes on the loopback server (ADR-0025): the ZCode configuration API, mounted again by the hub behind the token | | `hub-server.ts` | The hub singleton: token auth, instance discovery, byte-level WS proxying, heartbeat pruning, on-demand `?probe=1` liveness, idle exit, version self-upgrade, `GET /api/quota` direct query (ADR-0005), POST close proxy (ADR-0006) | When enabled, the same `AgentApp` serves the stdio editor and a loopback WebSocket endpoint. Every connection (editor or remote) joins the broadcast registry via `trackConnections`, so one turn's notifications reach all clients regardless of who prompted. The bridge registers itself with the machine-level the hub daemon (`zcode-acp hub`, `bin/hub.ts`), which is the only public entry point and holds no session state (see `docs/adr/0002`). The bridge's lifetime still follows the stdio client (ADR-0001); the listener is `unref()`'d so remote clients alone never keep the process alive. Hub upgrades are self-managing: each heartbeat carries the bridge's package version, and a hub that sees a NEWER bridge replies `{ok, restarting}`, exits, and is re-spawned by that bridge from its own (upgraded) `dist/` within a few seconds. Equal, older, or absent versions never trigger a restart — downgrades and mixed-version fleets are fine. Without this handshake a long-lived hub would keep running pre-upgrade code until its 10-minute idle exit. Discovery liveness has two layers: the heartbeat TTL (30s, pruned every 5s) drops bridges that stopped registering — the fallback for hard kills — and `GET /api/instances?probe=1` actively TCP-probes each registered loopback port on demand, so a client refresh gets an honest list with no background probing cost. A probe failure is not a verdict: the first one only marks the instance unhealthy, and ~8s of continuous unreachability (confirmed by a later probe) prunes it — a busy bridge's event loop can stall past the connect timeout while perfectly alive, and evicting it would kick every attached client. Session file access (ADR-0004) rides the same loopback server: the bridge serves read-only `GET /fs/list` + `GET /fs/file` scoped to each session's cwd, and the hub byte-proxies them at `/api/instances/{id}/fs/*` — no new port, and the hub still holds no path semantics. Running status and quota also have plain-HTTP channels (ADR-0005) so clients can poll without an ACP connection: the bridge's `GET /status` (pure in-memory assembly) is proxied at `/api/instances/{id}/status` and echoed coarsely in each heartbeat (`sessions[].status`), while `GET /api/quota` is queried by the hub itself — quota belongs to the machine's credentials, not to any instance, so it works with zero bridges registered. Session close (ADR-0006) is the surface's first write op: `POST /api/instances/{id}/sessions/{sid}/close` retires a conversation from discovery (running-guarded, self-healing if the editor still has it open). ### `settings/` — ZCode configuration management (ADR-0025–0028) The settings API exposes the desktop app's management surface as JSON so a native client can render it: models, skills, MCP servers, hooks, subagents, app usage, coding-plan reset cards, and app updates. All of it is machine-level state under `~/.zcode/` that the desktop app owns, which drives two design choices. First, **the bridge is not the owner of these files** — it is a second writer next to a running desktop app. Every write therefore goes through `file-lock.ts`, a re-implementation of ZCode's own cross-process lock (`.lock` directory + per-holder owner file + stale reclaim), and through `atomic-write.ts`'s read-modify-write that preserves unknown keys, self- validates, backs up, then temp-file + rename (ADR-0026). Without the shared lock a settings write landing during an app write silently discards the app's change. Second, **the API layer is mounted twice from one factory** (`remote/settings-endpoint.ts`): on the bridge's loopback server unauthenticated (exactly like `/status` and `/fs` — any local process can already write these files) and on the hub under `/api/settings/*` behind the token, plus the per-instance proxy `…/settings/*`. Writing the routing once and mounting twice is what keeps the two in sync (ADR-0025). Each write reports an **effect class**: `immediate` for the provider table (absorbed by the backend's ~1s poll) and skills enablement (read live), versus `needs-restart` for MCP servers, hooks, and subagent markdown, which the agent reads once at start. The client either prompts or calls `POST /api/instances/{id}/backend/restart`, which reuses the sandbox arm-flip's cancel-then-respawn path. App usage stats are aggregated locally from the agent database (`~/.zcode/cli/db/db.sqlite`) rather than proxying the backend's `usage/stats` RPC, so usage works with no backend process alive and degrades to `{available: false}` on a machine that never ran an agent (ADR-0027). Coding- plan reset cards are the one irreversible operation: they require the nonce from a fresh status read, are restricted to `account:*` providers, and carry an idempotency key so a retry cannot burn two cards. App updates (ADR-0028) reproduce the desktop app's own discovery half — the same manifest endpoint, platform/channel spelling, and sha512 verification — but stop short of privileged escalation. Whether the bundle can be swapped is probed at install time with `accessSync(dirname(app), W_OK)` rather than assumed: a writable location is replaced directly (old bundle renamed aside first, deleted only after the new one is in place, so a failed install leaves the previous version intact), and anything else is reported as `needs-user-install` with the artifact path so the client can hand the user a one-step instruction. The running app is never killed. No claim is made about which branch a given machine takes — that is what the probe is for. ## Key State Machines ### Turn state ``` subscribe | turn.started | +---------------------+ | model.streaming | | tool.updated | | session.updated | +---------------------+ | +---------------------+ | turn.completed | -> end_turn | turn.failed | -> error | turn.cancelled | -> cancelled | no protocol | | progress (120s) | -> check read watermark | watermark moved | -> forward throttled usage -> defer (alive) | active tool | -> refresh in_progress -> defer (alive) | frozen < 10 min | -> defer decision | frozen >= 10 min | -> fetch reply -> end_turn | | no reply, no output -> max_turn_requests | manual cancel | -> cancelled +---------------------+ ``` Projection polling is a recovery signal, not protocol progress. In particular, `projection.status=running` may be stale and therefore never refreshes the 120-second deadline. Neither is the prompt lock a liveness signal — verified against the Aug-28 app-server, `session/goal show` succeeds mid-turn, and a probe `session/send` is accepted (queued as steer input) while the turn runs; the lock is only held during finalisation. Instead the 15s stall-reconcile reads feed a liveness watermark (`contextUsed`/`totalTokenCount`/`turnCount`/`currentTurnId` from `session/read`): an advancing watermark proves a silently-working turn (typically a sub-agent) and defers the terminal decision indefinitely. Because an upstream ACP client can have a shorter idle deadline than this bridge, the bridge also forwards authoritative progress at most once per minute: a `usage_update` when the watermark advances, or an `in_progress` refresh for a known active tool while its watermark is temporarily quiet. These updates are state-bearing; the bridge never fabricates transcript text as a heartbeat. A watermark frozen for 10 minutes (STALE_FREEZE_MS) still marks the projection as truly stale unless a known foreground tool remains nonterminal. An active tool is direct lifecycle evidence that the turn has not completed, so the bridge keeps waiting and refreshing its existing card. Without an active tool, the turn ends gently (reply fetch first, bounded stop only when nothing was ever delivered). Already-queued events win the deadline race and are consumed first. ### Tool lifecycle ``` scheduled -> started -> progress -> result/error | | | ToolCallNew status=in_progress output (stdoutTail) | (seenToolIds.add) ``` ## Interaction Protocol Negotiation ZCode `interaction/*` requests are routed to different ACP client interaction mechanisms via protocol negotiation: ``` zcode interaction request received │ ├─ Tool auth (interaction/requestPermission) │ └─ Always uses session/request_permission (its native purpose) │ ├─ ExitPlanMode (interaction/requestUserInput + plan_approval) │ ├─ martty attached (hasMarttyClient) → elicitation/create (approve/reject form; │ │ its permission overlay draws only the title — the plan needs the form) │ └─ Otherwise → session/request_permission (editors render toolCall.content │ as full markdown; elicitation form descriptions are plain text there) │ └─ AskUserQuestion (interaction/requestUserInput) ├─ Client supports elicitation.form → elicitation/create (single form) └─ Otherwise → per-question session/request_permission (fallback) ``` **Key**: `server.supportsElicitationForm()` is detected at `initialize` time from `clientCapabilities.elicitation.form` (AskUserQuestion). ExitPlanMode keys on `server.hasMarttyClient()` instead — Zed ≥1.12 declares `elicitation.form` too, but its form descriptions are plain text while its permission popups render `toolCall.content` as markdown, so only martty benefits from the form. Tool auth always goes through request_permission, since that is its native purpose. ## Deferred Notification Mechanism The `available_commands_update` notification must be sent **after** the session response; otherwise the client's session state machine is not yet ready and drops the notification, leaving the `/` completion menu empty. ``` session/new|resume|load handler │ ├─ call newSession()/resumeSession()/loadSession() │ ├─ sendAvailableCommandsDeferred(cx, sid, SLASH_COMMANDS) │ └─ enqueue, send after 50ms (fire-and-forget) │ └─ return response │ └─ after the response is written to stdout, the 50ms timer fires sendAvailableCommands ``` `sendAvailableCommandsDeferred` (`io.ts`) encapsulates the 50ms delay logic, mirroring the Python bridge's `_pending_post_notifs` queue + `_drain_post_notifs` mechanism. ## Mode Reconciliation The session mode can change through four entry points, all of which must notify the editor UI: | Trigger | Path | Notifies UI | | -------------------------------------- | -------------------------------------------------------------- | :---------: | | `session/setMode` request | `extensions.ts:setMode` | yes | | `session/set_config_option` (mode) | `session.ts:setConfigOptionHandler` → `emitConfigOptionUpdate` | yes | | `/mode` slash command | `slash.ts` → `emitConfigOptionUpdate` | yes | | In-turn `EnterPlanMode`/`ExitPlanMode` | reconciled at turn completion | yes | The in-turn path bypasses the bridge entirely, so `prompt()` runs `emitModeIfChanged` (`session.ts`) at turn completion: it re-reads the authoritative mode via `buildModes`, compares against `server.lastMode` (the value last advertised to the client), and emits `current_mode_update` + `config_option_update` when they differ. Failures are swallowed so they cannot break the turn-completion path. ## Prompt-Lock Release on Stop `session/stop` is fire-and-forget, but ZCode has a startup delay: when stop arrives before the turn truly holds the lock the backend ignores it, the turn runs on, and the lock leaks (next `session/send` fails with "A prompt is already running"). `ensureTurnStopped` (`session.ts`) closes this gap. It mirrors the `expectLock:true` strategy from `waitForTurnIdle` (`extensions.ts`): 1. send `session/stop` 2. poll `session/goal show`; first REQUIRE seeing "prompt is running" once (proves the turn started), then wait for it to clear 3. if an 8s grace window elapses without ever seeing the lock, the turn never started (stop caught it in time) or already ended → treat as released 4. hard timeout 30s It is used at every stop site: the cancel check, the stall no-output path, the turn cancelled/failed result, the 120s no-progress timeout, and the `session/send` error path. Never throws (failures only log) so it cannot break the cancel path. ## Process Watchdog `close()` reaps the zcode process group via `process.kill(-pid)`, but only when the bridge exits cleanly enough for its signal handlers to fire (SIGTERM/SIGINT). If the bridge is SIGKILLed (Zed force-kill on reconnect, crash, OOM), the handler never runs and the zcode subprocess group is orphaned. The watchdog (`backend/jsonrpc-child.ts:startWatchdog`) closes that gap. It is a tiny detached child that polls the bridge pid every 2s and, once the bridge is gone, sends SIGKILL to the zcode process group, then exits. It is its own process-group leader and `unref`'d, so it never holds the event loop open and is not part of the zcode group it kills. It self-terminates as soon as the zcode process exits, so a normal shutdown leaves no lingering watchdog. ## Design Decisions ### Why a dual path? | Scenario | Real-time path | Snapshot path | | ---------------------- | -------------- | ------------------------------- | | Normal streaming | Low latency | Must wait for turn end | | Lost events | Loses data | Recovers from snapshot | | Deduplication | seenToolIds | seenMessageIds + markToolSeen() | | Turn-completion triage | Not triggered | PlanUpdate / usage_update | ### Why no polling fallback? ZCode CLI 0.14.5 ~ 0.14.7 used `session/read` polling to emulate streaming. 0.14.8+ introduced `session/subscribe` event push, which has lower latency and is more reliable. This project supports only 0.14.8+ and has removed the polling fallback code. ### Why does ProjectionDiffer need to persist across turns? - `seenMessageIds`: prevents historical messages from being re-emitted after resume - `lastToolStatus` + `seenToolIds`: ensures tool state is not lost across turns - `lastUsage`: avoids duplicate usage_update pushes - `lastPlanSig`: emits only when the plan changes