# Event Stream Module Documentation

## Purpose
This module contains the OpenChamber message-stream WebSocket protocol and runtime bridge. It keeps the browser-facing WebSocket transport separate from the upstream OpenCode SSE transport.

## Entrypoints and structure
- `packages/web/server/lib/event-stream/index.js`: public entrypoint re-exporting protocol and runtime helpers.
- `packages/web/server/lib/event-stream/global-hub.js`: shared global upstream SSE hub for server-side subscribers and browser WS fan-out.
- `packages/web/server/lib/event-stream/global-ws-bridge.js`: browser-facing global WS bridge that subscribes clients to the shared global hub.
- `packages/web/server/lib/event-stream/directory-ws-bridge.js`: browser-facing per-directory WS bridge that owns one scoped upstream reader per connection.
- `packages/web/server/lib/event-stream/protocol.js`: path constants, SSE envelope parsing, and WebSocket frame serialization helpers.
- `packages/web/server/lib/event-stream/diff-summary.js`: pure outbound FileDiff helpers. L1 message/session `summary.diffs` keep a slim file list `{ file, status?, additions, deletions }` plus additive `diffCount`/`hasDiffs` (never patch/before/after/from/to). `session.diff` event frames still keep file/status/additions/deletions and drop patch bodies. `summarizeFileDiff(s)` remains for explicit L2-style file lists.
- `packages/web/server/lib/event-stream/reasoning-projection.js`: OpenChamber outbound reasoning projection (`includeReasoning=false`). Shared pure snapshot helpers + per-connection stateful stream filter for HTTP parts, WS send/replay, and SSE proxy parse-filter paths. Never mutates hub/replay objects; never forwards the query param to OpenCode.
- `packages/web/server/lib/event-stream/upstream-reader.js`: reusable upstream SSE reader with event-id tracking, stall recovery, and reconnect handling.
- `packages/web/server/lib/event-stream/runtime.js`: thin WebSocket server runtime for upgrade handling and path dispatch to the global/directory bridges.
- `packages/web/server/lib/event-stream/protocol.test.js`: unit tests for protocol helpers.
- `packages/web/server/lib/event-stream/reasoning-projection.test.js`: unit + 10k-delta performance guards for reasoning projection.
- `packages/web/server/lib/event-stream/upstream-reader.test.js`: unit tests for upstream SSE reader behavior.
- `packages/web/server/lib/event-stream/runtime.test.js`: unit tests for runtime-side broadcaster behavior.

## Module direct exports

The following APIs are exported by their owning modules. `event-stream/index.js` aggregates `createGlobalUiEventBroadcaster`, `createMessageStreamWsRuntime`, `createGlobalMessageStreamHub`, `DEFAULT_UPSTREAM_CONNECT_TIMEOUT_MS`, `DEFAULT_UPSTREAM_STALL_TIMEOUT_MS`, and `UPSTREAM_STALL_TIMEOUT_CONCURRENT_MS`.

### Protocol helpers
- `MESSAGE_STREAM_GLOBAL_WS_PATH`: `/api/global/event/ws`
- `MESSAGE_STREAM_DIRECTORY_WS_PATH`: `/api/event/ws`
- `MESSAGE_STREAM_WS_HEARTBEAT_INTERVAL_MS`: heartbeat interval for browser-facing WS connections.
- `parseSseEventEnvelope(block)`: parses an SSE block into `{ eventId, directory, payload }`.
- `sendMessageStreamWsFrame(socket, payload)`: serializes and sends a JSON WS frame.
- `sendMessageStreamWsEvent(socket, payload, options)`: sends an event frame with optional `eventId`, `directory`, and per-connection `reasoningFilter`. Outbound `session.diff` keeps file/status/additions/deletions only (no full patch bodies). Message/session `summary.diffs` are L1-projected to a slim file list plus additive `diffCount`/`hasDiffs` (patch bodies never leave the host). When `reasoningFilter.projectEvent` returns `null`, the frame is dropped (still counts as a successful send so slow-client tracking is unchanged).

### Reasoning projection helpers (`reasoning-projection.js`)
Import from the module file (not the event-stream barrel). Public surface:
- `shouldIncludeReasoning` / `readIncludeReasoningQuery` / `readIncludeReasoningFromUrl` / `stripIncludeReasoningParam`
- `projectMessagesPayloadForReasoning` / `projectMessagesPayloadWithoutReasoning` — array / `{ records }` / `{ record }` / `{ parts }` only
- `createReasoningOutboundFilter` — per-connection stream filter (version-suffix-aware type match; preserves original `type` on kept events)
- `filterSseBlock` / `createSseBlockSplitter` — SSE disabled path (CR-safe chunk boundaries)

### Runtime helpers
- `createGlobalMessageStreamHub(...)`: creates a shared `/global/event` upstream SSE hub with event/status subscribers and bounded event-id replay.
- `createGlobalUiEventBroadcaster({ sseClients, wsClients, writeSseEvent })`: returns a broadcaster that fans out the same synthetic UI event to SSE and WS clients.
- `createMessageStreamWsRuntime(...)`: mounts the message-stream WS server, upgrade handler, and SSE-to-WS bridge onto the web HTTP server.

### Upstream reader helpers
- `DEFAULT_UPSTREAM_STALL_TIMEOUT_MS`: default idle timeout before an attached upstream SSE fetch is aborted for reconnect.
- `DEFAULT_UPSTREAM_CONNECT_TIMEOUT_MS`: default response-header timeout before an upstream SSE fetch is aborted for reconnect.
- `DEFAULT_UPSTREAM_RECONNECT_DELAY_MS`: default delay between upstream reconnect attempts.
- `createUpstreamSseReader(...)`: creates a start/stop reader for OpenCode SSE streams. The reader parses SSE blocks, tracks the latest `Last-Event-ID`, reconnects after closed or stalled upstream streams, and reports events through callbacks.

## Runtime behavior
- Browser clients connect to the WS endpoints above.
- OpenChamber still fetches OpenCode upstream event streams over SSE.
- The web server creates one shared global message-stream hub. OpenCode watcher side effects and global WS clients subscribe to that hub, so there is one upstream `/global/event` SSE reader for both server-side processing and browser fan-out.
- The global hub keeps a bounded replay buffer keyed by SSE `eventId` so reconnecting browser clients can receive buffered events after their requested `Last-Event-ID`.
- Outbound FileDiff payloads are summarized before hub fan-out/replay and again at WS send, so Relay never carries full patch frames larger than 64KB.
- **Reasoning projection (`includeReasoning=false`)**: per HTTP request or WS/SSE connection. The shared hub and internal subscribers always see full upstream events. Filtering runs only at the browser send boundary (live fan-out **and** replay) and on the direct SSE proxy parse path. HTTP turn-page / reconcile / exact message / official `session.messages` list / transcript-cache reads apply the same part strip. `tokens.reasoning` totals stay. The query param is OpenChamber-only and is stripped before OpenCode upstream calls.
- Global WS reconnect protocol is **replay events → ready barrier**. On initial connect and on each first-ready for a client, the bridge synchronously sends any `replayAfter(lastEventId)` frames first, then emits `{ type: 'ready', scope: 'global' }`. Clients must not start HTTP compensation until they observe ready, so buffered history can merge first.
- Directory WS clients still attach one upstream `/event?directory=...` SSE reader per connection because directory streams are scoped.
- If an upstream SSE connection delays response headers or an attached stream stalls, the reader aborts that upstream fetch and reconnects upstream with `Last-Event-ID`, keeping the browser WS alive when recovery is fast.
- Direct SSE proxy timing uses a 10-second downstream heartbeat, a 20-second upstream idle timeout, a 30-second client reconnect window, and a 40-second transport stale threshold. This order lets upstream recovery end the downstream response before client and transport liveness recovery activate.
- When the shared global upstream reconnects after it was previously ready, the global WS bridge sends a fresh `ready` frame to already-ready browser clients (no additional replay; live fan-out already covers the gap when possible). The browser treats this as a reconnect edge and can run scoped state repair without requiring the browser WS to close.
- Health checks are reserved for initial upstream connect failures and explicit upstream-unavailable responses, not for ordinary stall recovery on an already-established stream.
- Global synthetic events such as `openchamber:session-status`, `openchamber:session-activity`, `openchamber:notification`, and `openchamber:heartbeat` are preserved on the WS path, but heartbeat frames are emitted only while an upstream SSE stream is actively attached.
- Global UI broadcasts are fan-out capable across both SSE and WS clients.
- Each global upstream connection captures a sanitized runtime fence containing the normalized runtime key plus connection generation and derived token. Queue confirmation uses that generation only to validate the connection runtime identity and read current authority by runtime key. Active and paused authorities confirm exact directory, session, and message matches through the queue service; shadow authority skips confirmation while session-index ingestion continues.
- The reusable upstream reader centralizes SSE fetch/parsing/reconnect behavior for the WS runtime and OpenCode watcher. Additional event consumers should move to it only with parity tests for their lifecycle and error semantics.
- Browser transport concerns live in the WS bridge modules; server-side global stream ownership lives in `global-hub.js`.

## Notes for contributors
- Keep protocol helpers pure and small so they can be unit tested without spinning up a server.
- Keep `runtime.js` focused on WebSocket upgrade and endpoint dispatch. Put global browser-client lifecycle in `global-ws-bridge.js`, directory stream lifecycle in `directory-ws-bridge.js`, and upstream stream sharing in `global-hub.js`.
- Do not change upstream OpenCode transport assumptions here; OpenCode remains SSE-based.
- Keep global replay bounded; do not turn it into an unbounded event log.
- Preserve the global reconnect order `replay events → ready`. Reversing it races HTTP compensation against buffered history merge.

## Testing
- Run `vitest run packages/web/server/lib/event-stream/protocol.test.js`
- Run `vitest run packages/web/server/lib/event-stream/reasoning-projection.test.js`
- Run `vitest run packages/web/server/lib/event-stream/upstream-reader.test.js`
- Run `vitest run packages/web/server/lib/event-stream/runtime.test.js`
- Run repo validation before finalizing: `bun run type-check`, `bun run lint`, `bun run build`