# @happyvertical/smrt-chat

Chat rooms, threads, and agent sessions with app-controlled tool whitelisting.

## Dev Server

`pnpm --dir packages/chat dev` runs a package-local SvelteKit workbench. The root
route is an interactive chat surface with a dev-only `/api/dev-chat` endpoint:
it uses `@happyvertical/ai` when local provider credentials are present and
falls back to a deterministic local assistant otherwise. `/api/dev-chat-stream`
is its SSE companion (#1936) — the same provider/local-fallback resolution wired
through `createChatStreamHandler` in plain mode, so an embedded `SmrtChatBackend`
client can exercise token streaming locally. `/previews` hosts the shared
component playground entries from `src/svelte/playground.ts`.

The root workbench also has a dev-only voice conversation mode. It reads voice
gateway connection details through `/api/dev-voice/config`, streams browser mic
audio to `WS /ws/voice` as PCM16 mono, appends gateway transcripts/responses to
the chat, and plays returned TTS audio. Exposing
`SMRT_CHAT_DEV_VOICE_GATEWAY_TOKEN` to the browser requires
`SMRT_CHAT_DEV_VOICE_GATEWAY_EXPOSE_TOKEN=true`; keep that local-only.

## Models

Internal models — all mutations go through the membership/owner-checked `ChatService` (S5 #1392) or the voice adapter's binding-checked flow. EVERY `@smrt()` model in this package (ChatRoom, ChatMessage, ChatParticipant, ChatThread, ChatReaction, AgentSession, VoiceSession) has a READ-ONLY generated REST/MCP surface (`list`/`get` only); `create`/`update`/`delete` are intentionally NOT generated so the raw collection routes cannot skip the service-layer authorization. A structural regression test enumerates the registry to assert no chat model exposes a mutating op.

`ChatService` is a CLOSED FACADE (S5 #1392). The raw collections (`rooms`, `messages`, `participants`, `threads`, `agentSessions`, `reactions`, `voiceSessions`) are ES `#private` fields — they are NOT on the public `ChatService` type and the package index does NOT export the collection classes, so a consumer cannot do `chat.messages.create({senderProfileId, role})` / `new ChatParticipantCollection(...)` to mutate around the authorization. The security-sensitive internals (`#writeMessage`, `#emitAgentReply`, `#enrollParticipant`, `#loadActiveSession`, `#requireActiveMembership`, `#requireRoomAdmin`, `#extractToolName`) are ES `#private` too, so they are unreachable at runtime — TypeScript `private` alone is erased and would leave them callable on the prototype. The agent-reply bridge is a `Symbol`-keyed static (not the old enumerable `_runAgentReply`), reachable only by the module-local `sendAgentReply` that holds the non-exported symbol.

- **ChatRoom**: `roomType` (public/private/dm/agent), `status`, `topic`, `maxParticipants`, `lastMessageAt`. Tenant-scoped (required).
- **ChatMessage**: shared by users + agents. `role` (user/assistant/system/tool), `messageType` (text/system/action/file/tool_call/tool_result), `toolCallData` JSON. Unified model — no separate agent message type. Tenant-scoped (required).
- **ChatParticipant**: `role` (owner/admin/member/viewer), `onlineStatus`, `lastReadMessageId`, `isMuted`. Tenant-scoped (required).
- **ChatThread**: `rootMessageId`, `isResolved`, `messageCount`. Created via `ChatService.startThread()` (member-checked). Tenant-scoped (required).
- **ChatReaction**: `messageId`, `profileId`, `emoji`. Added/removed via `ChatService.addReaction()`/`removeReaction()` (member-checked, self-keyed). Tenant-scoped (required).
- **AgentSession**: `agentId` (string ref, not FK), `allowedTools` (JSON string array), `sessionContext` (JSON), `systemPrompt`, limits (`maxTokens`/`maxMessages`/`expiresAt`). Optional tenancy.
- **VoiceSession**: short-lived voice-gateway binding over `(tenant, actorProfileId, personaId, room/thread/agentSession)` plus a persona snapshot, gateway `session_id`, expiry, replay tracking, and metadata. Tenant-scoped (required). Generated surface is read-only; creation and turns go through `createVoiceChatSession()` / `handleVoiceGatewayTurn()`.

## ChatService

Every public write takes an explicit server-supplied `actorProfileId` (the authenticated principal the route injects) — never a caller-controlled `senderProfileId`/`role` (S5 #1392).

Facade: `sendMessage()` (authors as the actor with `role: 'user'`; room-membership-checked; no caller-supplied sender/role and no public membership-skip), `createRoom()` (acting actor becomes owner — no caller-supplied `createdByProfileId`), `startThread()` (member-checked; optional `rootMessageId` bound to the same room+tenant), `addParticipant()`/`removeParticipant()` (owner/admin-checked; self-leave allowed), `updateRoom()` (owner/admin-checked), `addReaction()`/`removeReaction()` (member-checked, self-keyed), `getOrCreateDM()` (actor must be a DM participant), `createAgentSession()` (acting actor becomes the session participant — no caller-supplied `participantProfileId`; the existing-session room lookup is tenant-bound; optional `sessionKey` scopes session identity to a conversation subject so distinct keys get distinct sessions/rooms and a session opened for one subject is never reused/rewritten for another). Tenant-bound read facade (replaces raw-collection reach-ins; consumers apply their own ownership/context checks on the returned rows): `getAgentSession({agentSessionId, tenantId})`, `findActiveAgentSessions({tenantId, agentId, participantProfileId})`, `getThread({threadId, tenantId})`, `listRoomThreads({roomId, actorProfileId, tenantId})` (membership-gated), `getThreadMessages({threadId, actorProfileId, tenantId, limit?})` (membership-gated, chronological), `getRoomMessages({roomId, actorProfileId, tenantId})`/`getRoomForMember(roomId, actorProfileId, tenantId)` (membership-checked reads gated on the server-supplied `actorProfileId`, never a caller-controlled subject id — confused-deputy avoidance; `tenantId` required), `updateAgentSessionConfig()` (owner-checked; `tenantId` mandatory and bound into the lookup). Agent session messaging is split by authority: `sendAgentUserMessage()` (caller `actorProfileId` must be the session participant; always authored as the participant). The agent-authored reply path `sendAgentReply(service, params)` is an exported **function — NOT a `ChatService` method and NOT on the package index**; it is reachable only via the dedicated `@happyvertical/smrt-chat/internal/agent-runtime` subpath (S5 #1392), so only trusted in-process agent-runtime code that explicitly opts into that subpath can author as the agent. It authors as `session.agentId`, accepts an optional same-room/tenant `threadId`, and gates tool calls fail-closed against `allowedTools`. The shared internal persistence path (`writeMessage`) is private — it alone may author an arbitrary profile/role or skip the membership check, and is unreachable from any route; it also validates every supplied `threadId`/`agentSessionId`/`replyToMessageId` belongs to the SAME room AND tenant (tenant/room-bound lookups) before use, rejecting cross-room/cross-tenant references. Auto-creates rooms/sessions/participants via an internal `enrollParticipant`.

## Agent Tool Whitelisting

`allowedTools` is a JSON array controlled by the consuming app. Fail-closed: an empty/unparseable whitelist permits NO tools. The internal `sendAgentReply(service, params)` function enforces the whitelist before emitting any `tool`/`tool_call` message; a caller cannot supply a `senderProfileId`/`role` to post as the agent, and the function is not reachable from the package index.

Principal-bound data discovery and reads are available as `PrincipalTool[]` via
`createDataSurfaceTools()` (re-exported by the chat package). Pass the tools as
`extraTools` to the persona conversation/tool loop. They use the same
fail-closed `allowedTools` offer/execution gates; RBAC, tenant, redaction,
bounded query results, and audit authority remain in the authenticated
`PrincipalRun` supplied by `@happyvertical/smrt-agents`.

## Conversational Harness (L3, #1891)

The "chat with your learning agent" surface — the real agentic runtime for `AgentSession` (the only shipping chat runtime before this was a single-shot completion). This is the new **acyclic `chat → personas` / `chat → agents` / `chat → users` edge**; keep it that way (personas/agents/users never depend back on chat).

- **`runToolLoop(options)`** (`tool-loop.ts`) — a bounded `tool_call → observe → respond` loop. Tools are **manifest operations** of installed packages: `buildManifestToolCatalog({ allowedTools })` reads the `PermissionCatalogService` catalog and keeps only the `(collection, action)` entries named in the persona's allow-list (the **offer gate**; absent/empty ⇒ NO tools). The loop runs inside one `executeAsPrincipal` context, and `invokeManifestTool` executes each op **in-process ("side door")** against `run.context.database` (the RLS tx when Postgres RLS is on), after re-asserting the fail-closed allow-list (`run.assertToolAllowed`) AND the catalog permission (`run.assertOperation`) — the **execution gate**. Bounded by a max-steps ceiling (`DEFAULT_MAX_STEPS = 8`): on the ceiling it disables tools for one final completion so the turn always terminates with text.
- **`runPersonaConversationTurn(options)`** (`persona-conversation.ts`) — binds a conversation to an `AgentPersona`/`ResolvedPersona`: runs as its principal (`runAsUserId`), offers only its `allowedTools`, speaks its instructions (`resolvePersonaInstructions`, layering approved learned directives), and injects its **recalled learning memory** (`personaLearningMemory`, isolated per `memoryScope`) into the system prompt. `bindPersonaToSession()` mirrors the persona's `allowedTools`/instructions onto the `AgentSession` so the chat authoring gate agrees with the loop's offer gate. Authors the reply (and each executed tool) via the internal `sendAgentReply` bridge.
- **Agent orchestration** (L3, #1892) — the loop accepts non-manifest **`extraTools`** (`PrincipalTool[]`, from `@happyvertical/smrt-agents`), gated by the *same* fail-closed allow-list. The standard **`invoke-agent`** tool (`createInvokeAgentTool`, slug `agents.invoke`) lets a conversational agent delegate to a **worker agent under its own principal** — the worker runs via `executeAsPrincipal` as the originating user (never its own authority), the principal is immutable along the chain, and its completion is surfaced back into the conversation. `runPersonaConversationTurn` filters `extraTools` by the persona's `allowedTools` (offer gate); the tool's `execute` re-asserts `assertToolAllowed` (execution gate). See `@happyvertical/smrt-agents` for the delegation envelope + transports.
- **Chat feedback capture** (`chat-feedback.ts`) — `captureChatFeedback()` + `acceptAppliedChange`/`rejectAppliedChange`/`correctResponse`/`rateResponse`/`thumbsUp`/`thumbsDown` write a `Feedback` row (personas) carrying the conversation's **correlation-id**, and (by default) reinforce the persona's learning memory (`reinforceFromFeedback`). So an in-chat reject decays a strategy below the reuse floor and it stops being recalled; a correction supersedes its stored value.

## Voice Gateway Turns (#1910)

Voice is an input mode for the existing persona chat harness, not a separate chat runtime. `createVoiceChatSession()` creates a short-lived `VoiceSession` for an authenticated actor/profile, binding tenant, persona, agent session, room, and optional thread. `handleVoiceGatewayTurn()` resolves that binding from `metadata.voiceSessionId`, checks the gateway's `session_id` and any supplied tenant/profile/persona/session/thread metadata against the server-side binding, persists the transcript through `ChatService.sendAgentUserMessage()`, runs `runPersonaConversationTurn()`, stamps voice/correlation metadata onto the persisted user/assistant/tool messages, records the gateway `turn_id`, and returns the gateway response contract. The Fetch-compatible `createVoiceGatewayTurnHandler()` adds the coarse gateway bearer-token check.

The gateway bearer token proves only "this request came from the gateway"; it never authorizes the end user. The short-lived `VoiceSession` binding is the user/session proof, and untrusted gateway metadata must be validated against that binding before any chat write or tool loop. Tool execution remains fail-closed through the persona allow-list mirrored onto `AgentSession` by `bindPersonaToSession()`.

## Token Streaming (SSE, #1936)

`chat-stream.ts` is the SSE seam for embeddable conversational UIs (first consumer: the Happy chat widget, `animation#5`): a client POSTs the conversation so far and receives a `text/event-stream` of `data: <json>` frames — `token` deltas as the model generates, then a final `done` frame with the message. The wire `ChatStreamEvent` union also declares `emotion` and `control` (#1921 host-page control commands) lanes for forward compatibility; the v1 engine emits `token`/`done`/`error`.

- **`runChatConversationStream({ context, messages })`** — the transport-agnostic engine (an `AsyncGenerator<ChatStreamEvent>`). Dispatches on `context.binding`: **persona-bound** runs the full `runPersonaConversationTurn` with a token sink wired through the tool loop (`onToken` → `ai.chat({ stream: true, onProgress })`), then persists via `ChatService` and emits the persisted message as `done`; **plain/unbound** streams `ai.stream()` directly and emits a synthesized (unpersisted) `done`. Streamed tokens are a live PREVIEW (a tool-call round may narrate before acting); the `done` message is authoritative. Failures surface as an in-band `error` event, never a throw (the 200 has already committed once streaming starts).
- **`createChatStreamHandler({ authorize, allowedOrigins?, allowCredentials? })`** — a Fetch-compatible handler returning `text/event-stream` (mirrors `createVoiceGatewayTurnHandler`). `authorize(request, body)` is the SOLE trust boundary and works exactly like the voice gateway: this module NEVER authorizes from the request's `session` metadata — the app validates the caller (bearer session id / cookie / same-origin) and the claimed ids against the authenticated principal, and returns an already-authorized `ChatStreamContext`. Generation caps (`model`/`maxTokens`/`maxSteps`) live on the context (server-resolved), never on the request. Cross-origin embedding uses the same fail-closed CORS posture as core `_events` (#1861): the `Origin` is echoed only when allow-listed (never `*`), credentials only when opted in.
- **`SmrtChatBackend` (`@happyvertical/smrt-chat/client`)** — the consume side of the same contract: a browser SSE client (`src/client.ts`) that POSTs the conversation and dispatches `token`/`emotion`/`done`/`error` frames to streaming handlers, tolerating heartbeat comments and frames split across chunks. The subpath is BROWSER-SAFE and dependency-free (no server runtime, no workspace imports — keep it that way), and its widget-facing types are structurally identical to `@happyvertical/animation`'s `ChatBackend` contract so an instance plugs straight into the floating chat widget. `src/client.contract.ts` carries the compile-time locks pinning it to `chat-stream.ts`'s `ChatStreamEvent`/`ChatStreamSession` — a NON-test module precisely so `pnpm typecheck` actually enforces them (`tsconfig.typecheck.json` excludes `.test.ts` files, and Vitest transpiles without typechecking). A clean close without a `done` frame surfaces as an error (never an empty reply), and a settled turn cancels the reader so the connection is released promptly.
- **Persona path reuses the harness's own gates unchanged** — persona principal, fail-closed `allowedTools` offer+execution gates, tenant binding. `onToken` is best-effort telemetry threaded through `runToolLoop`; it never changes what the loop persists or authorizes.
- **Custom tools stream via `binding.extraTools`** — the persona binding threads an optional `extraTools?: PrincipalTool[]` down to `runPersonaConversationTurn`, so a *streamed* persona chat can offer non-manifest, service-backed tools (the persona messaging tool `messages.send`, or an assistance-request/lead-ticket tool wrapping a `@smrt({ api:false, mcp:false })` service) and thus *act*, not only answer — matching the non-streaming persona path. It is resolved server-side by `authorize` (trusted), never from request input, and stays fully gated: each tool is filtered by the persona's `allowedTools` (offer gate) and re-asserts the bound principal's authority in `execute` (execution gate). Offering a tool is not authorizing it.

## Gotchas

- **sessionContext, not context**: `context` is reserved for slug scoping. Use `getSessionContext()`/`updateSessionContext()` for agent memory.
- **Agent rooms auto-created**: `roomType: 'agent'`, `maxParticipants` defaults to 2; the agent is enrolled as a member so its replies pass the membership check. `createAgentSession()` re-enrolls the participant AND the agent on the existing-session path, so legacy sessions created before the agent was enrolled self-heal.
- **Per-subject sessions need `sessionKey`**: `createAgentSession()` reuses ANY active session for the same `(agentId, participantProfileId, tenantId)`. Callers that open separate conversations per subject (e.g. one content-editor session per content id) MUST pass a stable `sessionKey` (stored in `sessionContext.__sessionKey`, read via `AgentSession.getSessionKey()`); otherwise a session opened for one subject is reused and its context overwritten for another, surfacing the wrong room/threads (S5 #1392). A keyed create never reuses a keyless/legacy session.
- **Session expiry**: check `isActive()` before allowing messages (expiresAt or limit-based)
- **DM identity**: derived from the deterministic per-tenant `canonicalDmRoomId()` and the authoritative `chat_participants` join, not client metadata; concurrent creates upsert onto one row.
- **Tenant-bound lookups**: membership/session/DM lookups REQUIRE `tenantId` and always bind it into the WHERE clause (`findActiveMembership`/`isActiveMember`/`findActiveSession` take a required `tenantId`; AgentSession's `null` tenant is an explicit bound scope, not "any tenant") so they can never resolve a row from another tenant.