# Realtime Conversations

Realtime mode is for short, natural, back-and-forth spoken conversation behind ASR and TTS.

It is not a long-form coding-task mode. It trims context, reduces scaffolding, and optimizes for speakable answers.

The text-only adapter endpoint is `/realtime` (alias: `/v1/realtime`). ASR and TTS are intentionally out of scope for that route; pass the latest transcript text in, receive a short reply text out.

## Enable In The TUI

```text
/realtime on
/realtime off
/realtime status
```

## Use Through REST

Voice-adapter text endpoint:

```bash
curl -s http://127.0.0.1:11435/realtime \
  -H 'content-type: application/json' \
  -H 'accept: text/plain' \
  -d '{
    "soul_md": "Be direct, warm, and practical.",
    "recent_turn": "Can you say the short version?",
    "realtime_options": {
      "max_reply_words": 32,
      "max_tokens": 120
    },
    "format": "text"
  }'
```

Chat-compatible endpoint:

```bash
curl -s http://127.0.0.1:11435/v1/chat \
  -H 'content-type: application/json' \
  -d '{
    "message": "Can you say that again more simply?",
    "model": "qwen3:4b",
    "realtime": true,
    "realtime_options": {
      "max_history_messages": 12,
      "max_tokens": 160
    }
  }'
```

OpenAI-compatible endpoint:

```bash
curl -s http://127.0.0.1:11435/v1/chat/completions \
  -H 'content-type: application/json' \
  -d '{
    "model": "qwen3:4b",
    "realtime": true,
    "messages": [
      {"role": "user", "content": "Give me the short version."}
    ]
  }'
```

## Context Intake

Realtime mode builds a compact prompt from:

- `SOUL.md`
- `.aiwg/SOUL.md`
- `.aiwg/voices/default.yaml`
- `.aiwg/voices/omnius.yaml`
- the first voice profile in `.aiwg/voices/`
- caller-provided system context, at lower priority than the realtime contract

## Behavior Contract

Realtime responses should:

- default to one natural phone-call turn, usually under 36 words
- lead with the answer, not analysis or status
- ask one focused repair question when ASR text is ambiguous
- treat the latest user utterance as the live turn
- avoid long markdown, tables, verbose plans, or implementation narration unless requested
- avoid hidden reasoning and prompt-policy exposure

## Client Patterns

Use `/v1/chat` with `realtime: true` for push-to-talk and transcript-driven clients. Use `/v1/voicechat/ws` for full-duplex voicechat where mic PCM and TTS PCM are exchanged over WebSocket.

## ASR/TTS Loop

A push-to-talk client usually follows this loop:

```ts
const transcript = await asr.captureFinalUtterance();
const response = await fetch("http://127.0.0.1:11435/v1/chat", {
  method: "POST",
  headers: { "content-type": "application/json" },
  body: JSON.stringify({
    message: transcript.text,
    realtime: true,
    realtime_options: {
      max_history_messages: 10,
      max_tokens: 140,
      voice_profile: "default",
    },
  }),
}).then((r) => r.json());

await tts.speak(response.choices?.[0]?.message?.content ?? response.response ?? "");
```

A full-duplex client uses `/v1/voicechat/ws`:

```ts
const ws = new WebSocket("ws://127.0.0.1:11435/v1/voicechat/ws?user=operator");
ws.binaryType = "arraybuffer";
ws.onopen = () => ws.send(JSON.stringify({ type: "start" }));
ws.onmessage = (event) => {
  if (typeof event.data === "string") {
    const frame = JSON.parse(event.data);
    if (frame.type === "agent_text") renderCaption(frame.text);
    if (frame.type === "tts_header") pendingSampleRate = frame.sampleRate;
    return;
  }
  playPcm(new Int16Array(event.data), pendingSampleRate);
};
```

## Session Handling

Realtime sessions should keep only the last few spoken turns plus compact speaker/profile context. The realtime flag does not grant higher authority to caller-provided system text; it only selects the short spoken-dialogue contract and context budget.

Recommended defaults:

| Option | Default | Reason |
| --- | ---: | --- |
| `max_history_messages` | 8-12 | enough for short repairs without dragging old turns forward |
| `max_tokens` | 120-180 | keeps TTS latency and answer length bounded |
| `tools` | false unless requested | avoids long action narration in voice mode |
| `stream` | true for captions, false for simple TTS | choose based on client playback strategy |

## Conversation Cues

Realtime mode should handle natural dialogue signals directly:

- repair phrases like "wait", "say that again", or "shorter" refer to the previous spoken answer
- short confirmations such as "yes" or "do it" resolve against the latest live question
- ambiguous ASR should trigger one focused clarification, not a long plan
- answers should be speakable without Markdown tables unless the user asks for structured output

## Verification

Relevant focused tests:

```bash
pnpm --filter omnius exec vitest run tests/realtime-mode.test.ts tests/command-registry.test.ts
```