# Native Pi Voice `/voice` now runs **inside Pi's terminal**, with native microphone capture and speaker playback. There is no browser page, localhost UI, transcript-confirmation button, or separate coding agent. The previous browser prototype has been removed. ## Installed globally on this machine The package is registered in personal Pi settings using: ```sh pi install /home/fufudevbook/Projects/personal/Playground/pi-voice-research ``` Global discovery was verified from `/tmp`: Pi reports `/voice` with `scope: user` and `origin: package`. **Run `/reload` in an existing Pi session, or restart Pi.** Do not also pass `-e` for this package after global installation. The global install references this directory; keep it here, or reinstall from its new location if you move it. ## Controls | Control | Behavior | |---|---| | `/voice` | Start a voice conversation; when running, toggle it off | | `/voice stop` | Stop microphone, speakers, transport and child processes | | `/voice mute` / `/voice unmute` | Stop/restart the microphone capture process; conversation stays connected | | **Ctrl+Alt+V** | Toggle microphone mute | | `/voice steer` | Spoken corrections steer Pi while busy (default) | | `/voice followup` | Spoken requests queue a follow-up while Pi is busy | | `/voice status` | Show state and request audio diagnostics | The widget immediately above Pi's editor shows listening/muted state, microphone/output meters, recent speech, and controls. Ctrl+Alt+V avoids overriding Pi's existing Ctrl+X copy binding. Speak naturally. The voice model handles brief conversation and delegates coding/project/reasoning work to **the current Pi session** automatically. Delegated prompts appear as ordinary Pi user messages. Pi retains its model, context, working directory, tools and permission decisions. Tool-name progress and the final settled answer go back to voice; raw tool output and private thinking do not. Queued follow-ups are not completed until their user message is actually delivered in Pi. Multiple steering requests can join the same run; their IDs are resolved as a batch with one spoken final result. ## Requirements and audio - Pi 1.0.0 (tested), Codex CLI **0.160.0**, signed into a ChatGPT account using `codex login`. - Linux with PipeWire and `pw-cat`; default system microphone and speaker devices. - Node 24 for the tests/setup scripts, Python with venv support, and the package-local audio dependencies. Tested with Python 3.14. Already installed here. To set up another checkout: ```sh npm ci --ignore-scripts --legacy-peer-deps npm run setup:audio pi install /absolute/path/to/pi-voice-research ``` The Python WebRTC worker uses aiortc/PyAV and PipeWire raw PCM; credentials are never supplied to it. A headset is recommended for full-duplex use: this version does **not** add acoustic echo cancellation, and laptop speakers can be picked up by the microphone. Select input/output devices using your normal system audio controls before starting voice. Starting `/voice` explicitly opens the microphone. Muting stops capture and sends silence on the media track. Stopping closes media resources. No microphone/audio/transcript recordings are written by this extension. ## Delegation ownership: fixed before Codex execution The original browser implementation stopped when delegation occurred because stock Codex's `clientManagedHandoffs` flag does not disable its backing coding agent. The pinned `rust-v0.160.0` source routes `HandoffRequested` before notifying clients. This implementation moves the boundary **ahead of that router**: 1. Codex creates the authenticated WebRTC call normally using its existing sign-in. 2. Its experimental sideband-base-URL setting points at a private loopback WebSocket tunnel. 3. The tunnel forwards control traffic to `wss://api.openai.com/v1/live/` but removes `delegation.created` events before Codex can parse them into coding requests. 4. The extracted client delegation ID and text go only to Pi. The native data channel's duplicate is not routed. 5. Pi returns progress/results through the pinned V3 `delegation.context.append` protocol with the original ID. Final results use `channel: speakable`; progress uses `channel: commentary`. UTF-8 appends are limited to 500 bytes per frame, matching the installed runtime. This is not a prompt-only prohibition or kill-after-delegation workaround. The real round-trip test received and completed one delegation while Codex never started a backing turn. An unexpected Codex `turn/started` remains a fail-closed safeguard. A separate empty, ephemeral, read-only Codex thread is used for media transport; transcript-tail flushing is disabled and app-server initiated tool/approval requests are refused. **Version-pinned:** the filter depends on Codex 0.160.0's V3 parser and protocol. Other Codex versions are rejected until their routing/parser behavior is reviewed. Unknown delegation shapes fail closed rather than silently switching coding backends. ## Security and lifecycle - Codex manages sign-in/refresh. The tunnel relays Codex-provided authorization headers in memory only to the fixed OpenAI upstream; it does not read credential files or log credentials. There is no browser or public HTTP API. - The tunnel binds to `127.0.0.1` on an ephemeral port and requires a random 256-bit capability path plus the expected Host. It accepts only the first call ID and rejects ordinary HTTP requests. - `OPENAI_API_KEY` and `OPENAI_API_BASE` are removed from the Codex child environment. No separately billed API-key voice route is substituted. - Speech, delegated text, Pi results, and a small recent-context summary go to the voice service. The startup summary contains the working directory and up to four recent visible user/assistant messages, capped at 8,000 characters; it excludes tool output and thinking. - RPC timeouts, disconnections and startup errors are visible in Pi. Stop, session switch, reload/shutdown and cancelled startup release dedicated processes/sockets. The user's unrelated Codex daemon/session is not terminated. - Subscription availability, quota consumption and billing remain account/service-dependent; this is experimental, not a promise of free/unlimited access. ## Validation `npm test`: **7 passing tests**, covering RPC ordering/timeouts/exit handling, refusal of server tool requests, pre-router delegation filtering, capability rejection, preservation of completion IDs, UTF-8 chunking, native Pi widget/commands, steering/follow-up delivery, result correlation, mute and cancellation cleanup. Real end-to-end test: `evidence/native-result.json`, reproduced with: ```sh python scripts/smoke-native.py --out evidence/native-result.json ``` This explicitly loaded fixture runs **the real Pi TUI in a pseudoterminal**, the real configured Pi model with tools disabled, and the authenticated native WebRTC connection. It uses the archived public prerecorded sample, not the microphone, and suppresses physical playback. Observed: - One delegation → one Pi user message → one Pi agent run → one settled result. - Pi answered the spoken sample; the voice service paraphrased that actual Pi answer. - V3 delegation-context acknowledgment arrived. - Native audio decoded **1,092,480 samples** with nonzero audio energy. - Mute, unmute and stop completed; no test errors or dedicated audio workers remained. A separate local worker test verified default PipeWire capture/playback nodes, capture-process shutdown on mute, restart on unmute, and clean exit without transmitting microphone audio to a remote service. Global extension discovery was verified separately from `/tmp`. **Still needs your physical listening check:** start `/voice` in your terminal and confirm your own microphone and speakers/headset. Decoded PCM and a connected playback node are not proof that a human heard the speakers. Real mid-task steering and acoustic feedback behavior also need a live-user acceptance test; their routing is covered by unit tests, not the prerecorded round trip.