# Optional local streaming ASR side-tap This process gives the conductor text transcripts without touching the direct MiniCPM audio/video WebSocket. Audio is copied to this side path, buffered under hard limits, and finalized only after `input.commit`. If it is slow, full, or offline, MiniCPM conversation continues independently. The service never downloads a model and never launches a separate model server. It binds to `127.0.0.1:8101` and accepts only an explicitly selected provider. A missing provider, endpoint, or model stops startup before a listening socket is opened. Provider URLs are restricted to loopback. The `transformers_local` provider loads only an absolute local Whisper directory before opening its socket when the tools profile is explicitly started; it accepts neither a Hub ID nor remote code. For the Compose sidecar only, a wildcard bind is accepted when both `OMNI_ASR_HOST=0.0.0.0` and `OMNI_ASR_ALLOW_CONTAINER_BIND=1` are set. The reference Compose service publishes no host port and stays on its internal network. Arbitrary interface addresses remain invalid, and the provider URL is still restricted to loopback. ## Wire contract Connect to `ws://127.0.0.1:8101/v1/stream`. Send a JSON `input.start` first: ```json {"type":"input.start","protocol_version":"1.0","session_id":"s1","turn_id":"t1","trace_id":"trace1","sample_rate_hz":16000,"channels":1,"encoding":"pcm_s16le","language":"de"} ``` Then send raw binary, signed 16-bit little-endian PCM frames. Finish with `{"type":"input.commit"}` or discard with `{"type":"input.cancel"}`. Responses are `input.transcript.partial`, `input.transcript.final`, or `error`. The HTTP providers below are final-only; the deterministic mock demonstrates partials. ## Recommended local quality profile The `tools` profile defaults to the in-process local provider. The checkpoint must already exist and the ASR image must already contain its matching ROCm PyTorch build plus the pinned Transformers dependency: ```powershell $env:ASR_IMAGE = "omni-asr:local" $env:ASR_MODEL_PATH = "D:\models\whisper-large-v3-turbo" $env:ASR_MODEL_REVISION = "" docker compose -f .\docker-compose.pipeline.yml -f .\docker-compose.rocm-mi300x.yml config --quiet docker compose -f .\docker-compose.pipeline.yml -f .\docker-compose.rocm-mi300x.yml --profile tools up -d asr ``` Inside Compose the checkpoint is mounted read-only at `/models/asr`; no provider endpoint is configured. `HF_HUB_OFFLINE=1`, `TRANSFORMERS_OFFLINE=1`, `local_files_only=True`, `trust_remote_code=False`, and `pull_policy: never` keep the path offline. PyTorch uses its CUDA-compatible device spelling on ROCm, so the device remains `cuda:0`. The MI300X tools default is explicit BF16 and fails closed if PyTorch does not report BF16 support; it never silently changes precision. Only one local inference is admitted at a time, including after a caller timeout whose underlying GPU thread is still winding down. Equivalent direct-process configuration is: ```powershell $env:HF_HUB_OFFLINE = "1" $env:TRANSFORMERS_OFFLINE = "1" $env:OMNI_ASR_PROVIDER = "transformers_local" $env:OMNI_ASR_MODEL = "D:\models\whisper-large-v3-turbo" $env:OMNI_ASR_DEVICE = "cuda:0" $env:OMNI_ASR_DTYPE = "bfloat16" python -m asr_service ``` The offline tests inject both the pipeline runner and the Transformers/PyTorch loader modules. They check the v4.51 call contract without importing either real ML package and without loading a checkpoint. ### Already-running HTTP alternatives Alternatively, for multilingual speech, use an already-running whisper.cpp server with the `large-v3-turbo` model. The recommendation is about the model/quality-latency tradeoff only; this package makes no claim about a particular AMD GPU or backend. Point OMNI at the exact server endpoint and the exact model identifier you loaded: ```powershell $env:OMNI_ASR_PROVIDER = "whisper_cpp" $env:OMNI_ASR_ENDPOINT = "http://127.0.0.1:8080/inference" $env:OMNI_ASR_MODEL = "large-v3-turbo" python -m asr_service ``` The endpoint must already be serving. The service submits a bounded WAV multipart request at commit; it does not launch whisper.cpp. An already-running local OpenAI-compatible transcription server is configured the same way, using its exact `/v1/audio/transcriptions` URL: ```powershell $env:OMNI_ASR_PROVIDER = "openai" $env:OMNI_ASR_ENDPOINT = "http://127.0.0.1:8090/v1/audio/transcriptions" $env:OMNI_ASR_MODEL = "large-v3-turbo" python -m asr_service ``` `OMNI_ASR_API_KEY` is optional for a loopback server that requires a bearer key. It is never returned by `/healthz`. For an offline protocol check with no model and no socket to a provider: ```powershell $env:OMNI_ASR_PROVIDER = "mock" python -m asr_service ``` ## Bounds | Variable | Default | Purpose | | --- | ---: | --- | | `OMNI_ASR_MAX_CONNECTIONS` | 8 | Hard WebSocket session limit | | `OMNI_ASR_MAX_PROVIDER_CONCURRENCY` | 2 | Hard transcription-call limit | | `OMNI_ASR_MAX_AUDIO_BYTES` | 8388608 | Per-session buffer ceiling | | `OMNI_ASR_MAX_AUDIO_SECONDS` | 90 | Format-aware duration ceiling | | `OMNI_ASR_MAX_FRAME_BYTES` | 262144 | Rejects oversized input frames | | `OMNI_ASR_MAX_RESPONSE_BYTES` | 262144 | Rejects oversized provider replies | | `OMNI_ASR_START_TIMEOUT_S` | 3 | Deadline for `input.start` | | `OMNI_ASR_IDLE_TIMEOUT_S` | 15 | Deadline between frames | | `OMNI_ASR_SESSION_TIMEOUT_S` | 120 | Hard session deadline | | `OMNI_ASR_QUEUE_TIMEOUT_S` | 0.05 | Fail-fast provider queue wait | | `OMNI_ASR_TRANSCRIBE_TIMEOUT_S` | 30 | Final-provider deadline | `transformers_local` additionally accepts `OMNI_ASR_DTYPE=auto`, `float16`, `bfloat16`, or `float32`. `auto` means float32 on CPU and float16 on an accelerator. CPU rejects reduced precision; explicit BF16 is capability-checked. Keeping the queue deadline short is intentional: ASR is a routing side-tap, so it must shed work rather than add latency to the live MiniCPM path.