# @openbrt/audioctl

Generic WeClawBot audio/voice endpoint control CLI and AI-agent plugin.

It is meant for user-owned agents such as Codex, Claude Code, OpenClaw, Hermes,
WorkBuddy, Gemini CLI, or any shell-capable agent runtime. It is not tied to one
speaker brand. Any ESP32/Linux audio or voice endpoint that implements the
WeClawBot MQTT/app envelope can use the same package. Music playback is one app
on top of the endpoint; the same VM/capability model can later host TTS, voice
remote control, smart-home control, and voice interaction apps.

## Tell your agent

```text
帮我在音箱建立一个工作 BGM 歌单，持续播放。请使用 @openbrt/audioctl 最新版。
```

Agent contract:

- If already bound, inspect the device and run the high-level task directly.
- If not bound, ask the user only for the short binding code, then bind and continue.
- Bluetooth/Wi-Fi provisioning is only for getting the device online and binding
  it to weclawbot.link. It must not decide whether the speaker is used for
  NetEase, Spotify, voice remote control, smart-home control, or anything else.
- Preferred BYOA flow: use a legal music platform API in the agent environment
  or another user-approved source to build a playable queue, then push it with
  `audioctl queue QUEUE.json --play --json`. The plugin suggests sources; the
  agent and user choose. NetEase Cloud Music Open Platform can be used this way
  to request the highest available quality URLs. Spotify should be driven through
  official Spotify Connect/Web Playback SDK or existing-user playback control;
  do not assume the Spotify Web API gives full-track direct audio URLs.
- If music-platform authorization is missing, use the current agent's official
  connector, OAuth flow, or secret store for that platform. Bluetooth/Wi-Fi
  provisioning is not a music-platform authorization entry and is not a
  product-purpose decision point.
- Never ask the user to paste AppSecret, PrivateKey, accessToken, refreshToken,
  Wi-Fi passwords, MQTT credentials, or other secrets into chat.

## Install and bind

```bash
npm install -g @openbrt/audioctl
audioctl bind 123456 --name codex
audioctl doctor --online
audioctl inspect
```

One-shot usage:

```bash
npm exec --package @openbrt/audioctl -- audioctl bind 123456 --name codex
```

Credentials are saved in an audioctl-managed local config file. Agents should
not inspect or parse that file; use `audioctl bind`, `doctor`, `inspect`, and
the control commands instead. `audioctl` serializes concurrent commands that use
the same local MQTT credentials, because the broker may require an exact MQTT
client id.
Agent-facing status is intentionally hardware-abstracted as
`audio_voice_ambient_endpoint`; agents should not infer or search for a device
brand/model/vendor from screenshots, BLE names, logs, or prior context.
MQTT is a control plane and deployment path, not a realtime feedback data plane:
agents should not read MQTT credentials, import `mqtt`, keep persistent MQTT
connections, or run BPM/while/sleep loops that repeatedly publish LED/light
commands. Continuous effects must be started as a device-local light timeline,
VM app, or compatibility feedback effect, then left to run on the device.

## Commands

```bash
audioctl play
audioctl pause
audioctl toggle
audioctl next
audioctl previous
audioctl volume 48
audioctl volumeup
audioctl volumedown
audioctl status
audioctl inspect --json
audioctl light describe --json
audioctl light set all --color amber --brightness 0.35
audioctl light timeline ./hero.light.json
audioctl light bind ./hero.light-bindings.json
audioctl light clear
audioctl wakeword status --json
audioctl wakeword configure ./wakeword.json --json
audioctl wakeword bind ./wakeword-bindings.json --json
audioctl wakeword trigger work_mode --source agent --json
audioctl firmware check --json
audioctl firmware update --yes --json
audioctl report "center LED active=true but user cannot see it" --kind capability_mismatch --area light --include-inspect --json
audioctl feedback signal ready
audioctl feedback flash cyan --repeat 2
audioctl feedback volume 66
audioctl feedback beep
audioctl feedback clear
audioctl queue ./work-bgm.queue.json --play
audioctl workbgm --play
audioctl rule ./my-playlist-rule.json
audioctl deepnight
audioctl prompt 123456 --agent codex --alias 视听房音箱
```

`inspect` asks the device for a redacted configuration/status snapshot over the
existing MQTT channel. Agent scripts can read Wi-Fi connection state, BLE
provisioning state, wake-word capability, playback, VM/app status, firmware
runtime version, and health without receiving Wi-Fi passwords, MQTT
credentials, or music-platform secrets.

`wakeword` is the logical voice-trigger binding layer. It does not train or
replace the low-level acoustic wake model. It maps existing device wake events,
ASR phrases, or agent-injected test phrases to VM/app events or safe local
actions. Agents should read `audioctl inspect --json` first and use reported
`wakeword.commands` / `voice.wakeword.*` capabilities, not product-specific
vendor research.

```bash
audioctl wakeword status --json
audioctl wakeword configure ./wakeword.json --json
audioctl wakeword bind ./wakeword-bindings.json --json
audioctl wakeword trigger work_mode --source agent --json
audioctl wakeword clear --json
```

Example wakeword config:

```json
{
  "schema": "weclawbot.wakeword.config.v1",
  "mode": "logical_voice_trigger",
  "language": "zh",
  "phrases": [
    { "id": "work_mode", "text": "开始工作", "aliases": ["工作模式"] }
  ],
  "bindings": [
    {
      "phrase_id": "work_mode",
      "action": "app.event",
      "feedback": ["feedback.beep", "feedback.signal:wake"]
    }
  ]
}
```

`firmware check` reads the official firmware index from
`https://weclawbot.link/firmware/audio/manifest.json`, compares it with
`inspect.firmware.version`, and reports whether a host/runtime update is
available. `firmware update` is intentionally gated: without `--yes` it only
prints the update plan and `confirmation_required`. An agent must show the
version, source and notes to the user first, then run `audioctl firmware update
--yes --json` only after the user explicitly confirms. Firmware updates are for
the device host/runtime and VM manager; they do not erase Wi-Fi, MQTT binding,
music-platform settings, queues, rules, or installed VM app slots.

```bash
audioctl firmware check --json
audioctl firmware update --json        # dry plan, no mutation
audioctl firmware update --yes --json  # runs only after explicit user approval
```

`light` controls expose programmable visual surfaces. Agents should use
`audioctl light describe --json` to inspect surfaces such as `all`, `ring`, and
`center`, plus event sources and runtime limits. For non-trivial visuals, build
a `weclawbot.light.timeline.v1` file and optionally a
`weclawbot.light.bind.v1` file so the device can start/stop visuals on local
events such as `media.playing`, `media.paused`, and `media.ended`.

```bash
audioctl light describe --json
audioctl light timeline ./hero.light.json
audioctl light bind ./hero.light-bindings.json
audioctl light clear
```

Example timeline:

```json
{
  "schema": "weclawbot.light.timeline.v1",
  "id": "hero_ambient",
  "loop": true,
  "stop_on": ["media.paused", "media.ended", "media.error"],
  "duration_ms": 3750,
  "tracks": [
    {
      "surface": "ring",
      "keyframes": [
        { "t": 0, "color": "#ff6000", "brightness": 0.18 },
        { "t": 470, "color": "#ff6000", "brightness": 0.55 },
        { "t": 940, "color": "#301800", "brightness": 0.12 }
      ]
    },
    {
      "surface": "center",
      "keyframes": [
        { "t": 0, "color": "#ffd080", "brightness": 0.08 },
        { "t": 1875, "color": "#ffd080", "brightness": 0.40 },
        { "t": 3750, "color": "#301800", "brightness": 0.05 }
      ]
    }
  ]
}
```

`feedback`/`led` controls expose simple semantic device feedback. Agents should prefer
semantic signals (`ready`, `busy`, `success`, `error`, `playing`, `paused`,
`volume_step_up`, `volume_step_down`, `button_press`) so each hardware adapter
can map them to its native LED ring, small screen, haptic motor, or speaker
prompt. Explicit LED controls are also available:

```bash
audioctl feedback signal playing
audioctl feedback flash '#00ffff' --repeat 2
audioctl feedback volume 42
audioctl feedback beep
audioctl feedback clear
audioctl led flash amber --repeat 1
```

`feedback effect` is a compatibility/demo shortcut for old agents. New agent
work should prefer `light describe` + `light timeline` + `light bind`, because
named effects such as `music_beat`, `pulse`, or `breathing` do not describe the
actual hardware topology. `feedback effect` requires an explicit effect name;
calling it with no name is rejected.

Physical keys/touch surfaces are reported as VM capabilities such as
`input.touch.read`, `input.gesture.read`, and `input.button.bind`. A VM app can
declare those capabilities and bind local gestures to safe device actions. The
agent should derive the binding from the user's goal and the reported
capabilities, instead of assuming a fixed product-specific layout. These
bindings are device-local for low latency and do not require the agent to be
online for every tap.

`queue --play` is the generic agent-built music path. The agent uses the user's
authorized music provider outside the speaker, writes a
`weclawbot.playable_queue.v1`/`schemaVersion: 1` queue containing only safe media
URLs and redacted metadata, then pushes it to the device. The queue must not
contain platform private keys, OAuth tokens, cookies, custom request headers, or
Wi-Fi/MQTT credentials. Expiring media URLs are allowed; refreshing them is the
agent/app's job.

`workbgm --play` installs a persistent `work-bgm-autopilot` VM app on the audio
device, writes the bundled 工作 BGM playlist rule through the device-side music
adapter, requests a refresh, and starts playback once. The app expresses a
continuous BGM intent and respects later user-initiated pause/resume controls.
If the device reports `netease_auth_refresh_failed`, that only means the legacy
device-side NetEase runtime cannot refresh itself. The better agent behavior is
to switch to the BYOA queue/app path: obtain the user's music-platform identity
in the agent environment, build a fresh queue, and push it with `audioctl queue`.
Only if the user explicitly wants the legacy device-side NetEase runtime should
the agent say a separate device-side music-account settings entry is required.
That entry must be separate from Bluetooth/Wi-Fi provisioning.

`deepnight` pushes the bundled “深夜工作” playlist rule and asks the device to
refresh/play according to its local runtime. The package never contains music
platform private keys.

## Control envelope

The MQTT control message is:

```json
{
  "schema": "weclawbot.control.v1",
  "id": "audio_<uuid>",
  "kind": "music_command",
  "music": {
    "command": "play"
  }
}
```

Supported commands include `play`, `pause`, `toggle`, `next`, `previous`,
`seek`, `set_volume`, `volume_up`, `volume_down`, `status`, and `set_rule`.

Feedback messages use the same MQTT envelope:

```json
{
  "schema": "weclawbot.control.v1",
  "id": "feedback_<uuid>",
  "kind": "feedback_command",
  "feedback": {
    "command": "flash",
    "color": { "red": 0, "green": 255, "blue": 255, "alpha": 255 },
    "repeat": 2
  }
}
```

Supported feedback commands are `signal`, `flash`, `volume`, `effect`, `beep`,
and `clear`. Devices that do not report a matching `feedback.*` capability
should reject the command instead of pretending to support it.

Light messages use a distinct command envelope:

```json
{
  "schema": "weclawbot.control.v1",
  "id": "light_<uuid>",
  "kind": "light_command",
  "light": {
    "command": "timeline",
    "timeline": {
      "schema": "weclawbot.light.timeline.v1",
      "id": "custom_visual",
      "loop": true,
      "stop_on": ["media.paused", "media.ended"],
      "duration_ms": 2000,
      "tracks": [
        {
          "surface": "all",
          "keyframes": [
            { "t": 0, "color": "#202020", "brightness": 0.05 },
            { "t": 1000, "color": "#ff6000", "brightness": 0.5 }
          ]
        }
      ]
    }
  }
}
```

Supported light commands are `describe`, `set`, `timeline`, `bind`, and
`clear`. Continuous light timing and event reactions run on the device VM.

## Agent feedback reports

`audioctl report` is the agent-to-maintainer feedback channel for issues,
suggestions, capability mismatches, UX gaps, and documentation gaps. It writes a
redacted JSON report to `~/WeClawBot/agent-feedback` by default, or to
`WEC_AUDIO_REPORTS_DIR` / `--reports-dir DIR` when set.

User observation wins over status. A device returning `applied`, `active=true`,
or a capability entry only means the runtime accepted the command; it does not
prove the user can physically see, hear, or feel the result. If a user reports
that a center LED is not visible, a touch gesture is unreliable, or a beep is not
audible, agents should fall back to a confirmed visible/audible/controllable
behavior, then create a local report:

```bash
audioctl report "center LED surface is reported active but not visually observable by the user" \
  --kind capability_mismatch \
  --area light \
  --observed "light_active=true, user still cannot see center LED" \
  --expected "visible center-light cue or capability should be marked unavailable" \
  --include-inspect \
  --json
```

Reports are local by default. Agents must ask the user before posting a report
to a public issue tracker, chat, email, or any other external system. Reports
must not include MQTT credentials, music-platform tokens, Wi-Fi passwords, or
other secrets.

Wakeword messages configure the logical voice-trigger layer:

```json
{
  "schema": "weclawbot.control.v1",
  "id": "wakeword_<uuid>",
  "kind": "wakeword_command",
  "wakeword": {
    "command": "configure",
    "config": {
      "schema": "weclawbot.wakeword.config.v1",
      "mode": "logical_voice_trigger",
      "phrases": [
        { "id": "work_mode", "text": "开始工作" }
      ],
      "bindings": [
        { "phrase_id": "work_mode", "action": "app.event" }
      ]
    }
  }
}
```

Supported wakeword commands are `status`, `configure`, `bind`, `trigger`, and
`clear`. This is a VM/app event binding layer, not low-level acoustic-model
training.

Firmware update messages also use the same MQTT envelope, but carry only an
official release descriptor. The device downloads the artifact itself from
`weclawbot.link`, enforces HTTPS, checks SHA-256 and byte size, installs from a
staging directory, restarts local services, and reports health through
`inspect`.

```json
{
  "schema": "weclawbot.control.v1",
  "id": "firmware_<uuid>",
  "kind": "firmware_update",
  "firmware": {
    "schema": "weclawbot.firmware.update.v1",
    "confirmed_by_user": true,
    "release": {
      "schema": "weclawbot.firmware.release.v1",
      "device_class": "audio_voice_ambient_endpoint",
      "runtime": "rokid_music_runtime",
      "version": "0.1.2",
      "channel": "stable",
      "artifact": {
        "url": "https://weclawbot.link/firmware/audio/rokid-music-runtime-0.1.2.tar",
        "sha256": "<64 hex chars>",
        "bytes": 123456
      }
    }
  }
}
```