# @microsoft/voice-widget-embed

The self-contained **CDN bundle** — the one-line `<script>` entry that auto-registers the
`<voice-agent>` custom element. Bundles the supported UI [`@microsoft/voice-widget-ui`](https://www.npmjs.com/package/@microsoft/voice-widget-ui)
(which in turn uses the headless [`@microsoft/voice-widget`](https://www.npmjs.com/package/@microsoft/voice-widget) core) plus the Voice Live
provider, so a single tag embeds a working voice agent.

```html
<script src="https://cdn.jsdelivr.net/npm/@microsoft/voice-widget-embed" async></script>
<voice-agent
  provider="voicelive"
  auth-endpoint="https://your-broker.example.com/session"
  provider-config='{"targetType":"model","model":"gpt-realtime"}'
></voice-agent>
```

Loading the bundle imports [`@microsoft/voice-widget-provider-voicelive`](https://www.npmjs.com/package/@microsoft/voice-widget-provider-voicelive) (which
self-registers `"voicelive"`) and calls `register()` to define `<voice-agent>`. On the user's first
gesture the widget acquires a session (POST the SDP offer to `auth-endpoint`) and connects over
WebRTC. On stop or terminal disconnect it releases browser media and ends the broker control
session; **Start a call** creates a fresh SDP session rather than resuming the previous conversation.

## Attributes

> The attributes below are the embed's **stable, supported API** (semver-guarded). For theming
> (`--va-*` variables) and internationalization, see [`@microsoft/voice-widget-ui`](https://www.npmjs.com/package/@microsoft/voice-widget-ui).
> Attributes listed as "not read yet" are **reserved** — they become supported in a future
> release.

| Attribute | Meaning |
| --- | --- |
| `provider` | **required** — provider name (e.g. `voicelive`) |
| `provider-config` | provider config as a JSON **object** — its shape depends on the `provider`. For the built-in `voicelive`: **model mode** `{"targetType":"model","model":"gpt-realtime"}` or **agent mode** `{"targetType":"agent","agentName":"…","projectName":"…"}` (optional `voice`, `agentVersion`, VAD, … pass through — see [`@microsoft/voice-widget-provider-voicelive`](https://www.npmjs.com/package/@microsoft/voice-widget-provider-voicelive)). A third-party provider defines its own shape. One of `provider-config` / `widget-id`. |
| `widget-id` | broker resolves stored config server-side from this id (alternative to inline `provider-config`). The page carries only the id; your broker maps it to the real config. See the [widget-id example](https://github.com/microsoft/voice-widget/blob/main/examples/broker-node/widget-id/). One of `provider-config` / `widget-id`. |
| `auth-endpoint` | optional neutral session endpoint; the adapter adds provider-specific fields (Voice Live adds `sdpOffer`) |
| `fetch-credentials` | credentials mode for the `auth-endpoint` fetch: `include` (default — sends cookies for same-domain cookie auth), `omit`, or `same-origin` |
| `variant` | `bar` / `compact` / `full` |
| `placement` | `bottom-right` / `bottom-left` / `top-right` / `top-left` / `inline` |
| `start-label` | nonblank trigger text; blank values use the built-in default |
| `default-expanded` | start expanded |
| `mute-button` | show the mic mute button — **opt-in, off by default**; add the bare attribute (or `mute-button="true"`) to enable |
| `show-transcript` | render the built-in live transcript — **opt-in, off by default** |
| `text-input` | render the typed-message form — **opt-in, off by default**; enabled while connected |
| `telemetry-console` | mirror structured telemetry events to `console.debug` — **opt-in, off by default** (a local-dev sink; see [Telemetry](#telemetry--the-voice-agenttelemetry-event)) |
| `theme` | color scheme: `auto` (default — follows the OS via `prefers-color-scheme`) / `light` / `dark`; observed live |
| `accent-color` | any CSS color for the accent (buttons, focus ring). Sets both accent stops to that color for a solid fill; falls back to the brand default if omitted or invalid; observed live |
| `orb-palette` | brand palette for the **animated orb**, as a JSON object `{ "base", "highlight" }` of CSS colors — e.g. `'{"base":"#0a5a4e","highlight":"#00e0a0"}'`. **Independent of `accent-color`** (see below); invalid or partial input falls back to the default orb; observed live |
| `lang-strings` | i18n text overrides as a JSON **object**, merged over the built-in English strings — e.g. `'{"idle":"Habla con el asistente","startCall":"Iniciar llamada"}'`. Keys: `idle`, `connecting`, `listening`, `thinking`, `speaking`, `connected`, `disconnected`, `error`, `startCall`, `endCall`, `expandAssistant`, `minimize`, `mute`, `unmute`, `panelLabel`. Unknown keys are ignored; a nonblank `start-label` still wins the trigger text; observed live |
| `ui-locale` | UI-locale metadata as a JSON **object** `{ "lang"?, "dir"?, "strings"? }` — sets the host `lang`/`dir` and string overrides in one update, for hosts driving locale from their own i18n runtime. `dir` is `ltr`/`rtl`/`auto`; `strings` uses the `lang-strings` keys. Only the provided fields apply; observed live |

Reserved for a future release — `avatar` and `dismissible` — are not read yet.

### When the widget doesn't appear

If a required attribute is missing or malformed, the element **cannot mount** and
nothing renders — it deliberately does not paint fallback text over your page.
The failure is reported three ways so it is still easy to find:

| Where | What you get |
| --- | --- |
| `voice-agent:error` event | `detail` is `{ code, message, recoverable }` — the same channel as session errors, so existing error handling catches setup failures too. `code` is `"mount_failed"` (`recoverable: false`) for a bad attribute, or `"provider_not_registered"` (`recoverable: true`) when a named `provider` never registers (see below) |
| `data-voice-agent-error` attribute | the reason, readable straight from devtools with no script wiring |
| console | `[voice-agent] failed to mount: …`, or the waiting-for-provider warning |

```js
document.addEventListener("voice-agent:error", (e) => {
  if (e.detail.code === "mount_failed") console.warn("check your attributes:", e.detail.message);
  if (e.detail.code === "provider_not_registered") console.warn("provider never showed up:", e.detail.message);
});
```

**A named but not-yet-registered `provider` is not an error — at first.** A provider can
self-register from a separate `<script>` that has not run yet (script order is not
guaranteed), so `<voice-agent>` waits for it instead of failing. While waiting, the element
sets a `data-voice-agent-awaiting-provider="<name>"` attribute (readable from devtools;
removed once it mounts or is removed from the page) and stays silent, because a slow
provider script is normal.

If the provider still has not registered **10 seconds** later, the wait reports itself on all
three channels above — a `voice-agent:error` event with `code: "provider_not_registered"` and
`recoverable: true`, a matching `data-voice-agent-error` attribute, and a console warning —
each naming the provider it is waiting for and which providers **are** registered. That is
the usual symptom of a misspelled `provider` value or a missing provider `<script>`.

The element keeps waiting after that report (hence `recoverable: true`, and hence
`data-voice-agent-awaiting-provider` stays set): if the provider does eventually register, the
element mounts normally and **both** attributes are cleared.

The most common cause is supplying neither `provider-config` nor `widget-id`.

### Transcript and typed messages

```html
<voice-agent
  provider="voicelive"
  auth-endpoint="/api/session"
  provider-config='{"targetType":"model","model":"gpt-realtime"}'
  show-transcript
  text-input
></voice-agent>
```

Both features are optional and independent. Partial agent transcript events update one in-progress
bubble; the final event replaces it. The transcript retains the latest 100 messages and resets on a
new call. Typed input is sent as a Realtime user message and appears in the same transcript.

### Theming & i18n

```html
<voice-agent
  provider="voicelive"
  auth-endpoint="/api/session"
  provider-config='{"targetType":"model","model":"gpt-realtime"}'
  theme="dark"
  accent-color="#0f6cbd"
  lang-strings='{"idle":"Habla con el asistente","startCall":"Iniciar llamada","endCall":"Colgar"}'
></voice-agent>
```

`theme` and `accent-color` are neutral, cross-provider theming knobs; for deeper restyling,
override the `--va-*` CSS variables directly (see [`@microsoft/voice-widget-ui`](https://www.npmjs.com/package/@microsoft/voice-widget-ui)).
`theme="auto"` tracks the OS light/dark setting live.

#### Control accent vs. orb palette

`accent-color` and `orb-palette` are **independent** knobs:

- **`accent-color`** styles the *controls* — the trigger pill, call button, focus rings, and the
  user's transcript bubbles. It never recolors the orb.
- **`orb-palette`** styles only the *animated orb*. You give two brand anchors — `base` (the
  sphere/body) and `highlight` (the crests and glow) — and the full orb gradient and wave colors
  are derived from them while preserving the original animation and gloss.

Use them together to, say, keep a neutral accent while giving the orb a distinct brand color:

```html
<voice-agent
  provider="voicelive"
  auth-endpoint="/api/session"
  provider-config='{"targetType":"model","model":"gpt-realtime"}'
  accent-color="#0f6cbd"
  orb-palette='{"base":"#0a5a4e","highlight":"#00e0a0"}'
></voice-agent>
```

An invalid or partial `orb-palette` (e.g. only `base`) falls back to the default orb rather than
throwing, so a typo never breaks the widget.

Change any of the UI attributes without removing the element or restarting its active session:

```js
const widget = document.querySelector("voice-agent");
widget.setAttribute("theme", "light");
widget.setAttribute("accent-color", "#2899f5");
widget.setAttribute("orb-palette", JSON.stringify({ base: "#7a1030", highlight: "#ff8a5c" }));
widget.setAttribute("lang-strings", JSON.stringify({ startCall: "Iniciar llamada" }));
```

Removing an attribute restores its default (`theme` returns to `auto`, accent overrides are cleared,
and strings return to English defaults while a nonblank `start-label` still wins the idle text).
An unsupported live theme also resolves to `auto`, and an invalid accent clears the override,
matching a fresh mount with those attribute values. Invalid live `lang-strings` JSON is logged and
leaves the current strings unchanged because it cannot be parsed safely.

To switch language, direction, and strings together — the typical move when a host drives locale from
its own i18n runtime — set `ui-locale` (or call the element's `setUiLocale()` method, preferred for
frequent in-call updates). Both keep the active session, mic, and connection intact:

```js
const widget = document.querySelector("voice-agent");
widget.setUiLocale({ lang: "ar", dir: "rtl", strings: { endCall: "إنهاء المكالمة" } });
// or declaratively:
widget.setAttribute("ui-locale", '{"lang":"ar","dir":"rtl","strings":{"endCall":"إنهاء المكالمة"}}');
```

`lang`/`dir` land on the widget host so screen readers announce the new language and the panel lays out
in the right direction; strings re-resolve from English defaults so an omitted key never keeps the prior
language. Invalid live `ui-locale` JSON is logged and leaves the current locale unchanged.

### Model vs. agent target

`provider-config` selects the Voice Live target:

```html
<!-- Model mode: a Realtime model. -->
<voice-agent provider="voicelive" auth-endpoint="…"
  provider-config='{"targetType":"model","model":"gpt-realtime"}'></voice-agent>

<!-- Agent mode: a voice-enabled Foundry agent (Entra-only; the broker holds the credential). -->
<voice-agent provider="voicelive" auth-endpoint="…"
  provider-config='{"targetType":"agent","agentName":"support-bot","projectName":"contoso-prod"}'></voice-agent>
```

The widget forwards `provider-config` to your broker unchanged; the broker resolves the target
(model or agent) and holds the credential. Agent mode is **Entra-only**. Optional fields (voice,
language, VAD, etc.) are passed through too — see
[`@microsoft/voice-widget-provider-voicelive`](https://www.npmjs.com/package/@microsoft/voice-widget-provider-voicelive) for the full config shape.

## Programmatic / advanced

The package ships two builds, and which one you use decides how you load it:

| Build | File | How to load |
| --- | --- | --- |
| **global (IIFE)** | `dist/index.global.js` — the `jsdelivr`/`unpkg` entry | a plain `<script src="…">` tag. Self-contained (it inlines its dependencies, because there is no module resolver on the page) and exposes `window.VoiceAgent` |
| **ESM** | `dist/index.js` — the `import` entry | `import` it **through a bundler**. It imports its dependencies by name rather than inlining them, so the page ends up with exactly one provider registry shared between the widget and any provider you register. Those bare specifiers are also why a browser-native `<script type="module">` cannot load this file directly — use the global build for that |

```js
import {
  register,
  mountVoiceAgent,
  registerProvider,
  createProvider,
  hasProvider,
} from "@microsoft/voice-widget-embed";
```

- `register()` — define `<voice-agent>` (idempotent; auto-called on load).
- `mountVoiceAgent(target, opts)` — mount programmatically, accepting values that can't be expressed
  as attributes (e.g. `clientTools`). See [`@microsoft/voice-widget-ui`](https://www.npmjs.com/package/@microsoft/voice-widget-ui).
- `registerProvider(name, factory)` — register the factory that creates one adapter per widget.
- `createProvider(name)` — create a fresh adapter for advanced direct use.
- `hasProvider(name)` — check whether a provider name is registered.
- `onProviderRegistered(listener)` — observe provider registrations as they happen; returns an
  unsubscribe. Not replayed for providers already registered, so check `hasProvider` first.

The same names are available on `window.VoiceAgent` when the global build is loaded — that is how
a third-party provider script registers itself on a page with no bundler.

### Bring your own provider

The default entry includes Voice Live. An application that supplies its own provider can omit that
code:

```ts
import "@microsoft/voice-widget-embed/provider-neutral";
import "@your-scope/voice-widget-provider-yourvendor";
```

For a no-bundler page, load `dist/provider-neutral.global.js` followed by the provider's IIFE. That
global still exposes `window.VoiceAgent`, including `registerProvider`, `createProvider`, and
`hasProvider`, but registers no provider itself. `createProvider(name)` creates an independent
adapter instance for advanced direct use.

## Client-side tools — the `voice-agent:call` event

The agent can call **functions the host page registers**. HTML attributes can't carry functions, so
the declarative element exposes a **pre-connect event**: `<voice-agent>` dispatches
**`voice-agent:call`** before it connects, and the host mutates `event.detail.config.clientTools`.

```html
<!-- Attach the listener BEFORE the element upgrades. `voice-agent:call` normally fires
     synchronously from connectedCallback, so a listener added after the bundle has
     registered the element has already missed it. (If `provider` names a provider
     that has not self-registered yet, the element waits and the event instead fires
     the moment that provider registers — see "Ordering matters" below.) -->
<script>
  document.addEventListener("voice-agent:call", (e) => {
    e.detail.config.clientTools = {
      // name (case-sensitive) must match the tool declared to the model on the session
      addToCart: ({ productName }) => {
        addItem(productName);
        return { ok: true }; // JSON return is handed back to the agent
      },
    };
  });
</script>

<voice-agent provider="voicelive" auth-endpoint="…" provider-config='{"targetType":"model","model":"gpt-realtime"}'></voice-agent>
<!-- The global (IIFE) build: self-contained, so a plain <script> tag can load it
     directly. `dist/index.js` is ESM with bare imports and needs a bundler — see
     "Programmatic / advanced". -->
<script src="/path/to/voice-agent-embed.global.js"></script>
```

**Ordering matters.** `<voice-agent>` dispatches `voice-agent:call` from its
`connectedCallback` — synchronously, the moment the element upgrades — as long as its
`provider` is already registered, which is the common case (a bundled first-party provider,
or any provider script that ran before the element upgraded). So register the listener
first. The event is `bubbles: true` / `composed: true`, so a `document`-level listener
added before the bundle loads (as above) catches it wherever the element sits. Attaching to
the element in a `<script>` that runs *after* an already-registered bundle upgraded it is
too late — the handler never fires and no tools are injected.

If `provider` names a provider that has not self-registered yet (e.g. its `<script>` has not
run — async script order is not guaranteed), the element instead waits and dispatches
`voice-agent:call` later, the moment that provider registers. A `document`-level listener
added up front still catches it either way; only a listener attached to the element itself,
after the fact, can miss it.

The tool **schemas** are declared to the model server-side (via your broker / session config); the
**handlers** are registered here. A `void` return is acked as `{ ok: true }`; an unknown tool or a
thrown handler returns an error to the agent. Only registered functions can be invoked. When you
mount in JS instead of using the tag, pass `clientTools` to `mountVoiceAgent` directly (see
[`@microsoft/voice-widget-ui`](https://www.npmjs.com/package/@microsoft/voice-widget-ui)); the underlying protocol lives in
[`@microsoft/voice-widget-provider-voicelive`](https://www.npmjs.com/package/@microsoft/voice-widget-provider-voicelive).

> **Client tools run untrusted input.** "Only registered functions can be invoked" bounds *which*
> functions run, not what they are asked to do. A handler's arguments are filled in by the model
> and can be prompt-injected — validate them before any sensitive action (e.g. allow only
> `http:`/`https:` URLs before navigating; never `eval` or inject a model-supplied string as
> HTML/JS). Its return value is sent to the model and may be spoken aloud, so return only
> end-user-safe values — no PII or internal error detail. See
> [Security - handler inputs and outputs](https://github.com/microsoft/voice-widget/blob/main/docs/client-tools.md#security-handler-inputs-and-outputs).

### `registerClientTool` (programmatic) and `voice-agent:unhandledtool`

The `<voice-agent>` element exposes no imperative methods — for a **declarative** embed, inject
tools before connect via the `voice-agent:call` event shown above. If you mount
**programmatically**, the controller returned by `mountVoiceAgent` exposes `registerClientTool`
(throws on a duplicate name; returns an unregister):

```js
import { mountVoiceAgent } from "@microsoft/voice-widget-embed";

const controller = mountVoiceAgent(document.querySelector("#slot"), {
  provider: "voicelive",
  config: { targetType: "model", model: "gpt-realtime" },
  authEndpoint: "/api/session",
});
const unregister = controller.registerClientTool("openModal", ({ id }) => openModal(id));
```

When the agent calls a name with no registered handler, the element fires a
**`voice-agent:unhandledtool`** `CustomEvent` (detail: `{ name, args, callId }`):

```js
el.addEventListener("voice-agent:unhandledtool", (e) => {
  console.warn("unhandled client tool:", e.detail.name, e.detail.args);
});
```

This is **notification only** — it cannot fulfill the call. The adapter still returns a standard
error output to the agent.

**Scope:** model **and** agent targets over WebRTC are supported.

## Telemetry — the `voice-agent:telemetry` event

The element dispatches a **`voice-agent:telemetry`** `CustomEvent` for every structured telemetry
event, with the event as `detail`:

```js
el.addEventListener("voice-agent:telemetry", (e) => {
  const event = e.detail; // VoiceAgentTelemetryEvent — { name, ...envelope, data }
  track(event.name, event);
});
```

Each event carries a `widgetInstanceId` / `correlationId` / `sessionId` envelope for tracing a
session across the widget and your broker, spanning failures that happen **before** a server-side
session exists. The event is opt-in, low-frequency, and the SDK-supported structured counterpart
to the raw-event firehose. The **`telemetry-console`** attribute additionally mirrors events to
`console.debug` (opt-in, off by default) — a local-dev sink that changes no behavior. See
[`docs/telemetry.md`](https://github.com/microsoft/voice-widget/blob/main/docs/telemetry.md) for the full event catalogs and wiring.