# narrator-avatar
[](https://www.npmjs.com/package/narrator-avatar)
[](https://www.npmjs.com/package/narrator-avatar)
React component for 3D talking avatars with lip-sync, Deepgram or Google TTS, content-aware hand gestures, and pause/resume. Built on [@met4citizen/talkinghead](https://www.npmjs.com/package/@met4citizen/talkinghead).
## Features
- **3D avatars** – Ready Player Me–compatible GLB models (full-body)
- **Lip-sync** – Word-level sync with Deepgram or Google TTS (English bundled; other languages need extra setup)
- **TTS** – Deepgram (streaming) or Google Cloud Text-to-Speech
- **Gestures** – Content-aware hand gestures: handup, index, ok, thumbup, side, shrug
- **Playback** – Speak, Pause, Resume, Stop (mid-phrase resume in accurate mode)
- **Speech reliability** – Prefetches upcoming phrases, retries TTS, resumes AudioContext on tab focus (reduces lag and silent dropouts)
- **Avatar Studio controls** – Visual presets, skin gloss/pores/warmth, eye contact, and mood
- **Accessibility** – Subtitle callback for closed captions
## Table of contents
- [Install](#install)
- [Usage](#usage)
- [Props](#props)
- [Ref API](#ref-api)
- [Environment variables](#environment-variables)
- [TypeScript](#typescript)
- [License](#license)
## Install
```bash
npm install narrator-avatar
```
Peer dependencies (React 18+) are installed automatically. The package bundles TalkingHead, Three.js, and English lip-sync—**no import maps or Vite config required.**
## Usage
```jsx
import { useRef } from 'react';
import NarratorAvatar from 'narrator-avatar';
function MyPage() {
const avatarRef = useRef(null);
return (
{}}
onSpeechStart={() => {}}
onSpeechEnd={() => {}}
onSubtitle={() => {}}
/>
);
}
```
## Props
| Prop | Description |
|------|-------------|
| `avatarUrl` | URL to GLB model (e.g. `/avatars/brunette.glb`) |
| `avatarBody` | `'M'` or `'F'` for posture |
| `cameraView` | Camera framing: `'full'`, `'mid'`, `'upper'`, `'head'` (default `'mid'`) |
| `cameraRotateEnable` | Allow mouse drag to rotate view (default `false`). Set `true` to enable. |
| `cameraZoomEnable` | Allow mouse wheel to zoom (default `false`). Set `true` to enable. |
| `cameraPanEnable` | Allow mouse to pan (default `false`) |
| `ttsService` | `'google'` or `'deepgram'` |
| `ttsVoice` | Deepgram: e.g. `aura-2-mars-en`, `aura-2-aurora-en`. Google: e.g. `en-GB-Standard-A` |
| `ttsApiKey` | API key (or set `VITE_DEEPGRAM_API_KEY` / `VITE_GOOGLE_TTS_API_KEY`) |
| `lipsyncModules` | Array of language codes (default `['en']`) |
| `lipsyncLang` | Lip-sync language (default `'en'`) |
| `visualPreset` | Lighting preset: `'cinematic'`, `'beauty'`, `'studio'`, `'sunset'`, `'broadcast'` |
| `visualQuality` | `'auto'` (default): ultra on desktop, balanced on mobile/slow network. `'ultra'` \| `'balanced'` \| `'performance'` |
| `lazyMount` | `true` \| `false` \| `'auto'` (default): defer WebGL init until visible on mobile |
| `skinGloss` | Skin sheen/sweat intensity from `0` to `2` |
| `skinPores` | Procedural pore/normal intensity from `0` to `2` |
| `skinWarmth` | Warm skin tone blend from `0` to `2` |
| `eyeContactIntensity` | Eye openness/contact strength from `0` to `2` |
| `modelFPS`, `modelPixelRatio` | Render cadence and resolution multiplier. **TalkingHead also multiplies `modelPixelRatio` by `devicePixelRatio`**, so use values near `1` (not `4`). Mobile is capped automatically. |
| `dracoEnabled`, `dracoDecoderPath` | Enable Draco-compressed avatar loading |
| `modelDynamicBones` | TalkingHead dynamic-bone config for rigged hair/body parts |
| `update` | Per-frame callback `(dt, talkingHead)` |
| `accurateLipSync` | `true` = REST per phrase, best lip-sync + pause/resume. Default `true` for stable tutor playback |
| `speechRate` | e.g. `0.9` for 10% slower (pitch-preserving) |
| `phraseGapMs` | Base teacher breath between speech units (default `560`). Adapts longer after `?` / long beats. Complete sentences are not glued together so breaths can land. |
| `speechGestures` | Content-aware hand gestures (default `true`) |
| `speechGestureStyle` | `'tutor'` (default): sparse LMS instructor gestures with cooldown. `'expressive'`: legacy busier mapping |
| `getGestureForPhrase` | `(phrase) => { name, dur?, mirror?, eyeContactMs? } \| null` to override per phrase |
| `avatarMode` | `'auto'` (default): **voice-only on mobile** (~22KB entry, no GLB/THREE); full 3D lazy-loaded on desktop. `'voice'` / `'full'` to force |
| `preloadText` | Lesson text to warm into the TTS cache when the avatar is ready (phrase 0 first, then ahead window) |
| `onPreloadReady` | Fires when phrase 0 is cached — Speak can start without waiting on Deepgram |
| `pauseWhenHidden` | `true` (default): pause on tab hide / phone call; resume on return. Leave/unmount always stops |
| `subtitleMaxWords` | Cap word-follow captions to last N words (`0` = full phrase). Reduces cut-off / layout shake |
| `onReady`, `onError`, `onSpeechStart`, `onSpeechEnd` | Lifecycle callbacks |
| `onSubtitle` | Word-follow caption callback as speech plays `(text: string) => void` |
## Ref API
| Method / property | Description |
|-------------------|-------------|
| `speakText(text, options?)` | Speak text via TTS |
| `preloadSpeech(text)` | Prefetch first phrases into TTS cache (call when lesson text is known) |
| `pauseSpeaking()` | Pause mid-phrase when `accurateLipSync` is true |
| `resumeSpeaking()` | Resume from the same point in the current phrase |
| `stopSpeaking()` | Stop and clear |
| `makeEyeContact(durationMs?)` | Ask the avatar to hold stronger eye contact |
| `setMood(mood)` | Change TalkingHead mood |
| `setLighting(options)` | Pass lighting options to TalkingHead |
| `setView(view, options?)` | Change camera view |
| `playGesture(name, dur?, mirror?, ms?)` | Play a built-in gesture |
| `playAnimation(url, onprogress?, dur?, ndx?, scale?)` | Play a Mixamo/RPM FBX animation |
| `playPose(url, onprogress?, dur?, ndx?, scale?)` | Play a Mixamo/RPM FBX pose |
| `stopAnimation()`, `stopPose()` | Stop active animation or pose |
| `setMixerGain(speech, background?, fadeSecs?)` | Adjust speech/background audio gain |
| `playBackgroundAudio(url)`, `stopBackgroundAudio()` | Control background audio |
| `isReady` | Whether the avatar has finished loading |
| `isSpeaking` | Whether the avatar is currently speaking |
## Environment variables
| Variable | Use |
|----------|-----|
| `VITE_DEEPGRAM_API_KEY` | Deepgram TTS (or pass `ttsApiKey` prop) |
| `VITE_GOOGLE_TTS_API_KEY` | Google TTS when `ttsService="google"` (or pass `ttsApiKey`) |
## TypeScript
The package is JavaScript. For TypeScript, add a declaration file (e.g. `src/narrator-avatar.d.ts`) that declares the component props and ref type, or use the component with `// @ts-expect-error` if you prefer.
## Performance (web and mobile, one GLB per avatar)
Use **one** model URL per avatar (e.g. `/avatars/tutor.glb`). The component adapts at runtime — no `-mobile` / `-web` copies.
| Mechanism | Desktop | Mobile / Save-Data / low-end |
|-----------|---------|------------------------------|
| `visualQuality="auto"` | `ultra` — 60 FPS, studio lights, 4K shadows | Phones → `balanced` (20 FPS, capped DPR ≤~1.35, no shadows/studio lights). Low-RAM / 2G / Save-Data → `performance` (15 FPS, minimal lights) |
| Pixel ratio | ~1.25 × DPR | Multiplier auto-capped so total device pixels stay ~1.15–1.35 |
| Motion | Full idle/speaking head sway | Reduced `modelMovementFactor` + head move |
| TTS preload | First 2–3 phrases | First phrase only; cache capped |
| `lazyMount="auto"` | init immediately | init when scrolled into view |
| Load timeout | 30s | 120s (large GLB on cellular) |
| Tab hidden | animation paused | animation paused |
```jsx
```
Force a lighter tier anytime: `visualQuality="performance"`.
**App tips:** mount **only one** avatar on narrow viewports; enable gzip/brotli (and Draco if you ship compressed GLBs) on your CDN.
Keep `modelPixelRatio` near **0.45–0.75 on mobile** if you override — TalkingHead multiplies it by `devicePixelRatio`.
**Next.js:** dedupe React in `next.config` (see `narrator-avatar-test/next.config.ts`).
## License
MIT