# Aniface
[](https://www.npmjs.com/package/aniface)
[](https://www.npmjs.com/package/aniface)
[](https://github.com/alysachan830/aniface/blob/main/LICENSE)
[](https://www.npmjs.com/package/aniface)
[](https://alysachan830.github.io/aniface/)
[🎠**Try Live Demo**](https://alysachan830.github.io/aniface/) | [📦 **npm Package**](https://www.npmjs.com/package/aniface)
**Animate 3D avatars with real-time facial tracking using MediaPipe and Three.js**
A powerful, easy-to-use library that brings your 3D avatar models to life with real-time facial expressions captured from a webcam.
## Features
- 🎠**Real-time facial tracking** - Powered by MediaPipe Face Landmarker
- 🎨 **Automatic blendshape mapping** - Works with ARKit-compatible models
- 🎯 **Simple API** - Get started in minutes
- 🔌 **Flexible input modes** - Built-in detection, custom sources, or direct data push
- 🎮 **Multiple rigging systems** - Support for Ready Player Me, QuickRig, and custom models
- 🔧 **Highly customizable** - Fine-tune every aspect of rendering and tracking
- 📦 **TypeScript support** - Full type definitions included
- âš¡ **Performance optimized** - Efficient rendering and blendshape updates
## Installation
```bash
npm install aniface three @mediapipe/tasks-vision
```
## Quick Start
```javascript
import { Aniface } from 'aniface'
// Get your video and canvas elements
const videoElement = document.getElementById('webcam')
const canvasElement = document.getElementById('avatar')
// Create the avatar
const avatar = new Aniface({
videoElement,
canvasElement,
modelPath: '/path/to/your/model.glb',
onReady: () => console.log('Avatar ready!'),
onError: (error) => console.error('Error:', error)
})
// Initialize and start
await avatar.initialize()
avatar.start()
// When done
avatar.stop()
avatar.destroy()
```
## Manual Landmark Input (Advanced)
Aniface supports two input modes:
### 1. Built-in MediaPipe Detection (Default)
The standard way - provide a video element and Aniface handles everything:
```javascript
const avatar = new Aniface({
videoElement: document.getElementById('webcam'),
canvasElement: document.getElementById('avatar'),
modelPath: '/models/avatar.glb'
})
await avatar.initialize()
avatar.start() // Automatic processing loop
```
### 2. Custom MediaPipe Configuration
Use your own MediaPipe Face Landmarker instance with custom settings:
```javascript
const avatar = new Aniface({
canvasElement: document.getElementById('avatar'),
modelPath: '/models/avatar.glb'
// No videoElement needed when using custom MediaPipe
})
await avatar.initialize()
// Create MediaPipe instance with custom configuration
const myLandmarker = await FaceLandmarker.createFromOptions(vision, {
runningMode: 'VIDEO',
minFaceDetectionConfidence: 0.7,
minFacePresenceConfidence: 0.7,
minTrackingConfidence: 0.7
// ... other custom options
})
// Manual animation loop with custom MediaPipe
function animate() {
requestAnimationFrame(animate)
const results = myLandmarker.detectForVideo(video, performance.now())
avatar.processLandmarkData(results)
}
animate()
```
### Use Cases for Custom MediaPipe
- **Custom detection thresholds**: Fine-tune confidence levels for your specific use case
- **Performance optimization**: Adjust settings for better FPS on lower-end devices
- **Custom delegates**: Use GPU acceleration or WebAssembly optimizations
- **Multiple faces**: Process multiple faces from the same video stream
- **Data processing**: Apply custom filters or transformations before rendering
- **Integration**: Combine with other MediaPipe tasks (hand tracking, pose detection)
## Configuration Options
### Basic Configuration
```javascript
new Aniface({
// Video element (optional if using processLandmarkData for manual updates)
videoElement?: HTMLVideoElement,
// Canvas element (always required)
canvasElement: HTMLCanvasElement,
// Path to GLB/GLTF model (always required)
modelPath: string,
// Optional callbacks
onReady: () => void,
onError: (error: Error) => void,
onLandmarksDetected: (results) => void,
onNoFaceDetected: () => void
})
```
### Examples
#### Head-only Avatar (Raccoon avatar)
```javascript
const avatar = new Aniface({
videoElement,
canvasElement,
modelPath: './raccoon_head_small.glb',
cameraConfig: {
fov: 60,
enableControls: false,
enableZoom: false
},
blendshapeMultipliers: {
eyeBlinkLeft: 1.2,
eyeBlinkRight: 1.2,
jawOpen: 1.0,
mouthSmileLeft: 1.1,
mouthSmileRight: 1.1
},
modelOptions: {
scale: 1.0
},
onReady: () => console.log('Avatar ready!'),
onError: (error) => console.error('Error:', error)
})
await avatar.initialize()
avatar.start()
```
#### Half-body Avatar (Ready Player Me avatar)
```javascript
const avatar = new Aniface({
videoElement,
canvasElement,
modelPath: 'https://models.readyplayer.me/[YOUR_ID].glb?morphTargets=ARKit&useHands=false',
cameraConfig: {
fov: 60,
enableControls: false,
enableZoom: false
},
blendshapeMultipliers: {
eyeBlinkLeft: 1.3,
eyeBlinkRight: 1.3,
browInnerUp: 1.2,
browOuterUpLeft: 1.2,
browOuterUpRight: 1.2,
jawOpen: 1.0,
mouthSmileLeft: 1.0,
mouthSmileRight: 1.0
},
lightingConfig: {
ambientIntensity: 1.2,
directionalIntensity: 1.5,
directionalPosition: [2, 3, 3]
},
modelOptions: {
scale: 1.8,
center: true,
autoRotate: false,
rotation: 0,
fullBodyAvatar: true
},
onReady: () => console.log('Avatar ready!'),
onError: (error) => console.error('Error:', error)
})
await avatar.initialize()
avatar.start()
```
#### QuickRig / Different Rigging Systems
Different 3D models use different coordinate systems for bone rotations. Use `axisMapping` to specify your model's rigging system:
```javascript
const avatar = new Aniface({
videoElement,
canvasElement,
modelPath: './quickrig_avatar.glb',
cameraConfig: {
fov: 60,
position: [0, 1.65, 1.2],
target: [0, 1.6, 0]
},
modelOptions: {
scale: 1.8,
center: true,
autoRotate: false,
rotation: 0,
position: [0, -0.5, 0],
fullBodyAvatar: true,
axisMapping: 'quickrig' // Specify QuickRig coordinate system
},
blendshapeMultipliers: {
eyeBlinkLeft: 1.3,
eyeBlinkRight: 1.3,
jawOpen: 1.0
},
onReady: () => console.log('Avatar ready!'),
onError: (error) => console.error('Error:', error)
})
await avatar.initialize()
avatar.start()
```
**Available axis mappings:**
- `'standard'` (default): For Ready Player Me and most manually rigged models
- `'quickrig'`: For QuickRig rigged models
**How to know which to use:**
- If your avatar looks in wrong directions when you move your head, try switching the `axisMapping`
- Check your model's bone naming (e.g., `QuickRigCharacter_Head` suggests `'quickrig'`)
## Advanced Configuration Examples
### Blendshape Customization
Fine-tune expression intensity:
```javascript
const avatar = new Aniface({
videoElement,
canvasElement,
modelPath: '/models/avatar.glb',
// Adjust specific blendshapes
blendshapeMultipliers: {
eyeBlinkLeft: 1.2, // Make blinks more pronounced
eyeBlinkRight: 1.2,
jawOpen: 0.8, // Reduce mouth opening
mouthSmileLeft: 1.5, // Exaggerate smiles
mouthSmileRight: 1.5
}
})
```
### Camera & Lighting
```javascript
const avatar = new Aniface({
videoElement,
canvasElement,
modelPath: '/models/avatar.glb',
// Camera settings
cameraConfig: {
fov: 60, // Field of view
position: [0, 1.65, 1.2], // Camera position [x, y, z] (default: [0, 0, 1.5])
target: [0, 1.6, 0], // Camera lookAt target [x, y, z] (default: [0, 0, 0])
enableControls: true, // Orbit controls
enableZoom: false // Disable zoom
},
// Lighting settings
lightingConfig: {
ambientIntensity: 0.6, // Ambient light intensity
directionalIntensity: 0.9, // Main light intensity
directionalPosition: [5, 5, 5] // Light position
}
})
```
### Model Options
Configure model positioning, scaling, and rigging system:
```javascript
const avatar = new Aniface({
videoElement,
canvasElement,
modelPath: '/models/avatar.glb',
modelOptions: {
scale: 1.5, // Uniform scale factor
center: true, // Center model at origin
autoRotate: true, // Apply automatic Y rotation
rotation: Math.PI, // Rotation in radians (default: 180°)
position: [0, -0.5, 0], // Position offset [x, y, z]
fullBodyAvatar: true, // Enable for full-body models (vs head-only)
axisMapping: 'standard' // Rigging system: 'standard' or 'quickrig'
}
})
```
**Axis Mapping Options:**
- `'standard'`: Ready Player Me, most manual rigs (default)
- `'quickrig'`: QuickRig rigged models
💡 **Tip:** If your avatar moves in wrong directions, try changing `axisMapping`.
### MediaPipe Configuration
Fine-tune facial detection and tracking:
```javascript
const avatar = new Aniface({
videoElement,
canvasElement,
modelPath: '/models/avatar.glb',
// Optional: Adjust facial tracking sensitivity
landmarkConfig: {
minDetectionConfidence: 0.5, // 0-1: Lower = more detections (default: 0.5)
minTrackingConfidence: 0.5 // 0-1: Lower = smoother tracking (default: 0.5)
}
})
```
**Configuration tips:**
- Lower confidence values = more sensitive detection but may be less stable
- Higher confidence values = more stable but may miss subtle movements
- WASM files load from CDN automatically - no `wasmPath` needed unless self-hosting
## Model Requirements
Your 3D model must have:
1. **ARKit-compatible blendshapes** - Standard facial animation targets
2. **GLB or GLTF format** - Three.js compatible
3. **Proper rigging** - Face mesh with morph targets
### Supported Rigging Systems
Aniface supports multiple rigging conventions:
- **Ready Player Me** - Use default settings or `axisMapping: 'standard'`
- **QuickRig** - Use `axisMapping: 'quickrig'`
- **Custom models** - Try `'standard'` first, switch to `'quickrig'` if movements are incorrect
### Common Blendshape Names
Aniface maps MediaPipe blendshapes to these standard names:
- `eyeBlinkLeft`, `eyeBlinkRight`
- `jawOpen`, `jawLeft`, `jawRight`
- `mouthSmileLeft`, `mouthSmileRight`
- `browInnerUp`, `browOuterUpLeft`, `browOuterUpRight`
- And many more... (52 total)
## API Reference
### Aniface Class
#### Methods
```typescript
// Initialize the avatar system
await avatar.initialize(): Promise
// Start facial tracking and animation
avatar.start(): void
// Stop the animation loop
avatar.stop(): void
// Clean up all resources
avatar.destroy(): void
// Update canvas size (call on window resize)
avatar.updateSize(width: number, height: number): void
// Check if avatar is running
avatar.running: boolean
// Check if avatar is initialized
avatar.ready: boolean
```
## License
MIT - see [LICENSE](LICENSE) file for details