--- name: incanto-audio description: Sound in Incanto — zero-asset PROCEDURAL SFX presets (coin/jump/hurt/explosion/laser/win…, the art-free audio analog of particle presets), the AudioPlayer node (preset vs src file, autoplay + browser gesture-unlock, the finished signal, 3D spatial/positional audio), the music manager (crossfade/loop background music), global volume buses (master/sfx/music + mute), and the built-in audio catalog of real sound files. Use whenever a game needs sound, sfx, audio effects, music, spatial/positional audio, or volume control. --- # Audio — sound, SFX, and music > Shipped inside the `incanto` npm package — this document always matches the > installed engine version. Sibling skills live in `node_modules/incanto/skills/`. Sound comes from the **`AudioPlayer`** node. It has two modes: 1. **`preset`** — a built-in **procedural SFX** synthesized on the fly. **Zero asset bytes, instant, deterministic.** This is the headline path: the audio analog of the `Particles2D` presets / art-free prototyping. Reach for this first — a working game can ship sound with no files at all. 2. **`src`** — play an audio file (URL). Good for music and longer clips, and for the curated built-in sounds (§4). Used when `preset` is `"custom"`. All volume routes through global **buses** (§3): `master × (sfx|music) × volume`. `AudioPlayer` is headless-safe — with no audio backend (node, the verify VM) every method is a silent no-op, so scenes load and simulate everywhere. --- ## 1. Procedural SFX presets — the zero-asset path (lead with this) Set `preset` to one of the names below and call `play()`. No files, no loading. ```jsonc { "type": "AudioPlayer", "name": "Coin", "props": { "preset": "coin" } } ``` ```ts import type { AudioPlayer } from 'incanto'; // from a Behavior — `getNode` returns a `Node`, so name the type you asked for const coin = this.node.getNode('Coin') as AudioPlayer; coin.play(); // synthesizes + plays instantly; rapid calls OVERLAP (no cutoff) ``` ### The preset set | preset | sound it makes | typical use | |-------------|-------------------------------------------------|------------------------| | `coin` | bright ascending chime | collect a coin | | `pickup` | soft two-tone blip | generic item pickup | | `jump` | rising whoop | jump | | `hurt` | harsh downward buzz | take damage | | `hit` | short noisy thud | impact / melee hit | | `explosion` | big falling-pitch boom | explosion (+particles) | | `powerup` | cheerful rising sweep | power-up / level up | | `laser` | zappy descending beam | laser / energy shot | | `shoot` | punchy square shot | gunfire / projectile | | `blip` | tiny neutral tick | UI feedback | | `select` | confident two-step | menu select / confirm | | `step` | soft muted noise tap | footstep | | `win` | happy fanfare rise | victory | | `lose` | sad descending tone | game over | (Authoritative list: `incanto-node-reference.md` → `AudioPlayer.preset` options, or `import { SFX_PRESET_NAMES } from 'incanto'`.) ### Variation: `pitch` + `seed` Repeated identical SFX feel robotic. Vary them for free: - **`pitch`** (default 1) — multiplies the frequency. `0.8` lower, `1.4` higher. - **`seed`** (default 0) — varies the noise in noisy presets (`explosion`, `hit`, `step`). Same seed → byte-identical sound (deterministic, replay-safe). ```jsonc { "type": "AudioPlayer", "name": "Step", "props": { "preset": "step", "pitch": 1.1, "seed": 3 } } ``` To re-roll variation at runtime, set `seed` before each `play()`: ```ts step.seed = (step.seed + 1) | 0; step.play(); ``` ### Firing presets from gameplay Wire them through scene `connections` — e.g. play `coin` when a Collector picks something up, `hurt` when Health takes damage: ```jsonc "connections": [ { "signal": "collected", "from": "Player/Collector", "to": "Coin", "handler": "play" }, { "signal": "damaged", "from": "Player/Health", "to": "Hurt", "handler": "play" } ] ``` `play` is a method on `AudioPlayer`, so a connection can target it directly. > Procedural SFX use **WebAudio** (low-latency, overlap-friendly) — not the > HTMLAudio element. WebAudio is a browser API behind a headless guard, so it's > a no-op in the verify VM. Like all audio it needs a user gesture first; the > same unlock listener that retries `src` players also resumes the SFX context > (see §2). ### Continuous voices — engine hum, wind, thrusters (`startVoice`) One-shot presets can't do a sound that **lasts and retunes** — a motor that follows the RPM, wind that rises with speed. `engine.sfx.startVoice(preset)` starts a live synth voice you retune every frame: ```ts const motor = engine.sfx.startVoice('engine'); // presets: engine | wind | hum | noise // per frame (e.g. in a script's update): motor.set({ pitch: 0.8 + speed / maxSpeed, volume: throttle * 0.6 }); // on destroy / scene change: motor.stop(); // fades out over 0.15s and frees the WebAudio nodes motor.stop(1.2); // …or a slow custom fade ``` - **`pitch`** — 1 = the preset's base tone; drive it from speed/RPM/height. - **`volume`** — 0..1; `startVoice(preset, gain)` sets an overall ceiling. - Presets: `engine` (detuned-saw motor growl), `wind` (band-passed rush), `hum` (soft pad — shields, auras, reactors), `noise` (thrusters, static). - Headless the handle is **inert** (`set`/`stop` are no-ops) — keep the same call sites, no `if (browser)` guards. Voices need the same first-gesture unlock as everything else. --- ## 2. The AudioPlayer node (preset vs src file) ```jsonc { "type": "AudioPlayer", "name": "Bgm", "props": { "src": "incanto/assets/audio/...mp3", // file URL (preset = "custom") "preset": "custom", // or a preset name → ignores src "volume": 1, // 0..1, before bus gain "bus": "sfx", // "sfx" (default) or "music" "loop": false, // loop the src clip (music) "autoplay": false // play on first frame in the tree } } ``` Methods: **`play()`**, **`stop()`**. Signal: **`finished`** — connect it to clean up or chain sounds. It fires when a non-looping `src` clip ends AND when a procedural preset does: a preset's length is `attack + sustain + decay`, known exactly, so `player.playing` is true for that long and `finished` arrives at the end. It is measured on the unscaled clock, because a paused game still hears the tail of the hit that paused it, and it fires headlessly too — the length is arithmetic, not a device. `stop()` is not a finish and does not emit. ### Autoplay + the browser gesture-unlock Browsers block audio before the first user gesture. `autoplay: true` (or any early `play()`) that gets blocked is marked pending; **`createGame2D` / `createGame3D` automatically retry every pending player AND resume the WebAudio SFX context on the first gesture anywhere on the page** — you don't wire anything, and a tap on the on-screen touch controls counts (they sit above the canvas, not on it). A blocked play is not an error; it just waits. **A file the browser CANNOT play is a different thing, and it says so.** A 404 or an undecodable clip used to be filed as "waiting for a gesture" too, so it retried forever in silence — and a quiet game looks exactly like a game with no sound in it. Now it lands in the engine log and in **`game.assetErrors()`** (which 2D games can also ask now), beside the textures and models that 404'd: ``` [incanto] AudioPlayer 'Boom' could not load '/audio/explosion.mp3' — it is silent. ``` ### Music: loop a `src` clip on the music bus ```jsonc { "type": "AudioPlayer", "name": "Music", "props": { "src": "incanto/assets/audio/...mp3", "loop": true, "autoplay": true, "bus": "music", "volume": 0.6 } } ``` > For crossfading between tracks (e.g. exploration → boss music), use the **music > manager** (§5) instead of an `AudioPlayer` — it owns a single track and ramps. --- ## 2b. Spatial (positional) audio Set **`spatial: true`** on an `AudioPlayer` and the sound pans (left/right) and attenuates by **distance** — the emitter is the node's world position, the **listener is the active camera** (`Camera3D` or `Camera2D`, the one marked `current`). The adapter feeds the emitter + listener pose to the panner every frame, so moving the emitter or the camera updates the sound live. Works in both dimensions and for BOTH paths (procedural `preset` and a `src` file). ```jsonc // Attach under a moving Node3D — it emits from THAT node's world position. { "type": "Node3D", "name": "Enemy", "props": { "position": [8, 0, -3] }, "children": [ { "type": "AudioPlayer", "name": "Footsteps", "props": { "preset": "step", "spatial": true, // ← turn on 3D positional audio "refDistance": 2, // full volume within 2 units "maxDistance": 40, // inaudible past 40 units "rolloff": "inverse" // "inverse" | "linear" | "exponential" } } ] } ``` | prop | default | meaning | |---------------|-------------|--------------------------------------------------------| | `spatial` | `false` | off = ordinary non-positional sound (back-compat) | | `refDistance` | `1` | distance at which gain is full; closer never louder | | `maxDistance` | `50` | distance past which gain stops falling | | `rolloff` | `"inverse"` | attenuation curve (WebAudio PannerNode distance model) | - **`inverse`** (default): `ref / (ref + d−ref)` — natural-sounding 1/d falloff. - **`linear`**: straight ramp to ~0 at `maxDistance`. - **`exponential`**: steeper near falloff. Under the hood spatial audio uses a WebAudio **`PannerNode` (HRTF)** with the listener driven by the camera's world position + orientation. The pure distance-gain + pan-sign math (`spatialGain`, `spatialPan`, exported) is what the non-WebAudio paths use, so headless gameplay is unaffected. > **2D scenes** work the same way: the listener is the active `Camera2D` (its > world position is the view centre, and its own rotation turns the stereo image > with it). The one difference is the UNIT — a 2D scene measures in PIXELS, > and `refDistance: 1` / `maxDistance: 50` are metre-shaped defaults that put a > sound at its quietest about one sprite away. Set them to your level's scale > (`refDistance: 100`, `maxDistance: 900` is a reasonable start); `incanto check` > warns when it sees `spatial: true` in a 2D scene with the defaults left alone. > **Headless / verify VM:** no sound comes out — and the ANSWER still does. The > audio record carries where each spatial sound arrived from, computed from the > scene's own current camera when no renderer has fed a pose: ```ts const ring = engine.audio.recent().at(-1); ring.pan; // −1 hard left … +1 hard right (absent when the sound is not spatial) ring.gain; // 0..1 after the distance rolloff — 1 inside `refDistance` ring.distance; // metres (px in 2D) from the listener when it sounded ``` Thirty shipped scenes set `spatial: true` before this existed and not one of them could be checked: the record said a sound FIRED and nothing about whether it was audible or which side it came from. Now "walk toward the ringing" is a mechanic a harness can verify — `examples/earshot-3d` is a whole game of it (a bell on the left records `pan −0.98`; turn the listener around and the same bell reads `+0.98`). Two things to know when you measure it: - **A sound that is not spatial has no `pan`/`gain`/`distance` at all** — the fields are absent rather than zero, so a UI click never looks centred. - **A scene with no current camera has no listener**, and the record says nothing rather than inventing a sound in the middle of your head. --- ## 3. Volume buses (global volume + mute) Every sound's final loudness is `master × bus × volume`, where `bus` is the node's `sfx` or `music` gain. The buses live on the engine as `engine.audio`: ```ts engine.audio.master = 0.8; // dim everything (0..1) engine.audio.sfx = 1; // SFX bus gain engine.audio.music = 0.4; // quieter background music engine.audio.muted = true; // global mute toggle (true → all sound 0) ``` Values clamp to `[0,1]`; non-finite sets are ignored. `engine.audio.changed` fires on any change (drives a settings UI). Procedural SFX go to whichever `bus` the node declares (default `sfx`); music-ish loops typically set `bus: "music"`. Changes apply to currently-playing `src` clips on the next frame and to every new sound immediately. A settings slider just writes these numbers — no per-node bookkeeping. And **the slider is a node**, so an audio menu is scene JSON like everything else: ```json { "name": "Options", "type": "UiPanel", "props": { "anchor": "center" }, "children": [ { "name": "Music", "type": "UiVolumeSlider", "props": { "bus": "music" } }, { "name": "Sound", "type": "UiVolumeSlider", "props": { "bus": "sfx" } }, { "name": "Mute", "type": "UiMuteToggle" } ] } ``` | node | prop | writes | |---|---|---| | `UiVolumeSlider` | `bus`: `master` (default) / `sfx` / `music` | `engine.audio[bus]` | | `UiMuteToggle` | — | `engine.audio.muted` (checked = silent) | No behavior, no `connections`, nothing to save: the buses persist themselves (below), and each control follows a change made anywhere else, so two menus can never disagree. Left unlabelled a slider names its own bus (`Volume` / `Sound` / `Music`, translated when the scene declares `settings.volume.*`). --- ## 4. Built-in audio catalog (real sound files) Alongside the procedural set, the package ships a small set of **real**, owner-licensed (MIT-ish) sound files — handy when you want recorded SFX. They're in the same catalog as the art (`kind: "audio"`): ```bash bunx incanto-assets list --json # filter kind === "audio" ``` Built-ins include `explosion`, `attacked` (hurt), `gold-loot` (coin), `slash`, `hit-metal-bang`, `heal`, `spells-cast`, `ice-spear`, `monster-died`, `smite`, `walk` (footstep), `ui-click`. Each entry's **`url`** is the drop-in contract: `incanto/assets/audio/` — exactly like the packaged sprites. Put it in `AudioPlayer.src`: ```jsonc { "type": "AudioPlayer", "name": "Boom", "props": { "src": "incanto/assets/audio/explosion.mp3", "bus": "sfx" } } ``` In a bundler game (vite, the usual target) you can `import url from 'incanto/assets/audio/explosion.mp3'` and write that into `src`; or `bunx incanto-assets copy --out public/assets` to copy the file. For more or different sound, any audio URL works (asset MCP servers, your own CDN) — never invent URLs; use the catalog or a real source. Large music tracks are NOT bundled (keep the package small) — reference them by URL. ### Editor The scene composer (`bunx incanto-editor`) shows `preset`, `bus`, and `rolloff` as dropdowns, `spatial` as a checkbox, `refDistance`/`maxDistance` as numbers, and `AudioPlayer.src` as an **audio asset picker** (browse the built-in sounds by name, or paste a URL). Zero-asset SFX = pick a `preset` instead of a `src`. --- ## 5. Music manager (crossfade / loop, single background track) For **background music** you usually want ONE track at a time and the ability to **crossfade** to another (exploration → boss, level A → level B). That's `engine.music` — a single-track manager that streams a clip and ramps gains: **The bundle ships SFX only** — twelve effects, no music track. A background track is your game's own file (`public/audio/…`) or a URL; `incanto/assets/audio/…` has nothing to loop. ```ts // Loop a track (full volume, music bus) — YOUR file, from public/: engine.music.play('/audio/theme.mp3'); // Options: { loop = true, fadeIn = 0 (seconds), bus = 'music' } engine.music.play('intro.mp3', { loop: false, fadeIn: 2 }); // Equal-power crossfade to a new track over N seconds (old fades out as new // fades in — no mid-fade volume dip): engine.music.crossfadeTo('boss.mp3', 3); // Fade the current track out and stop (0 = instant): engine.music.stop(2); // Per-manager volume (on top of the bus gain): engine.music.setVolume(0.5); // What's playing (the logically-current track src, or null): engine.music.current; ``` It routes through the **music bus** by default, so `engine.audio.music` / `master` / `muted` dim it for free (a settings slider needs no extra wiring). Fades advance automatically each frame (the engine ticks the manager). Like all audio it needs a user gesture; `createGame2D`/`createGame3D` resume + (re)start a gesture-blocked track on the first pointer/key, same as SFX. **Why a manager and not an `AudioPlayer` node?** A node is a one-shot/loop with no concept of "the current music"; crossfading needs to hold two tracks and ramp both. The imperative manager (call it from a behavior on scene change) is the clean fit; the looping-`src` `AudioPlayer` (§2) stays the right tool for a *fixed* background loop authored in scene JSON. Wire `crossfadeTo` from gameplay: ```ts // e.g. in a behavior when the boss spawns (`this.engine` is the accessor a // Behavior has — there is no `this.tree`): this.engine.music.crossfadeTo('/audio/boss.mp3', 3); ``` **A track that will never play says so.** A 404 or an undecodable file lands in the engine log (`[incanto] music '/audio/boss.mp3' will not play — could not load.`) instead of being indistinguishable from a game with no music. The autoplay gate is still just the gate — a track waiting for a tap is not an error. > Large music files are NOT bundled — reference them by URL (see §4). **Headless the state machine still runs and `current` still updates** — nothing plays, but the question a test has is answerable: ```ts session.engine.music.crossfadeTo('/audio/boss.wav', 2); session.step(100); session.engine.music.current; // '/audio/boss.wav' session.engine.audio.countOf('/audio/boss.wav'); // 1 ``` --- ## Sound stops when the thing making it goes away Teardown is SILENCE, and you do not wire it: - **A freed node stops its clip.** Swap to level 2 and level 1's looping background `AudioPlayer` stops with it — it used to play on over the new level forever, with its node freed and no handle left to stop it. Being *moved* is not this: a reparented node keeps playing, because reparenting is not destroying. - **`game.dispose()` stops the music and every continuous voice**, and hands the AudioContexts back. An SPA that mounts the game a few times would otherwise run out of them (browsers allow only a handful per page). So the one thing you still own is a voice you want to outlive a node — and the one thing you must NOT rely on is a sound stopping itself because the scene changed under it. `engine.music` deliberately survives a scene swap (it belongs to the engine, not the scene): call `engine.music.stop(1)` or `crossfadeTo` when the music should change. ## Sounds on a GRID: rhythm games and anything charted to a soundtrack A sound fired from `update()` cannot land closer than one frame to where a chart wants it — 16.67 ms at 60 Hz, 33.33 ms at 30, and that is the whole difference between a rhythm game that feels tight and one that does not. The engine's frame clock is exact (3600 steps land on `engine.time` 60.000000000 s and beat error never compounds); the last 16 ms is the gap. Schedule on the AUDIO clock instead. Queue a short LOOKAHEAD ahead of now, every frame, and let Web Audio place the sound: ```ts const LEAD = 0.08; // schedule this far ahead of the speaker let cursor = 0; // next un-queued note engine.updated.connect(() => { const horizon = engine.sfx.now + LEAD; while (cursor < chart.length && startedAt + chart[cursor].atSec <= horizon) { hit.playAt(startedAt + chart[cursor].atSec); // NOT play() — see below cursor++; } }); ``` - **`engine.sfx.now`** — seconds on the audio clock, the one the sound is actually placed on. With no AudioContext — headless, and before the first sound — it is the engine's own elapsed real time, so the loop above queues the whole chart in the verify VM exactly as it does in a browser. (It was a frozen `0`, which made that loop queue the first 80 ms of notes and then nothing ever again: 1 of 21 beats, with no error anywhere.) - **`AudioPlayer.playAt(when)`** and **`engine.sfx.play(params, gain, { when })`** — PRESETS only. A `src` clip goes through an `