# netblocks > Multiplayer XR for [xrblocks](https://github.com/google/xrblocks). `netblocks` is a batteries-included addon that turns any xrblocks app into a shared, real-time XR experience. It gives you: - 👥 **Presence** — see other users' heads (and hands when they're in XR with hand tracking) as live avatars, with an SDF name label floating above each one. - 📦 **Shared objects** — `NetObject` syncs `position`/`quaternion`/`scale` with cooperative ownership (claim on grab, release on drop). - 📨 **Typed events** — `session.events.emit('chat', payload)` style RPC. - 🎙️ **Spatial voice** — opt-in WebRTC audio that pans with each peer's head. - 🔌 **Pluggable transports** — pick the right one for your environment: - `BroadcastChannelTransport` — zero-config, two browser tabs see each other instantly. Perfect for local dev. - `WebRTCTransport` — peer-to-peer over the public PeerJS broker (or your own); no backend, low latency, ≤ ~12 peers. - `WebSocketTransport` — connect to a tiny relay (`server/relay.js`) for reliable, scalable multi-user rooms. The whole system is one mental model: **a `Transport` moves opaque bytes between peers; a `NetSession` layers presence, RPC, replicated objects, and voice on top.** Swap transports without changing app code. --- ## Quick start ```ts import * as xb from 'xrblocks'; import { enableNet, BroadcastChannelTransport, } from 'xrblocks/addons/netblocks/src/index.js'; class MyApp extends xb.Script { async init() { const net = enableNet(); const session = await net.joinRoom('my-room', { transport: new BroadcastChannelTransport(), displayName: 'Alice', }); session.events.on('chat', (text, fromPeerId) => { console.log(`${fromPeerId}: ${text}`); }); } } xb.add(new MyApp()); xb.init(); ``` Open the same page in two browser tabs and you'll instantly see the other tab's head rendered as a colored ball-and-stick avatar with a name label hovering above it. Open the page in XR with hand tracking and the avatar also gets two stick-figure hands. --- ## Porting an existing xrblocks sample to multiplayer Adding netblocks to an existing scene is small. The pattern is: 1. Call `enableNet()` once after `xb.init()` (e.g. from your script's `init()`). 2. `await xb.core.net.joinRoom(...)`. That's it — `xb.core.net` ticks on the standard xrblocks frame loop, so there's no `update()` to wire up. From there you wrap any `Object3D` you want shared in a `NetObject` and add it to `xb.core.net.session.netObjects`, and use `xb.core.net.session.events` for chat / cursor pings / button presses. Concretely, taking the standard xb starter: ```diff import * as xb from 'xrblocks'; +import {enableNet, NetObject, WebRTCTransport} from 'xrblocks/addons/netblocks/src/index.js'; class App extends xb.Script { cube = new THREE.Mesh(/* ... */); + sharedCube = new NetObject({id: 'cube', object: this.cube}); async init() { this.add(this.cube); + const net = enableNet(); + await net.joinRoom('demo', { + transport: new WebRTCTransport(), + displayName: 'Alice', + }); + net.session?.netObjects.add(this.sharedCube); } } ``` That's it — open the page in two tabs and dragging the cube in one tab moves it in the other. Add `events.on('chat', cb)` for typed RPC, or `net.session?.voice.enable()` for spatial voice. See `demos/netblocks/` for a fully wired example. --- ## Concepts ### NetCore The single facade you instantiate per app. Holds the active `NetSession` and exposes `joinRoom`, `leaveRoom`, `send`, and `update`. `joinRoom` defaults to `WebRTCTransport` when no `transport` option is given, so `net.joinRoom('lobby')` is enough for the common case. `send(topic, data)` is shorthand for `session.events.emit(topic, data)`. Also exposes two session-spanning shortcuts: - `xb.core.net.peers` — `list()`, `remoteUsers`, `on('join'|'leave', cb)`, and `events` (the active session's RPC bus, for `emit`/`on` of custom topics). Subscriptions persist across rejoins. - `xb.core.net.user` — local network identity (`peerId`, `displayName`). Distinct from `xb.user`, which is the local XR input device. ### NetSession The brain. Owns the transport, dispatches inbound messages to subsystems, and broadcasts outbound presence + object updates each frame. You'll mostly interact with `session.events`, `session.users`, and `session.createNetObject()`. ### NetUser Per-peer state — `peerId`, `displayName`, `role` (`'user' | 'device' | 'agent'`, self-reported), `avatar` (a `RemoteUserAvatar` `THREE.Group`), and `lastSeenMs`. Iterate via `session.users`. Pass `role` to `joinRoom({role: 'agent'})` to label the local peer; defaults to `'user'`. ### Transport A `Transport` is just **send bytes** + **peer-join/leave/message events**. Implement your own (e.g., to plug into a Liveblocks room or a Unity bridge) by extending the `Transport` base class. ### NetObject Set `automaticSnapshots: false` when a higher-level protocol owns initial state and catch-up. The session then excludes that object from generic snapshot sends and receives; claims, releases and continuous transform updates are unchanged. The default is `true`. Roomcraft uses this opt-out so unversioned backend snapshots cannot bypass its scene revision. A `THREE.Group` whose transform is replicated on a fixed cadence (default 20 Hz). Owners broadcast; non-owners interpolate. Ownership is cooperative — call `session.claim(obj)` on grab and `session.release(obj)` on drop. The protocol is race-aware: stale transforms from a previous owner are ignored after a claim, releases include a final canonical xform so all peers converge on the same resting position, and the sample shows how a grabber that loses ownership mid-drag should drop its local override. Explicit grabs carry a per-object logical claim counter. A grab after observing the previous claim uses a higher counter and takes over; crossed claims at the same counter select the lexicographically smaller peer ID on every peer. `obj.claim` records the last claim, including after release. Transform/release messages carry that generation, and catch-up snapshots preserve it so late joiners can take over normally. A snapshot without a claim cannot erase an observed generation. Legacy unstamped claims retain their previous arrival-order behavior; deterministic crossed-grab convergence requires every peer to run the updated build. To replicate an existing object's local transform without wrapping it, use `new NetObject({id: 'shared-object', object: existingObject})`. The readonly `netObject.object` is the supplied target, or the NetObject itself when omitted. Serialization, snapshot snaps, and interpolation operate directly on that target's position, quaternion, and scale: no reparenting, resource disposal, or per-frame transform-copy loop is needed. Keep the target under the equivalent parent coordinate system on every peer. ### NetEvents A typed pub/sub bus over the wire. `events.on(topic, handler)`, `events.emit(topic, payload)`, `events.emitTo(peerId, topic, payload)`. Use this for chat, button presses, emoji bursts, cursor pings — anything that isn't a recurring transform. ### VoiceChat + SpatialVoice WebRTC audio that piggybacks on the **active transport** for signaling (SDP/ICE flow as `voice` messages over whatever NetSession is using — BroadcastChannel, WebSocket, or WebRTC data channels). Audio itself always flows directly between browsers over a dedicated WebRTC peer connection, parented to each peer's `headPivot` via `THREE.PositionalAudio` so it spatializes naturally. Enable via `{voice: true}` in `joinRoom()` or `session.voice.enable(...)` later. `voice.setMuted(true)` silences outgoing microphone tracks while keeping incoming audio connected. `isMuted()` reports transmission state; `isEnabled()` reports whether microphone capture is acquired. Muting is retained when peers join or renegotiate, and unmuting does not request another stream. `disable()` is full teardown: it releases capture and audio connections and cancels a pending microphone grant. Listening-only peers can receive audio without enabling their microphone, independent of peer ID ordering. If microphone capture ends unexpectedly, incoming audio stays connected and the capture failure is reported. Call `enable()` explicitly to acquire a new microphone stream; stopped audio senders are reused without interrupting listening. Capture is never reacquired automatically. Use `voice.cancelPendingEnable()` to invalidate outstanding microphone permission requests without closing incoming connections or changing an already enabled microphone. Late-granted tracks from those requests are stopped. This is distinct from `disable()`, which intentionally closes the whole voice session. Incoming playback is separately controlled on `NetSession`: ```ts session.setPlaybackMuted(true); // All incoming voice, for this listener only. session.playbackMuted; // Readonly master preference. session.setPeerPlaybackMuted(peerId, true); // A current session.users peer. session.isPeerPlaybackMuted(peerId); // Individual choice, ignoring master. ``` Both preferences default to false. A peer is silent when **master OR individual** mute is true, so turning master mute off preserves individual choices. Muting affects only the peer's existing spatial audio gain, never microphone capture, voice announcements, connections, the shared listener, or other scene sounds. New/replacement streams are gated before connecting, including when the listener initializes after preferences were set. Stream removal retains the choice; peer leave removes it. `close()` disposes the incoming graph and clears all playback preferences without disposing the shared listener. Applications can retain and reapply the master preference across sessions; display names are not peer identity. The setters emit a local `playback-state` `CustomEvent` only on preference changes: `{muted}` for master, `{peerId, muted}` for individual (not effective) mute. Observe `user-leave` / `close` for lifecycle resets. Mute arguments must be booleans and peer IDs non-empty strings (`TypeError` otherwise); setting a peer not in `session.users` throws `RangeError`. Reading an unknown/departed peer's individual choice returns false. `VoiceChatOptions.onLocalStateChange` retains its enable/disable semantics. The optional `onLocalMuteChange` reports mute changes separately, and `onError` reports capture or peer-connection failures. `NetSession` combines mic state into `local-voice-state`, exposes received mic intent through `peer-voice-state`, and forwards errors through `voice-error`. Mic intent is not proof that another person can hear audio. Applications can separately subscribe to `user-update` to refresh metadata that arrives after a peer's initial join. --- ## Transports in detail ### BroadcastChannelTransport Same-origin, same-machine, zero infrastructure. Perfect for local development and for QA-ing UX without juggling two devices. ```ts new BroadcastChannelTransport(); ``` ### WebRTCTransport Peer-to-peer using the public PeerJS broker for signaling. Audio and data flow directly between browsers, so latency is excellent. Limitations: - The public broker is best-effort and rate-limits aggressive reconnects; supply your own `signalingUrl` (`npx peerjs --port 9000`) for anything beyond local demos. - Without TURN, NAT traversal can fail across some networks. Pass `iceServers` to add TURN. - Full mesh with a 12-slot pool — best for ≤ 12 participants. ```ts new WebRTCTransport({ iceServers: [ {urls: 'stun:stun.l.google.com:19302'}, // {urls: 'turn:turn.example.com:3478', username: 'u', credential: 'p'}, ], }); ``` #### Signaling recovery `WebRTCTransport.reconnectSignaling()` explicitly reopens a closed broker registration using the same local peer ID, without replacing live data channels. Concurrent calls share one attempt; a taken slot or unreachable broker rejects for caller-managed retry. A successful reopen prompts one discovery pass; `isSignalingOpen` reports broker health. Calls while signaling is already open are no-ops and do not reset the normal discovery backoff. This does not opt multiplayer sessions into automatic recovery or change the room's full-mesh policy. `protectOpenPeers: true` opts into protecting established DTLS connections from broker-ID reuse. Once the reliable data channel opens, routed LEAVE/OFFER/ANSWER/CANDIDATE messages and stale EXPIRE notifications cannot mutate or tear down that entry. Native channel closure or ICE/DTLS failure must end it before replacement; a transient `disconnected` state alone does not revoke it. The default is **false**: a different offer from a reused slot replaces the open connection, preserving the normal reload behavior. The phone companion enables this protection for its default WebRTC transport. With protection enabled, inbound offers must also name one of the room's 12 canonical slot IDs (excluding our own); off-room and arbitrary broker IDs are rejected before allocating a peer connection. `dropPeer(peerId)` locally closes just that peer, even with protection enabled, and emits the usual departure event for an open channel. This lets an application discard a connection that fails application-level authentication. It leaves the broker and other peers intact; call `discoverPeers()` explicitly for a prompt retry. `discoverPeers()` is an explicit, one-off discovery request for callers that need to react promptly to a departed peer. Same-turn duplicate requests are coalesced; a later real departure is not delayed by a cooldown. It does not reschedule or reset the normal discovery backoff. Ordinary multiplayer sessions do not opt into this automatically; the phone companion requests it only for an authenticated counterpart's departure when its recovery factory is enabled. Regular and explicit discovery share an in-flight probe reservation per slot; overlapping scans cannot start duplicate probes or offers. The reservation covers only the broker probe: it is released on completion, timeout, failure or disposal. Once initiation creates an entry, that entry prevents duplicate connections; unsettled native SDP promises never retain the probe reservation after teardown. #### Optional unreliable lane For latency-sensitive direct transport messages, opt into `new WebRTCTransport({unreliableChannel: true})` on both endpoints. Support is negotiated per peer through offer/answer metadata. Only after both agree and the reliable channel opens does the offerer create a separate `ordered: false, maxRetransmits: 0` data channel. Older peers without this metadata and peers with the option off keep a reliable-only connection. Unexpected unreliable channels are closed; optional-channel creation failures leave the reliable connection usable. The main `netblocks` channel must be created by the offerer, ordered, and fully reliable (no retransmission or packet-lifetime limit). Exact duplicate offers are ignored. By default a different offer replaces the slot's connection, including an open one. With `protectOpenPeers: true`, it can replace only a pending handshake; reloads must close their old connection cleanly or wait for native transport failure. In-flight probe deduplication is always enabled. Call `transport.sendUnreliable(bytes, peerId)`. **True means queued, not delivered**; packets may be lost or arrive out of order. It returns false when the lane is unavailable, the send fails, or the message exceeds a limit: - At most **1,200 bytes** per message, also bounded by `pc.sctp.maxMessageSize` when available (zero means no SCTP limit). - At most **16 KiB** for `bufferedAmount + bytes.byteLength`. It never falls back to the reliable channel. Received messages carry `event.detail.delivery === 'unreliable'`. Ordinary `send()` and NetSession traffic stay reliable and ordered. `supportsUnreliable` reports local opt-in, not a particular peer's negotiated readiness; a send can return false just after `peer-join` while the optional lane opens. BroadcastChannel provides a local test equivalent, while WebSocket does not. ### WebSocketTransport Connects to a small relay server included in the addon (`server/relay.js`). Only the Node.js relay needs the optional `ws` peer dependency; browser clients do not. Install it in the project where `xrblocks` is installed: ```sh npm i ws node node_modules/xrblocks/src/addons/netblocks/server/relay.js # or, in this repo: node src/addons/netblocks/server/relay.js ``` Then point the client at it: ```ts new WebSocketTransport({url: 'ws://localhost:8765'}); ``` The relay is dumb fan-out — under 200 LOC, no auth, no persistence. For production, add auth in front (e.g., reverse-proxy with an ID token check) and consider adding rate-limiting. --- ## Wire protocol All messages are JSON envelopes carrying a tagged union: | `type` | Direction | Purpose | | ------------------- | ------------- | -------------------------------------------------- | | `hello` | broadcast | Announce capabilities + display name on join. | | `welcome` | unicast | Bring a new peer up to speed on existing peers. | | `bye` | broadcast | Graceful leave. | | `pose` | broadcast | Compact binary head + hands snapshot (base64). | | `netobject` | broadcast | Replicated transform + optional state for one obj. | | `netobject.claim` | broadcast | Request ownership of a NetObject. | | `netobject.release` | broadcast | Drop ownership of a NetObject. | | `rpc` | uni/broadcast | Typed pub/sub event. | | `voice` | unicast | WebRTC SDP/ICE for spatial voice negotiation. | | `ping` / `pong` | reserved | Keepalive (currently unused). | Pose snapshots are encoded as a fixed 386-byte binary blob (head pose + two hands × 25 quantized joints) wrapped in base64 inside the JSON envelope. See `src/core/codec/PoseCodec.ts` for the byte layout. --- ## Samples See [`demos/netblocks/SAMPLES.md`](../../../demos/netblocks/SAMPLES.md). Highlights: - `demos/netblocks/basic/presence` — see remote heads (and hands in XR with hand tracking). - `demos/netblocks/basic/objects` — drag a shared cube; ownership transfers on grab. - `demos/netblocks/basic/events` — broadcast emoji bursts via the RPC bus. - `demos/netblocks/basic/voice` — push-to-talk spatial voice chat. WASD/mouse (or gamepad sticks) to walk around in 2D; the headset's pose drives it in XR. - `demos/netblocks/basic/transports` — switch transports at runtime. - `demos/netblocks/` — assembled "shared room" demo combining presence + objects + chat + emoji-burst RPC + voice. (Top-level headline demo.) All samples default to `BroadcastChannelTransport` (two tabs in the same browser, no signaling, no rate-limit risk). A small HUD lets you "Start new room" — that flips the URL to `?room=ABCD`, switches the sample to `WebRTCTransport`, and gives you a 4-character code to share with a friend on another device. The library's `joinRoom` default is still `WebRTCTransport`; only the in-tree samples opt into local mode for zero-friction first-run. --- ## Design choices and trade-offs - **Cooperative ownership over server authority.** netblocks does not assume a backend — most setups use peer-to-peer or a dumb relay. We resolve conflicts by preferring the lower peer id on race, which is good enough for pickup-and-throw semantics. For competitive/anti-cheat workloads, add a server-authoritative arbiter on top of the same protocol. - **JSON envelopes, binary pose payloads.** JSON keeps the protocol introspectable (open dev-tools and read traffic) without sacrificing pose bandwidth — pose is the only frame we send at frame-rate, so we spend the complexity budget there. - **Fixed 20 Hz pose / object updates.** Tunable, but defaults work for every browser/device combination we tested. Avatars use lerp/slerp smoothing so 20 Hz looks like 60 Hz. - **No CRDT.** Shared documents are out of scope. If you need them, layer Yjs or Automerge on top of `session.events.emit('crdt-update', ...)`. ## Security and threat model netblocks is a **demo / prototype** networking layer. It is meant for small, cooperative rooms (think hack-day demos, internal share-screen moments, classroom co-presence) where every participant is trusted to play nice. A hardening pass against malicious peers is **out of scope** for this addon. In particular: - **Ownership claims are cooperative.** Any peer can claim, hold, or release any `NetObject` simply by saying so. There is no central arbiter; conflicts are tie-broken by lexicographic peer id, not by authority. See `NetObjectRegistry`. - **Event sources are cooperative.** `session.events` handlers receive the transport-reported sender id, but payloads are not signed. A peer-to-peer transport cannot prevent a malicious peer from forging topics, spoofing payloads, or impersonating another peer. See `NetEvents`. - **Transports do not authenticate peers.** Both `BroadcastChannelTransport` and `WebRTCTransport` (via the public PeerJS broker) accept any browser tab that knows the room id. Treat room ids as the only access control. - **No rate limiting.** A misbehaving peer can saturate your room with presence updates or RPC traffic. For adversarial environments — anti-cheat, payments, anything where a malicious peer's actions cost real money or trust — terminate state at a server you control, validate every incoming message, and use netblocks only as a presence/transport veneer above that arbiter.