# svelte-tuio

A Svelte 5 library for handling TUIO (Tangible User Interface Objects) protocol events through WebSocket connections. Built with runes and reactive state management for tracking tangible objects on interactive surfaces.

## Features

- **WebSocket-based TUIO Handler** - Manages TUIO protocol events.
- **Reactive State** - Built with Svelte 5 runes for automatic reactivity.
- **Tangibles Manager** - Track and manage multiple tangible objects with positions and rotations.
- **Debug Component** - Visual debugger with real-time updates.
- **Simple Hook API** - Easy context-based access via `useTUIO()`.

## Overview

This library is designed to receive TUIO messages from a **TUIO server** (such as TouchDesigner) and make them available in your Svelte web frontend.

### How It Works

1. **TUIO Server** - Your TouchDesigner (or other TUIO) project runs a WebSocket server
2. **WebSocket Connection** - Pass a WebSocket instance to `TUIOProvider` that connects to your TUIO server
3. **Event Transmission** - The TUIO server transmits touch and tangible events over the WebSocket
4. **Reactive State** - The library automatically updates reactive state that your Svelte components can consume

### Requirements

- A TUIO server (e.g., TouchDesigner) running a WebSocket server on the same network
- The server should send TUIO events in JSON format with the following structure:
  ```json
  {
    "timestamp": 123456789,
    "touchesStart": [...],
    "touchesMove": [...],
    "touchesEnd": [...],
    "touchesNoChange": [...]
  }
  ```

## Installation

```bash
npm install svelte-tuio
```

## Usage

### Basic Setup

The simplest way to use the library is with `TUIOProvider`, which automatically creates a `TUIOHandler` for you:

```svelte
<script>
	import { TUIOProvider } from 'svelte-tuio';
	import { SvelteSocket } from '@hardingjam/svelte-socket';
	let { children } = $props();

	// Create WebSocket connection to your TUIO server
	// Replace with your TouchDesigner WebSocket server address
	const svelteSocket = new SvelteSocket('ws://localhost:8080');
</script>

<TUIOProvider {svelteSocket}>
	<!-- Your app components can now access TUIO data via useTUIO() -->
	{@render children?.()}
</TUIOProvider>
```

**With Configuration:**

You can also pass configuration options directly to `TUIOProvider`:

```svelte
<script>
	import { TUIOProvider } from 'svelte-tuio';
	import { SvelteSocket } from '@hardingjam/svelte-socket';
	let { children } = $props();

	const svelteSocket = new SvelteSocket('ws://localhost:8080');
</script>

<TUIOProvider
	{svelteSocket}
	config={{
		onFingerTouchEnd: (u, v) => {
			console.log(`Touch at ${u}, ${v}`);
		},
		throttleTime: 100
	}}
>
	{@render children?.()}
</TUIOProvider>
```

**Pre-configured Handler:**

Alternatively, you can create your own `TUIOHandler` with custom callbacks and pass it to the provider:

```svelte
<script>
	import { TUIOProvider, TUIOHandler } from 'svelte-tuio';
	import { SvelteSocket } from '@hardingjam/svelte-socket';
	let { children } = $props();

	const svelteSocket = new SvelteSocket('ws://localhost:8080');

	const tuioHandler = new TUIOHandler({
		svelteSocket,
		onFingerTouchEnd: (u, v) => {
			console.log(`Touch at ${u}, ${v}`);
		},
		onPlaceTangible: (touch) => {
			console.log(`Tangible ${touch.classId} placed`);
		}
	});
</script>

<TUIOProvider {tuioHandler}>
	<!-- Your app components can now access TUIO data via useTUIO() -->
	{@render children?.()}
</TUIOProvider>
```

### TouchDesigner Setup

Your TouchDesigner project needs to run a WebSocket server that sends TUIO events in the JSON format shown above.

> **Note:** An example Python callbacks file for TouchDesigner is included in this package at [`src/python/example_tuio_callbacks.py`](https://github.com/monomango-jamie/svelte-tuio/blob/main/src/python/example_tuio_callbacks.py)

### Using the Hook

Access reactive TUIO state in your components using `$derived`:

```svelte
<script>
	import { useTUIO } from 'svelte-tuio';

	const tuioHandler = useTUIO();

	// Access reactive state with $derived
	let tangibles = $derived(tuioHandler.tangiblesManager.tangibles);
	let tangibleClassIds = $derived(tuioHandler.tangiblesManager.tangibleClassIds);
	let isConnected = $derived(tuioHandler.svelteSocket.isConnected);

	// Derived computations
	let tangibleCount = $derived(tangibles.length);
	let hasTangibles = $derived(tangibleCount > 0);
</script>

<div>
	<p>Connected: {isConnected}</p>
	<p>Active tangibles: {tangibleCount}</p>

	{#if hasTangibles}
		{#each tangibles as tangible (tangible.classId)}
			<div>
				Tangible {tangible.classId} at ({tangible.u.toFixed(2)}, {tangible.v.toFixed(2)})
			</div>
		{/each}
	{:else}
		<p>No tangibles detected</p>
	{/if}
</div>
```

### Debug Component

```svelte
<script>
	import { TangiblesDebugger } from 'svelte-tuio';
</script>

<TangiblesDebugger />
```

## API

### `TUIOProvider`

Wrapper component that sets up TUIO context for your app.

**Props:**

- `tuioHandler` - Optional pre-configured TUIOHandler instance. If provided, `svelteSocket` and `config` are ignored.
- `svelteSocket` - SvelteSocket instance from `@hardingjam/svelte-socket`. Required if `tuioHandler` is not provided.
- `config` - Optional configuration object for creating a TUIOHandler. Only used if `tuioHandler` is not provided.

**Usage:**

You can use `TUIOProvider` in two ways:

1. **Automatic handler creation** - Pass `svelteSocket` (and optionally `config`):

   ```svelte
   <TUIOProvider {svelteSocket} config={{ throttleTime: 100 }}>
   	<!-- Your app -->
   </TUIOProvider>
   ```

2. **Pre-configured handler** - Create your own `TUIOHandler` and pass it:
   ```svelte
   <TUIOProvider {tuioHandler}>
   	<!-- Your app -->
   </TUIOProvider>
   ```

### `TUIOHandler`

Main class for managing TUIO WebSocket connections.

```typescript
import type { TUIOHandlerConfig } from 'svelte-tuio';
import { SvelteSocket } from '@hardingjam/svelte-socket';

const svelteSocket = new SvelteSocket('ws://localhost:8080');

const config: TUIOHandlerConfig = {
	svelteSocket,
	onFingerTouchEnd: (u, v) => {
		/* ... */
	},
	onFingerTouchStart: (u, v) => {
		/* ... */
	},
	onPlaceTangible: (touch) => {
		/* ... */
	},
	onRemoveTangible: (touch) => {
		/* ... */
	},
	onMoveTangible: (touch) => {
		/* ... */
	},
	throttleTime: 100, // Throttle callbacks to once per 100ms
	touchZones: [
		// Optional: initial touch zones to register
		{
			id: 'my-zone',
			u: 0.1,
			v: 0.1,
			normalisedWidth: 0.3,
			normalisedHeight: 0.3,
			onPlaceTangible: (touch) => {
				/* ... */
			}
		}
	]
};

const handler = new TUIOHandler(config);

// Properties
handler.tangiblesManager; // TangiblesManager instance
handler.touchZones; // Array of touch zones ($state)
handler.svelteSocket; // SvelteSocket instance
```

### Touch Zones

Touch zones allow you to define regions of the screen and attach callbacks for touch/tangible events within those regions. Touch zones are **user-managed** - you register zones and implement your own logic to check if touches fall within zones.

You can register touch zones in two ways:

**1. In the constructor:**

```typescript
import type { TouchZone, TUIOHandlerConfig } from 'svelte-tuio';

const zone: TouchZone = {
	id: 'my-zone',
	u: 0.1, // Horizontal position (0=left, 1=right)
	v: 0.1, // Vertical position in TUIO coords (0=bottom, 1=top)
	normalisedWidth: 0.3,
	normalisedHeight: 0.3,
	onPlaceTangible: (touch) => {
		console.log('Tangible placed in zone', touch);
	},
	onRemoveTangible: (touch) => {
		console.log('Tangible removed from zone', touch);
	},
	onMoveTangible: (touch) => {
		console.log('Tangible moved in zone', touch);
	},
	onTouchStart: (touch) => {
		console.log('Finger touch started in zone', touch);
	},
	onTouchMove: (touch) => {
		console.log('Finger touch moved in zone', touch);
	},
	onTouchEnd: (touch) => {
		console.log('Finger touch ended in zone', touch);
	}
};

const config: TUIOHandlerConfig = {
	svelteSocket,
	touchZones: [zone] // Register zones at creation time
};

const handler = new TUIOHandler(config);
```

**2. Using the `registerTouchZone` method:**

```typescript
import type { TouchZone } from 'svelte-tuio';

const tuioHandler = useTUIO();

// Define a touch zone
const zone: TouchZone = {
	id: 'my-zone',
	u: 0.1, // Horizontal position (0=left, 1=right)
	v: 0.1, // Vertical position in TUIO coords (0=bottom, 1=top)
	normalisedWidth: 0.3,
	normalisedHeight: 0.3,
	onPlaceTangible: (touch) => {
		console.log('Tangible placed in zone', touch);
	},
	onRemoveTangible: (touch) => {
		console.log('Tangible removed from zone', touch);
	},
	onMoveTangible: (touch) => {
		console.log('Tangible moved in zone', touch);
	},
	onTouchStart: (touch) => {
		console.log('Finger touch started in zone', touch);
	},
	onTouchMove: (touch) => {
		console.log('Finger touch moved in zone', touch);
	},
	onTouchEnd: (touch) => {
		console.log('Finger touch ended in zone', touch);
	}
};

// Register the zone
tuioHandler.registerTouchZone(zone);

// Remove a zone when done
tuioHandler.unregisterTouchZone('my-zone');
```

**Note:** The library automatically handles hit detection. When a touch or tangible event occurs, the handler checks if it falls within any registered zone's bounds and calls the appropriate callbacks.

### `TangiblesManager`

Manages tangible objects with reactive state. All tangible operations are handled internally by `TUIOHandler`.

```typescript
// Public Properties (reactive $state - use with $derived)
manager.tangibles; // TUIOTouch[] - All active tangibles with full data
manager.tangibleClassIds; // number[] - Just the class IDs for performance
```

**Example: Accessing Reactive State**

```svelte
<script>
	import { useTUIO } from 'svelte-tuio';

	const tuioHandler = useTUIO();
	const manager = tuioHandler.tangiblesManager;

	// Use $derived to reactively access state
	let tangibles = $derived(manager.tangibles);
	let classIds = $derived(manager.tangibleClassIds);

	// Derived computations update automatically
	let specificTangible = $derived(tangibles.find((t) => t.classId === 14));
	let hasSpecificTangible = $derived(classIds.includes(14));
</script>

{#if specificTangible}
	<p>Tangible 14 is at position: {specificTangible.u}, {specificTangible.v}</p>
{/if}
```

### `useTUIO()`

Hook to access TUIOHandler from context.

```typescript
const tuioHandler = useTUIO();
```

## Custom Event Callbacks

The library provides optional callbacks for all TUIO events. By default:

- **`onFingerTouchEnd`** - Does nothing (no-op)
- **`onFingerTouchStart`** - Does nothing (no-op)
- **Tangible events** (`onPlaceTangible`, `onRemoveTangible`, `onMoveTangible`) - Automatically update TangiblesManager state, custom callbacks are called in addition to state updates

> **Tip:** If you want finger touches to simulate clicks on HTML elements, you can implement a click handler like this:
>
> ```typescript
> const tuioHandler = new TUIOHandler({
> 	svelteSocket,
> 	onFingerTouchEnd: (u, v) => {
> 		const x = u * window.innerWidth;
> 		const y = (1 - v) * window.innerHeight; // Invert v for screen coordinates
> 		const element = document.elementFromPoint(x, y);
> 		if (element instanceof HTMLElement) {
> 			element.click();
> 		}
> 	}
> });
> ```

**Custom Callbacks:**

You can provide custom callbacks for any TUIO event:

```svelte
<script>
	import { TUIOProvider, TUIOHandler } from 'svelte-tuio';
	import { SvelteSocket } from '@hardingjam/svelte-socket';

	const svelteSocket = new SvelteSocket('ws://localhost:8080');

	const tuioHandler = new TUIOHandler({
		svelteSocket,
		onFingerTouchEnd: (u, v) => {
			console.log(`Finger touch end at: ${u}, ${v}`);
		},
		onFingerTouchStart: (u, v) => {
			console.log(`Finger touch start at: ${u}, ${v}`);
		},
		onPlaceTangible: (touch) => {
			console.log(`Tangible ${touch.classId} placed`);
		},
		onRemoveTangible: (touch) => {
			console.log(`Tangible ${touch.classId} removed`);
		},
		onMoveTangible: (touch) => {
			console.log(`Tangible ${touch.classId} moved to ${touch.u}, ${touch.v}`);
		}
	});
</script>

<TUIOProvider {tuioHandler}>
	<!-- Your app -->
</TUIOProvider>
```

> **Note:** Custom tangible callbacks are called _in addition to_ the automatic TangiblesManager updates, allowing you to add extra functionality without losing the built-in state management.

### Throttling Callbacks

For performance optimization, especially with high-frequency touch events, you can throttle callback invocations using the `throttleTime` option:

```typescript
const tuioHandler = new TUIOHandler({
	svelteSocket,
	throttleTime: 100, // Minimum 100ms between callback invocations
	onMoveTangible: (touch) => {
		// This will only be called at most once per 100ms per tangible
		console.log(`Tangible ${touch.classId} moved`);
	}
});
```

**How it works:**

- `throttleTime` sets the minimum time (in milliseconds) between callback invocations
- Each callback type (fingerTouchStart, fingerTouchEnd, etc.) is throttled independently
- For tangible callbacks, throttling is per tangible (by classId)
- TangiblesManager state updates are NOT throttled (state always stays up-to-date)
- Set to `0` (default) to disable throttling

**Use cases:**

- Throttle `onMoveTangible` to reduce expensive operations during dragging
- Throttle touch events to prevent UI lag from rapid touches
- Improve performance on lower-end devices

## TypeScript Support

Full TypeScript support with exported types:

```typescript
import type { TUIOTouch, TUIOEvent, TouchZone, TUIOHandlerConfig } from 'svelte-tuio';
```

## Development

```bash
npm install
npm run dev
npm run test
npm run build
```

## License

MIT