# soupdraw

[![npm](https://img.shields.io/npm/v/soupdraw.svg)](https://www.npmjs.com/package/soupdraw)
[![license](https://img.shields.io/npm/l/soupdraw.svg)](./LICENSE)

Draw on your webcam with **hand gestures**, on-device. A tiny loader around the
**full SoupDraw pipeline**, the exact engine that powers the
[SoupDraw browser extension](https://draw.soupup.ai).

Give it a mount point, call `start()`, and you get back an **augmented
`MediaStream`** (your camera with hand-drawn annotations) plus a simple control
API. The camera and hand tracking never leave the browser.

> **Live demo:** [draw.soupup.ai](https://draw.soupup.ai)

```js
import { SoupDraw } from "soupdraw";

const sd = new SoupDraw({ mount: "#stage" });
const stream = await sd.start();   // pipe into a call, a recording, or a <video>

sd.setColor("#00F0FF");
sd.on("status", (s) => console.log(s.gesture));
```

## Install

```sh
npm install soupdraw
```

Or from a CDN, no build step:

```html
<div id="stage" style="max-width:720px"></div>
<script type="module">
  import { SoupDraw } from "https://esm.sh/soupdraw";
  await new SoupDraw({ mount: "#stage" }).start();
</script>
```

## Full parity with the extension

The SDK runs the real pipeline, so every gesture and feature is the same:

| Gesture | Does |
| :-- | :-- |
| 🤏 Pinch (thumb + index) | Draw |
| 👍 Thumb-out fist | Erase (eraser rides the thumb tip) |
| ✌️ Victory | Move a shape, or drag a box to marquee-select |
| ✊ Closed fist | Pan the canvas |
| 🖐️🖐️ Two-hand five-finger pinch | Scale, rotate, and pan (the selection, or everything) |
| 🖐️ Five-finger pinch | Restore a board from the history strip |
| ✊✊ Double fist-clench | Clear |

Plus: a **history strip** you can restore cleared boards from, a **minimap**,
**spotlight** (dim + glow on your hands), **whiteboard** mode, **shape-assist**
(rough strokes snap to clean shapes), and guided **per-user pinch calibration**.

## API

```ts
new SoupDraw(options)
```

| Option | Default | Notes |
| :-- | :-- | :-- |
| `mount` | — | selector/element to inject the output `<video>` into |
| `video` | — | bring your own `<video>` instead of `mount` |
| `color` | `"#ff2d55"` | pen colour |
| `size` | `6` | pen size |
| `mirror` | `true` | selfie-mirror the camera |
| `assist` | `true` | snap rough strokes into clean shapes |
| `history` | `true` | history strip (restore cleared boards) |
| `minimap` | `false` | show a minimap of off-frame drawings |
| `spotlight` | `false` | dim the feed and glow around hands |
| `whiteboard` | `false` | draw on a solid board instead of the camera |
| `boardColor` | `"#14151a"` | whiteboard colour |
| `runtimeBase` | package CDN | where to load the runtime + recognizer from |

**Methods:** `start()` → `Promise<MediaStream>`, `stop()`, `setColor`, `setSize`,
`setMirror`, `setAssist`, `setHistory`, `setMinimap`, `setSpotlight`,
`setWhiteboard`, `setBoardColor`, `setBinding(gesture, action)`, `undo`, `redo`,
`clear`, `calibrate`, `on(event, cb)`.

**Events:** `ready`, `status`
(`{ gesture, hands, strokes, drawing, erasing, transforming, modelReady, historyMode, selection }`),
`error`, `calibrated`, `stop`.

## How it works

On `start()`, the SDK injects two things (from the package's CDN by default, or
your `runtimeBase`): the **pipeline runtime** (engine + gesture catalog + the
compositor) and a hidden **recognizer iframe** that runs MediaPipe. It then drives
everything by config messages. The pipeline patches `getUserMedia` so it can hand
back the augmented feed, so `start()` returns a `MediaStream` that is a drop-in
replacement for a raw camera anywhere a `MediaStream` works.

The ~20 MB hand model and MediaPipe wasm load once from public CDNs. To run with no
third-party fetch, self-host the runtime (`dist/`) and MediaPipe assets and set
`runtimeBase`.

## Status

`0.2.x`. Full gesture and feature parity with the extension. Issues and use cases
welcome.

## License

MIT © Abhishek Janjalkar ([Soup Up](https://soupup.ai)). Runs
[MediaPipe Tasks Vision](https://www.npmjs.com/package/@mediapipe/tasks-vision)
(Apache-2.0) in the recognizer iframe.