# @netless/appliance-extend-auto-draw-plugin

An intelligent auto-draw plugin for Netless Appliance that automatically recognizes hand-drawn sketches and suggests matching icons.

## Features

- 🎨 **Automatic Sketch Recognition**: Automatically analyzes hand-drawn sketches when using the pencil tool
- 🤖 **Dual Recognition Modes**: 
  - Google Handwriting API for quick recognition based on ink data
  - OpenRouter multi-modal AI models for advanced image-based recognition
- 🎯 **Icon Suggestions**: Fetches matching icons from Iconify based on recognition results
- ⚡ **Smart Replacement**: Replace hand-drawn sketches with professional icons with a single click
- ⏱️ **Configurable Delay**: Customizable delay before triggering recognition (default: 2000ms)

## Installation

```bash
npm install @netless/appliance-extend-auto-draw-plugin
# or
pnpm add @netless/appliance-extend-auto-draw-plugin
# or
yarn add @netless/appliance-extend-auto-draw-plugin
```

## Usage

### Basic Usage

```typescript
import { AutoDrawPlugin } from '@netless/appliance-extend-auto-draw-plugin';

// Create a container for displaying icon suggestions
const iconsContainer = document.createElement('div');
document.body.appendChild(iconsContainer);

// Initialize the plugin
const autoDrawPlugin = new AutoDrawPlugin({
  container: iconsContainer,
  delay: 2000, // Optional: delay in milliseconds before recognition
});

// Mount the plugin to your appliance control
control.pluginManager.addPlugin(autoDrawPlugin);
autoDrawPlugin.mount();
```

### Advanced Usage with OpenRouter API

For more accurate recognition using AI vision models:

```typescript
import { AutoDrawPlugin } from '@netless/appliance-extend-auto-draw-plugin';

const iconsContainer = document.createElement('div');
document.body.appendChild(iconsContainer);

const autoDrawPlugin = new AutoDrawPlugin({
  container: iconsContainer,
  apiKey: 'your-openrouter-api-key',
  customModel: 'openai/gpt-4o', // Optional: specify a custom model
  delay: 2000,
  uploadFile: async (file: File) => {
    // Upload the file to your OSS server and return the URL
    const formData = new FormData();
    formData.append('file', file);
    const response = await fetch('https://your-oss-server.com/upload', {
      method: 'POST',
      body: formData,
    });
    const data = await response.json();
    return data.url; // Return the image URL
  },
});

control.pluginManager.addPlugin(autoDrawPlugin);
autoDrawPlugin.mount();
```

## API

### `AutoDrawPlugin`

#### Constructor Options

```typescript
interface AutoDrawOptions {
  /** Manage your API key to access all models from OpenRouter */
  apiKey?: string;
  /** Custom models to use */
  customModel?: string;
  /** Container to render the icons */
  container: HTMLDivElement;
  /** Delay to render the icons, default is 2000ms */
  delay?: number;
  /**
   * If you upload the file to the oss server and return the URL address, undefined will be returned
   * @param file File object
   * @returns image url string
   */
  uploadFile?: (file: File) => Promise<string | undefined>;
}
```

#### Methods

- `mount()`: Mount the plugin and start listening to drawing events
- `unMount()`: Unmount the plugin and clean up resources
- `onDestroy()`: Called when the plugin is destroyed

## How It Works

1. **Drawing Detection**: The plugin monitors pencil tool usage and tracks hand-drawn strokes
2. **Recognition Trigger**: After a configurable delay (default 2000ms) after drawing stops, recognition is triggered
3. **Recognition Process**:
   - **Mode 1 (Google API)**: Analyzes ink data directly using Google's handwriting recognition API
   - **Mode 2 (OpenRouter)**: Captures a snapshot of the drawing area, uploads it, and uses AI vision models to identify the content
4. **Icon Fetching**: Based on recognition results, fetches matching icons from Iconify
5. **Icon Display**: Displays suggested icons in the provided container
6. **Icon Selection**: When a user clicks an icon, the hand-drawn sketch is replaced with the selected icon

## Requirements

- `@netless/appliance-plugin` >= 1.1.26

## License

MIT