# Rhino Binding for Web

## Rhino Speech-to-Intent engine

Made in Vancouver, Canada by [Picovoice](https://picovoice.ai)

Rhino is Picovoice's Speech-to-Intent engine. It directly infers intent from spoken commands within a given context of
interest, in real-time. For example, given a spoken command:

> Can I have a small double-shot espresso?

Rhino infers that the user would like to order a drink and emits the following inference result:

```json
{
  "isUnderstood": "true",
  "intent": "orderBeverage",
  "slots": {
    "beverage": "espresso",
    "size": "small",
    "numberOfShots": "2"
  }
}
```

Rhino is:

* using deep neural networks trained in real-world environments.
* compact and computationally-efficient, making it perfect for IoT.
* self-service. Developers and designers can train custom models using [Picovoice Console](https://console.picovoice.ai/).

## Compatibility

- Chrome / Edge
- Firefox
- Safari

## Requirements

The Rhino Web Binding uses [SharedArrayBuffer](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/SharedArrayBuffer).

Include the following headers in the response to enable the use of `SharedArrayBuffers`:

```
Cross-Origin-Opener-Policy: same-origin
Cross-Origin-Embedder-Policy: require-corp
```

Refer to our [Web demo](../../demo/web) for an example on creating a server with the corresponding response headers.

Browsers that don't support `SharedArrayBuffers` or applications that don't include the required headers will fall back to using standard `ArrayBuffers`. This will disable multithreaded processing.

### Restrictions

IndexedDB is required to use `Rhino` in a worker thread. Browsers without IndexedDB support
(i.e. Firefox Incognito Mode) should use `Rhino` in the main thread.

Multi-threading is only enabled for Rhino when using on a web worker.

## Installation

### Package

Using `Yarn`:

```console
yarn add @picovoice/rhino-web
```

or using `npm`:

```console
npm install --save @picovoice/rhino-web
```

### AccessKey

Rhino requires a valid Picovoice `AccessKey` at initialization. `AccessKey` acts as your credentials when using
Rhino SDKs.
You can get your `AccessKey` for free. Make sure to keep your `AccessKey` secret.
Signup or Login to [Picovoice Console](https://console.picovoice.ai/) to get your `AccessKey`.

## Usage

There are two methods to initialize Rhino:

### Public Directory

**NOTE**: Due to modern browser limitations of using a file URL, this method does __not__ work if used without hosting a server.

This method fetches [the model file](https://github.com/Picovoice/rhino/blob/master/lib/common/rhino_params.pv) from the public directory and feeds it to Rhino. Copy the model file into the public directory:

```console
cp ${RHINO_MODEL_FILE} ${PATH_TO_PUBLIC_DIRECTORY}
```

The same procedure can be used for the [Rhino context](https://github.com/Picovoice/rhino/tree/master/resources/contexts) (`.rhn`) files.

### Base64

**NOTE**: This method works without hosting a server, but increases the size of the model file roughly by 33%.

This method uses a base64 string of the model file and feeds it to Rhino. Use the built-in script `pvbase64` to base64 your model file:

```console
npx pvbase64 -i ${RHINO_MODEL_FILE} -o ${OUTPUT_DIRECTORY}/${MODEL_NAME}.js
```

The output will be a js file which you can import into any file of your project. For detailed information about `pvbase64`,
run:

```console
npx pvbase64 -h
```

The same procedure can be used for the [Rhino context](https://github.com/Picovoice/rhino/tree/master/resources/contexts) (`.rhn`) files.

### Rhino Model

Rhino saves and caches your model (`.pv`) and context (`.rhn`) files in the IndexedDB to be used by Web Assembly.
Use a different `customWritePath` variable to hold multiple model values and set the `forceWrite` value to true to force an overwrite of the model file.
If the model (`.pv`) or context (`.rhn`) files change, `version` should be incremented to force the cached model to be updated. Either `base64` or `publicPath` must be set to instantiate Rhino. If both are set, Rhino will use the `base64` parameter.

```typescript
// Context (.rhn)
const rhinoContext = {
  publicPath: ${CONTEXT_RELATIVE_PATH},
  // or
  base64: ${CONTEXT_BASE64_STRING},

  // Optionals
  customWritePath: 'custom_context',
  forceWrite: true,
  version: 1,
  sensitivity: 0.5,
}

// Model (.pv)
const rhinoModel = {
  publicPath: ${MODEL_RELATIVE_PATH},
  // or
  base64: ${MODEL_BASE64_STRING},

  // Optionals
  customWritePath: 'custom_model',
  forceWrite: true,
  version: 1,
}
```

Additional engine options are provided via the `options` parameter.
Set `processErrorCallback` to handle errors if an error occurs while processing audio.
Use `endpointDurationSec` and `requireEndpoint` to control the engine's endpointing behaviour.
An endpoint is a chunk of silence at the end of an utterance that marks the end of spoken command.

```typescript
// Optional. These are the default values
const options = {
  endpointDurationSec: 1.0,
  requireEndpoint: true,
  processErrorCallback: (error) => {},
}
```

### Initialize Rhino

Create a `inferenceCallback` function to get the results from the engine:

```typescript
function inferenceCallback(inference) {
  if (inference.isFinalized) {
    if (inference.isUnderstood) {
      console.log(inference.intent)
      console.log(inference.slots)
    }
  }
}
```

Create an `options` object and add a `processErrorCallback` function if you would like to catch errors:

```typescript
function processErrorCallback(error: string) {
...
}
options.processErrorCallback = processErrorCallback;
```

Initialize an instance of `Rhino` in the main thread:

```typescript
const handle = await Rhino.create(
  ${ACCESS_KEY},
  rhinoContext,
  inferenceCallback,
  rhinoModel,
  options // optional options
);
```

Or initialize an instance of `Rhino` in a worker thread:

```typescript
const handle = await RhinoWorker.create(
  ${ACCESS_KEY},
  rhinoContext,
  inferenceCallback,
  rhinoModel,
  options // optional options
);
```

### Process Audio Frames

The result is received from `inferenceCallback` as defined above.

```typescript
function getAudioData(): Int16Array {
... // function to get audio data
  return new Int16Array();
}
for (; ;) {
  await handle.process(getAudioData());
  // break on some condition
}
```

### Clean Up

Clean up used resources by `Rhino` or `RhinoWorker`:

```typescript
await handle.release();
```

### Terminate

Terminate `RhinoWorker` instance:

```typescript
await handle.terminate();
```

## Contexts

Create custom contexts using the [Picovoice Console](https://console.picovoice.ai/).
Train and download a Rhino context file (`.rhn`) for the target platform `Web (WASM)`.
This model file can be used directly with `publicPath`, but, if `base64` is preferable, convert the `.rhn` file to a
base64 JavaScript variable using the built-in `pvbase64` script:

```console
npx pvbase64 -i ${CONTEXT_FILE}.rhn -o ${CONTEXT_BASE64}.js -n ${CONTEXT_BASE64_VAR_NAME}
```

Similar to the model file (`.pv`), context files (`.rhn`) are saved in IndexedDB to be used by Web Assembly.
Either `base64` or `publicPath` must be set for the context to instantiate Rhino.
If both are set, Rhino will use the `base64` model.

```typescript
const contextModel = {
  publicPath: "${CONTEXT_RELATIVE_PATH}",
  // or
  base64: "${CONTEXT_BASE64_STRING}",
}
```

## Switching Languages

In order to make inferences in different language you need to use the corresponding model file (`.pv`).
The model files for all supported languages are available [here](https://github.com/Picovoice/rhino/tree/master/lib/common).

## Train Models over API

You can train models over API without going to the console:

```javascript
Rhino.trainContextFromDynamicSlots(
  "${ACCESS_KEY}",   // AccessKey obtained from Picovoice Console (https://console.picovoice.ai/)
  "${WRITE_PATH}",   // Custom path/key used to store the trained model in IndexedDB
  "${LANGUAGE}",     // Two-character language code
  "${YAML_CONTEXT}", // RhinoContext object containing a base64 representation of or path to a public binary Rhino context model
  "${MODEL}",        // RhinoModel configuration to be used for training
  {
    "${SLOT_NAME}": new Set(["${SLOT1}", "${SLOT2}"])
  }                 // Additional slot values
);
```

(or)

```javascript
Rhino.trainContextFromYaml(
  "${ACCESS_KEY}",     // AccessKey obtained from Picovoice Console (https://console.picovoice.ai/)
  "${WRITE_PATH}",     // Custom path/key used to store the trained model in IndexedDB
  "${LANGUAGE}",       // Two-character language code
  "${YAML_CONTENT}",   // YAML configuration in string
);
```

`trainContextFromDynamicSlots` is better suited if you would like the add additional slot values to your current Rhino context.

Check [Rhino Model API](https://picovoice.ai/docs/model-api/rhino/) docs for a list of supported languages.

## Demo

For example usage refer to our [Web demo application](https://github.com/Picovoice/rhino/tree/master/demo/web).