# Banuba WebAR Agora extension

`@banuba/agora-extension` - Agora extension and video processor based on `@banuba/webar`.

Take a look at the [`@banuba/webar`](https://www.npmjs.com/package/@banuba/webar) npm package and [Banuba WebAR docs](https://docs.banuba.com/face-ar-sdk-v1/web/web_overview) for a deep dive.

## Quickstart

### CDN

Via `<script>` tag:

```html
<div id="video-container" style="width: 640px; height: 640px"></div>

<!-- Exposes AgoraRTC variable on window -->
<script src="https://cdn.jsdelivr.net/npm/agora-rtc-sdk-ng@4.16.1/AgoraRTC_N-production.min.js"></script>
<!-- Exposes BanubaSDK variable on window -->
<script src="https://cdn.jsdelivr.net/npm/@banuba/webar/dist/BanubaSDK.browser.js"></script>
<!-- Exposes BanubaAgoraExtension variable on window -->
<script src="https://cdn.jsdelivr.net/npm/@banuba/agora-extension/dist/index.browser.js"></script>

<script>
  const banuba = new BanubaAgoraExtension.Extension({
    clientToken: "xxx-xxx-xxx",
    locateFile: BanubaSDK.locateFile
  })
  AgoraRTC.registerExtensions([banuba])

  const processor = banuba.createProcessor()
  await processor.addModule(
    ...["face_tracker", "background"].map((m) => "https://cdn.jsdelivr.net/npm/@banuba/webar/dist/modules/" + m + ".zip"),
  )
  const video = await AgoraRTC.createCameraVideoTrack()

  video.pipe(processor).pipe(video.processorDestination)
  video.play(document.querySelector("#video-container"))
</script>
```

Via `<script type="module">` tag:

```html
<div id="video-container" style="width: 640px; height: 640px"></div>

<!-- Exposes AgoraRTC variable on window -->
<script src="https://cdn.jsdelivr.net/npm/agora-rtc-sdk-ng@4.16.1/AgoraRTC_N-production.min.js"></script>
<!-- Exposes BanubaSDK variable on window -->
<script src="https://cdn.jsdelivr.net/npm/@banuba/webar/dist/BanubaSDK.browser.js"></script>

<script type="module">
  import * as BanubaAgoraExtension from "https://cdn.jsdelivr.net/npm/@banuba/agora-extension/dist/index.js"

  const banuba = new BanubaAgoraExtension.Extension({
    clientToken: "xxx-xxx-xxx",
    locateFile: BanubaSDK.locateFile
  })
  AgoraRTC.registerExtensions([banuba])

  const processor = banuba.createProcessor()
  await processor.addModule(
    ...["face_tracker", "background"].map(
      (m) => "https://cdn.jsdelivr.net/npm/@banuba/agora-extension/dist/modules/" + m + ".zip",
    ),
  )
  const video = await AgoraRTC.createCameraVideoTrack()

  video.pipe(processor).pipe(video.processorDestination)
  video.play(document.querySelector("#video-container"))
</script>
```

### NPM

```bash
npm i --save agora-rtc-sdk-ng @banuba/agora-extension
```

```js
import AgoraRTC from "agora-rtc-sdk-ng"
import { Extension } from "@banuba/agora-extension"
import data from "@banuba/webar/dist/BanubaSDK.data"
import wasm from "@banuba/webar/dist/BanubaSDK.wasm"
import simd from "@banuba/webar/dist/BanubaSDK.simd.wasm"
import FaceTracker from "@banuba/webar/dist/modules/face_tracker.zip"
import Background from "@banuba/webar/dist/modules/background.zip"

const banuba = new Extension({
  clientToken: "xxx-xxx-xxx",
  locateFile: {
    "BanubaSDK.data": data,
    "BanubaSDK.wasm": wasm,
    "BanubaSDK.simd.wasm": simd,
  },
})
AgoraRTC.registerExtensions([banuba])

const processor = banuba.createProcessor()
await processor.addModule(FaceTracker, Background)
const video = await AgoraRTC.createCameraVideoTrack()

video.pipe(processor).pipe(video.processorDestination)
// assume the page has a div element with id="video-container" and `width` and `height` set
video.play(document.querySelector("#video-container"))
```

Read the [Integration tutorials](https://docs.banuba.com/face-ar-sdk-v1/web/web_tutorials_integrations) intro for the explanation of the `locateFile` option.

Read the [Bundlers](https://docs.banuba.com/face-ar-sdk-v1/generated/typedoc/#bundlers) section of the [Integrations guides](https://docs.banuba.com/face-ar-sdk-v1/generated/typedoc/#integrations) for more receipts for different bundlers.

## API Reference

### Extension 
The `Extension` constructor accepts all the same options as the [Player.create()](https://docs.banuba.com/face-ar-sdk-v1/generated/typedoc/classes/Player.html#create) method from Banuba WebAR SDK, most noticeable options are:

- `clientToken` - Banuba Client Token, required. See [Obtaining Banuba Client token](#obtaining-banuba-client-token) for details
- `locateFile` - specifies where to look for the Banuba WebAR’s `.wasm` and `.data` files, optional. Check the [Integration tutorials](https://docs.banuba.com/face-ar-sdk-v1/web/web_tutorials_integrations) intro for explanation.
- `devicePixelRatio` - image scaling factor for HiDPI devices, optional. If not specified it has a value of `1` as a reasonable default for WebRTC AR.

#### `Extension.checkCompatibility()` 

Checks if the browser is capable to run the `Extension` and `Processor`:

```ts
const banuba = new Extension({
  clientToken: "xxx-xxx-xxx",
  locateFile: BanubaSDK.locateFile
})

if (!banuba.checkCompatibility()) alert("The browser does not support the extension :(")
```

### Processor

The `Processor` instance created by the `Extension.createProcessor()` method has the following methods:

#### `Processor.addModule(url | WebArModule)`

Adds AR modules (neural networks) to the processor, so effects can consume them:

```ts
// This is a syntax sugar over
// import { Module } from "@banuba/agora-extension"
// const face_tracker = new Module("/path/to/face_tracker.zip")
// await processor.addModule(face_tracker)

await processor.addModule("/path/to/face_tracker.zip")
```

See the [WebAR API Reference](https://docs.banuba.com/face-ar-sdk-v1/generated/typedoc/classes/Module.html) for the list of available AR modules and usage code examples.

Also you may want to check the docs of the [Player.addModule()](https://docs.banuba.com/face-ar-sdk-v1/generated/typedoc/classes/Player.html#addModule)from Banuba WebAR SDK as the processor’s method is a simple wrapper around the [Player.addModule()](https://docs.banuba.com/face-ar-sdk-v1/generated/typedoc/classes/Player.html#addModule).

#### `Processor.applyEffect(url | WebArEffect)`

Applies an AR effect to the processor:

```ts
// This is a syntax sugar over
// import { Effect } from "@banuba/agora-extension"
// const octopus = new Effect("/path/to/Octopus.zip")
// await processor.applyEffect(octopus)

await processor.applyEffect("/path/to/Octopus.zip")
```

You can download and use test effect from the [Demo face filters](https://docs.banuba.com/face-ar-sdk-v1/overview/demo_face_filters) page.

Also you may want to check the  docs of the [Player.applyEffect()](https://docs.banuba.com/face-ar-sdk-v1/generated/typedoc/classes/Player.html#applyEffect) from Banuba WebAR SDK as the processor’s method is a simple wrapper around the [Player.applyEffect()](https://docs.banuba.com/face-ar-sdk-v1/generated/typedoc/classes/Player.html#applyEffect).

#### `Processor.clearEffect()`

Clears the applied effect if any:


```ts
await processor.applyEffect("/path/to/Octopus.zip")

// ... a few moments later ...

processor.clearEffect()
```

#### `Processor.destroy()`

Destroys the processor instance, frees up resources and allocated memory:

```ts
processor.destroy()
```

Try to not to forget to destroy the processor instance once it’s not needed to [avoid accidental memory leaks](https://docs.banuba.com/face-ar-sdk-v1/web/web_known_issues#page-running-webar-sdk-consumes-too-much-memory).

The processor instance con not be re-used once it’s destroyed.

## Obtaining Banuba Client token

[Banuba Client token](https://docs.banuba.com/face-ar-sdk-v1/overview/token_management) is required to get Banuba SDK Web AR working.

To receive a new trial client token please fill in the form on [banuba.com](https://www.banuba.com) website, or contact us via info@banuba.com.

## Resources and links

- [Banuba WebAR overview](https://docs.banuba.com/face-ar-sdk-v1/web/web_overview)

- [Banuba WebAR demo applications](https://docs.banuba.com/face-ar-sdk-v1/overview/getting_started#banuba-sdk-demo-applications)

- [Demo face filters](https://docs.banuba.com/face-ar-sdk-v1/overview/demo_face_filters)

- [Banuba WebAR JS API docs](https://docs.banuba.com/face-ar-sdk-v1/generated/typedoc/)

- [Changelog](https://docs.banuba.com/face-ar-sdk-v1/overview/releases)