# AutoCapture

This project contains Datachecker's AutoCapture tool, that captures images of paper documents + identity documents (ID/Passport/Driver license). The tool only takes a capture once a document is detected and it passes the quality control.

The tool will be run in the browser and is therefore written in JavaScript.

## Trigger mechanism

The tool performs the following checks:

- Is the environment not too dark (under exposure)?
- Is there a document?
- Is the detected document not too far?
- Is the detected document not too close?
- Is the image sharp?

## Prerequisites

Please visit [Datachecker API documentation](https://developer.datachecker.nl/).

- Datachecker [OAuth Token](#oauth-token)
- Datachecker [SDK Token](#sdk-token)

## Compatibility

The SDK requires a browser that supports at least ECMAScript 12 (ES12). It is highly recommended to use the latest version of your preferred browser to ensure compatibility and access to the latest features and security updates.

Using the latest browser versions will ensure that all modern JavaScript features required by the SDK are supported.

## Steps

1. Request [OAUTH Token](#oauth-token)
2. Put OAuth in header
3. SDK [configuration](#configuration) (add SDK token)
4. Run SDK

## OAuth Token

Datachecker uses OAuth authorization. In order to request the [SDK token](#sdk-token) you will need to provide a valid OAuth token in the header.

Example header:

```javascript
header = {'Authorization': `Bearer ${response.accessToken}`}
```

This OAuth token can be retrieved with the [Datachecker OAuth Token API](https://developer.datachecker.nl/?urls.primaryName=v2#/ProductApi/ProductApi_OAuthToken). The scope `"productapi.sdk.read"` needs to be present to make use of the [SDK token](#sdk-token). If this scope is missing you will not be able to retrieve an SDK token.

Example OAuth:

```javascript
fetch(<BASE_ENDPOINT>+"/oauth/token", {
    method: 'POST',
    body: JSON.stringify({
        "clientId": <CLIENTID>,
        "clientSecret": <CLIENTSECRET>,
        "scopes": [
            "productapi.sdk.read",
        ]
    })
})
.then(response => response.json())
```

Note: Contact Datachecker for *client_id* and *client_secret*.

## SDK Token

The SDK is locked. In order to use the SDK in production a *token* is required. The application can only be started with a valid token. This token is a `base64` string. The token can be generated by calling the [Datachecker SDK Token API](https://developer.datachecker.nl/?urls.primaryName=v2#/ProductApi/ProductApi_SdkToken).

Example:

```javascript
fetch(<BASE_ENDPOINT>+"/sdk/token?customer_reference=<CUSTOMER>&services=AUTO_CAPTURE", {
    method: 'GET',
    headers: {
        'Accept': 'application/json',
        'Content-Type': 'application/json',
        'Authorization': `Bearer <ACCESSTOKEN>`
    }
})
.then(response => response.json())
```

## Configuration

To run this tool, you will need initialise with the following variables.

| **ATTRIBUTE**       | **FORMAT**              | **DEFAULT VALUE**                           | **EXAMPLE**                                 | **NOTES**                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        |
| ------------------- | ----------------------- | ------------------------------------------- | ------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| `ALLOWED_DOCUMENTS` | object                  | see [ALLOWED DOCUMENTS](#allowed-documents) | see [ALLOWED DOCUMENTS](#allowed-documents) | **optional**<br> Enable or disable flipping of certain documents.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                |
| `APPROVAL`          | bool                    | `false`                                     | `false`                                     | **optional**<br> Approval screen after capture as an extra quality check.                                                                                                                                                                                                                                                                                                                                                                                                                                                                        |
| `ASSETS_FOLDER`     | string                  | `""`                                        | `"../"`                                     | **optional**<br> Specifies location of **locally hosted** assets folder. (see [Asset Fetching Configuration](#asset-fetching-configuration))                                                                                                                                                                                                                                                                                                                                                                                                     |
| `ASSETS_MODE`       | string                  | `"CDN"`                                     | `"LOCAL"`                                   | **optional**<br> Specifies mode of asset fetching, either through CDN or locally hosted assets. (see [Asset Fetching Configuration](#asset-fetching-configuration))                                                                                                                                                                                                                                                                                                                                                                              |
| `BACKGROUND_COLOR`  | string (Hex color code) | `"#1d3461"`                                 | `"#1d3461"`                                 | **optional**<br> Specifies the background color using a hex color code.                                                                                                                                                                                                                                                                                                                                                                                                                                                                          |
| `CAPTURE_BTN_AFTER` | int                     | `0` (AutoCapture) / `5000` (PaperCapture)   | `0`                                         | **optional**<br> Configures a delay (in milliseconds) after which an instructional dialog and manual capture button appear. Setting to `0` disables the feature. When enabled, users can manually trigger capture at their convenience while auto-capture remains active. The dialog appears once per session.                                                                                                                                                                                                                                   |
| `CONTAINER_ID`      | string                  |                                             | `"AC_mount"`                                | **required**<br> *div id* to mount tool on. If the `div` does not exist it will be created and placed in `<body>`.                                                                                                                                                                                                                                                                                                                                                                                                                               |
| `DEBUG`             | bool                    | `false`                                     | `false`                                     | **optional**<br> When debug is `true` more detailed logs will be visible.                                                                                                                                                                                                                                                                                                                                                                                                                                                                        |
| `DESKTOP_MODE`      | bool                    | `false`                                     | `false`                                     | **optional**<br> Enables all cameras for testing/development purposes. **FOR TESTING ONLY - DO NOT USE IN PRODUCTION.** Desktop cameras are often labeled with `facingMode: 'user'` instead of `'environment'`, which would normally be filtered out. This mode bypasses camera filtering to allow testing on desktop devices, including virtual cameras. Production environments should always use `false` to ensure only back-facing cameras (environment) are available, preventing accidental use of front-facing cameras on mobile devices. |
| `LANGUAGE`          | string                  | `"nl"`                                      | `"nl"`                                      | **required**<br> Notifications in specific language.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                             |
| `onComplete`        | javascript function     |                                             | `function(data) {console.log(data)}`        | **required**<br> Callback function on *complete*.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                |
| `onError`           | javascript function     | `function(error) {console.log(error.code)}` | `function(error) {console.log(error.code)}` | **required**<br> Callback that fires when an error occurs. Receives `{ code, stack }`. See [Error Codes](#error-codes).                                                                                                                                                                                                                                                                                                                                                                                                                          |
| `onUserExit`        | javascript function     | `function(error) {console.log(error)}`      | `function(error) {window.history.back()}`   | **required**<br> Callback function on *user exit*.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                               |
| `ROI_MODE`          | string                  | `"landscape-landscape"`                     | `portrait-landscape`                        | **optional**<br> Frame orientation options: `"portrait-landscape"`, `"landscape-landscape"`                                                                                                                                                                                                                                                                                                                                                                                                                                                      |
| `SDK_MODE`          | string                  | `"autocapture"`                             | `"papercapture"`                            | **optional**<br> Specifies mode of the SDK, supported modes are: `"autocapture"`, `"papercapture"` (see [SDK Modes](#sdk-modes))                                                                                                                                                                                                                                                                                                                                                                                                                 |
| `TOKEN`             | string                  |                                             | see [SDK Token](#sdk-token)                 | **required**<br> Datachecker SDK token.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          |

## SDK Modes

The SDK offers two operational modes:

- **AutoCapture**: Optimized for real-time capture of identity documents such as IDs, passports, and driver licenses. A capture is triggered only when a document is detected and passes all quality checks.
- **PaperCapture**: Tailored for capturing paper documents (e.g., work permits). Like AutoCapture, it ensures a capture occurs only after document detection and successful quality validation.

Refer to [Configuration](#configuration) for details on how to set the desired mode.

### PaperCapture Configuration

```javascript
let AC = new AutoCapture();
AC.init({
    CONTAINER_ID: ...,
    LANGUAGE: ...,
    TOKEN: ...,
    SDK_MODE: "papercapture",
    onComplete: ...,
    onError: ...,
    onUserExit: ...
});
```

## Asset fetching Configuration

AutoCapture requires fetching assets, which can be done either through a CDN or by hosting them locally. Configure this in the tool settings as follows:

### CDN Configuration

```javascript
// configuration
{
    ASSETS_MODE: "CDN",
    // other configurations
}
```

### Locally Hosting Configuration

To host assets locally, first copy them to your desired location:

```bash
cp -r dist/assets/ path/to/hosted/assets/
```

Then, configure the tool to use these local assets:

```javascript
// configuration
{
    ASSETS_MODE: "LOCAL",
    ASSETS_FOLDER: "path/to/hosted/assets/",
    // other configurations
}
```

For comphrehensive integration examples, please refer to our [Integration Examples](examples/README.md).

### Version Control

To ensure compatibility:

- **Separate Asset Versioning**: The assets directory contains a version file, separate from the main file's version.
- **Compatibility Check**: The main file will perform a version check and throw an error if the versions are incompatible.

## Content Security Policy (CSP)

AutoCapture is designed to work with Content Security Policy (CSP) enabled. The SDK requires specific CSP directives to load assets (e.g., scripts, models, images) and execute WebAssembly for document detection.

### Required CSP Directives

Ensure your CSP policy includes the following directives. Adjust domains based on your environment (e.g., use `https://developer.datachecker.nl` for Datachecker production api).

- `default-src 'self';`
- `script-src 'self' https://cdn.jsdelivr.net 'wasm-unsafe-eval' 'unsafe-inline' blob:;`
- `style-src 'self' 'unsafe-inline';`
- `connect-src 'self' https://developer.datachecker.nl https://cdn.jsdelivr.net data:;`
- `img-src 'self' data: blob: https://cdn.jsdelivr.net;`
- `worker-src 'self' blob:;`
- `object-src 'self' blob:;`
- `frame-src 'self' blob:;`
- `base-uri 'none';`

## Handling callbacks

Within the application, you can take advantage of four callback functions to enhance the user experience and manage the flow of your process.

Note: When integrating the application into Native Apps using web views, it's essential to adapt and utilize these callback functions according to the conventions and requirements of the native platforms (e.g., iOS, Android). Native app development environments may have specific ways of handling JavaScript callbacks, and you should ensure seamless communication between the web view and the native code.

Example Web (JS):

```javascript
let AC = new AutoCapture();
AC.init({
    CONTAINER_ID: 'AC_mount',
    LANGUAGE: 'en',
    TOKEN: "<SDK_TOKEN>",
    onComplete: function(data) {
        console.log(data);
    },
    onError: function(error) {
        console.error(error.code, error.stack)
    },
    onUserExit: function(error) {
        console.log(error);
        window.history.back();
    }
});
```

| **ATTRIBUTE** | **FORMAT**          | **DEFAULT VALUE**                           | **EXAMPLE**                                 | **NOTES**                                                                                                               |
| ------------- | ------------------- | ------------------------------------------- | ------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------- |
| `onComplete`  | javascript function |                                             | `function(data) {console.log(data)}`        | **required**<br> Callback that fires when all interactive tasks in the workflow have been completed.                    |
| `onError`     | javascript function | `function(error) {console.log(error.code)}` | `function(error) {console.log(error.code)}` | **required**<br> Callback that fires when an error occurs. Receives `{ code, stack }`. See [Error Codes](#error-codes). |
| `onUserExit`  | javascript function | `function(error) {console.log(error)}`      | `function(error) {window.history.back()}`   | **required**<br> Callback that fires when the user exits the flow without completing it.                                |

### onComplete

This callback function will be called once all the tasks within the workflow succesfully have been completed. This callback function is **required**. The `data` parameter within the function represents the [output](#output) of the completed process. You can customize this function to handle and display the data as needed.

Example Web (JS):

Within the example below we are logging the output (`data`) to console.

```javascript
let AC = new AutoCapture();
AC.init({
    ...,
    onComplete: function(data) {
        console.log(data);
    }
});
```

### onError

This callback fires when an error occurs during the SDK lifecycle. This callback function is **required**. The `error` parameter is an object with two properties:

- **`code`** — A structured error code (e.g., `capture_error:4004`). Use the category prefix to determine the appropriate UI response. See [Error Codes](#error-codes) for the full list of categories.
- **`stack`** — A stack trace string. Include this when reporting issues to support for faster diagnosis.

Example Web (JS):

```javascript
let AC = new AutoCapture();
AC.init({
    ...,
    onError: function(error) {
        console.error(error.code, error.stack);

        if (error.code.startsWith('capture_error')) {
            // Camera issue — show retry UI or prompt for camera permission
        } else if (error.code.startsWith('init_error')) {
            // Initialization failed — prompt user to refresh the page
        } else if (error.code.startsWith('not_allowed')) {
            // Document type not permitted — prompt user to use a different document
        }
    }
});
```

### onUserExit

This callback can be used to implement actions like returning users to the previous page or prompting them for confirmation before exiting to ensure they don't lose any unsaved data or work. This callback function is **required**. The `error` parameter within the function contains information about the specific error encountered, allowing you to log or display error messages for debugging or user guidance. The error that is thrown is `"exit"`.

Example Web (JS):

Within the example below we are logging the output (`error`) to console. Finally, we move back one page in the session history with `window.history.back()`.

```javascript
let AC = new AutoCapture();
AC.init({
    ...,
    onUserExit: function(error) {
        console.log(error);
        window.history.back()
    }
});
```

## Usage/Examples

The tool first needs to be initialised to load all the models.
Once its initialised, it will be started.

```javascript
let AC = new AutoCapture();
AC.init({
    CONTAINER_ID: ...,
    LANGUAGE: ...,
    TOKEN: ...,
    onComplete: ...,
    onError: ...,
    onUserExit: ...
});
```

To stop the camera and empty the container with its contents the `stop` function can be called. This function will automatically be called within `onComplete`, `onError` and `onUserExit` thus do not have to be called within your own custom versions of these functions.

```javascript
AC.stop();
```

If you wish to completely remove the container (identified by `CONTAINER_ID`), use the remove command:

```javascript
AC.remove();
```

Example below:

```javascript
let AC = new AutoCapture();
AC.init({
    CONTAINER_ID: 'AC_mount',
    LANGUAGE: 'nl',
    TOKEN: "<SDK_TOKEN>",
    onComplete: function(data) {
        console.log(data);
    },
    onError: function(error) {
        console.error(error.code, error.stack)
    },
    onUserExit: function(error) {
        console.log(error);
        window.history.back()
    }
});
```

## Importing SDK

Import the SDK with one of the three methods: Script tag, ES6 or CommonJS.

### Script Tag

Easily add AutoCapture to your HTML files using the Script Tag method.

```html
<!-- Add AutoCapture directly in your HTML -->
<script src="dist/autocapture.obf.js"></script>
```

### NPM

For projects using NPM and a module bundler like Webpack or Rollup, you can import AutoCapture as an ES6 module or with CommonJS require syntax.

```js
// Import AutoCapture in your JavaScript file

// ES6 style import
import AutoCapture from '@datachecker/autocapture';

// CommonJS style require
let AutoCapture = require('@datachecker/autocapture')
```

## Demo

```html
<!DOCTYPE html>
<html>
<head>
<title>AutoCapture</title>
</head>

<body>
    <div id="AC_mount"></div>
</body>

<script src="autocapture.obf.js" type="text/javascript"></script>

<script>
    let AC = new AutoCapture();
    AC.init({
        CONTAINER_ID: 'AC_mount',
        LANGUAGE: 'nl',
        TOKEN: "<SDK_TOKEN>",
        onComplete: function (data) {
            console.log(data)
        },
        onError: function(error) {
            console.error(error.code, error.stack);
        },
        onUserExit: function(error) {
            console.log(error);
            window.history.back()
        }
    });
</script>

</html>
```

For comphrehensive integration examples, please refer to our [Integration Examples](examples/README.md).

## Languages

There are two ways in which notifications can be loaded: from file, from object (json).

### File

The languages can be found in `assets/language/`. The current support languages are `en` and `nl`. More languages could be created.

The notifications can be loaded in `configuration` like the following:

```javascript
let AC = new AutoCapture();
AC.init({
    LANGUAGE: 'en',
    ...
```

To create support for a new language, a js file needs to be created with specific keys.
The keys can be derived from the current language js files (`assets/language/en.js`).

Example:

```javascript
var LANGUAGE = {
    approval_prompt: "Is the image right?",
    capture_error: "We were unable to capture an image. Camera access is required.",
    camera_selection: "Please select the back camera",
    confirm: "Accept",
    continue: "Continue",
    corners: "Not all corners detected",
    exp_bright: "Environment is too bright",
    exp_dark: "Environment is too dark",
    flip: "Flip the document",
    flip_backside: "Flip the document to the backside",
    flip_frontside: "Flip the document to the frontside",
    focus: "Hold still...",
    glare: "Glare detected",
    init_error: "Initialization failed. Please refresh the page.",
    manual_mode: "Manual capture mode. Press Capture when ready.",
    model_error: "Failed to load required resources. Please check your connection.",
    not_allowed: "Document type not allowed. Please use a different document.",
    occlusion: "Document is occluded",
    opencv_error: "A required component failed to load. Please refresh the page.",
    settings_error: "Configuration error. Please contact support.",
    size: "Move closer",
    start_prompt: "Tap to start",
    std_msg_0: "Place your document",
    retry: "Try again",
    rotate_phone: "Please rotate your phone upright",
    token_error: "Authorization failed. Please try again later.",
    tutorial: "Follow the instructions"
}
```

### Object (json)

Notifications can also be loaded as a json object like the following:

```javascript
let AC = new AutoCapture();
AC.init({
    LANGUAGE: JSON.stringify(
        {
            approval_prompt: "Is the image right?",
            capture_error: "We were unable to capture an image. Camera access is required.",
            camera_selection: "Please select the back camera",
            confirm: "Accept",
            continue: "Continue",
            corners: "Not all corners detected",
            exp_bright: "Environment is too bright",
            exp_dark: "Environment is too dark",
            flip: "Flip the document",
            flip_backside: "Flip the document to the backside",
            flip_frontside: "Flip the document to the frontside",
            focus: "Hold still...",
            glare: "Glare detected",
            init_error: "Initialization failed. Please refresh the page.",
            manual_mode: "Manual capture mode. Press Capture when ready.",
            model_error: "Failed to load required resources. Please check your connection.",
            not_allowed: "Document type not allowed. Please use a different document.",
            occlusion: "Document is occluded",
            opencv_error: "A required component failed to load. Please refresh the page.",
            settings_error: "Configuration error. Please contact support.",
            size: "Move closer",
            start_prompt: "Tap to start",
            std_msg_0: "Place your document",
            retry: "Try again",
            rotate_phone: "Please rotate your phone upright",
            token_error: "Authorization failed. Please try again later.",
            tutorial: "Follow the instructions"
        }
    ),
    ...
```

## Error Codes

The `onError` callback receives an object `{ code, stack }`. The `code` follows the format `category:NNNN` (e.g., `capture_error:4004`). Use the category prefix to determine the type of error and the appropriate response.

| Category         | Description                               | Recommended Action                                |
| ---------------- | ----------------------------------------- | ------------------------------------------------- |
| `capture_error`  | Camera or processing failure              | Show camera retry UI or prompt for permission     |
| `init_error`     | Initialization failed                     | Prompt user to refresh the page                   |
| `model_error`    | ML model failed to load                   | Check network connection, retry initialization    |
| `not_allowed`    | Document type not permitted               | Prompt user to use an accepted document type      |
| `opencv_error`   | Required component failed to load         | Prompt user to refresh or try a different browser |
| `settings_error` | Invalid configuration or version mismatch | Verify SDK configuration and assets               |
| `token_error`    | Token missing, invalid, or not permitted  | Verify token credentials                          |

When reporting issues to support, include both `error.code` and `error.stack` — the numeric identifier in the code and the stack trace allow for precise diagnosis.

The user-facing alert message shown in the SDK overlay is determined by the category prefix, which maps to a key in the [Languages](#languages) dictionary (e.g., `capture_error` maps to the `capture_error` language key). If a custom language file does not include a key for a new category (e.g., `init_error`), the SDK falls back to its built-in English default.

## Models

The tool uses a collection of neural networks. Make sure that you host the full directory so the models can be accessed. The models path can be configured. (see [Configuration](#configuration))
The models are located under `models/`.

## Allowed documents

The `ALLOWED_DOCUMENTS` setting serves two important purposes:

1. **Document Type Validation**: Restricts which document types can be captured. If a user attempts to capture a document type not listed in `ALLOWED_DOCUMENTS`, the application will display an error message (using the `not_allowed` language key) with a "forbidden" icon, prompting them to use a different document.

2. **Side Permissions**: Specifies which sides (front, back, or both) are allowed for each document type. This controls the flipping behavior and which sides can be captured.

### Configuration Examples

**Allow both sides of documents:**

```javascript
let AC = new AutoCapture();
AC.init({
    CONTAINER_ID: 'AC_mount',
    ALLOWED_DOCUMENTS: {
        IDENTITY_CARD: ['FRONT', 'BACK'],     // Both front and back sides allowed
        PASSPORT: ['FRONT', 'BACK'],          // Both front and back sides allowed
        DUTCH_PASSPORT: undefined,            // Inherits from PASSPORT settings
        RESIDENCE_PERMIT: ['FRONT', 'BACK'],  // Both front and back sides allowed
        DRIVING_LICENSE: ['FRONT', 'BACK'],   // Both front and back sides allowed
    },
    TOKEN: "<SDK_TOKEN>",
    onComplete: function (data) {
        console.log(data)
    },
    onError: function(error) {
        console.error(error.code, error.stack)
    },
    onUserExit: function (error) {
        console.log(error)
        window.history.back()
    }
})
```

**Restrict to only front side:**

```javascript
ALLOWED_DOCUMENTS: {
    IDENTITY_CARD: ['FRONT'],   // Only front side allowed, back side will be rejected
    PASSPORT: ['FRONT'],        // Only front side allowed, back side will be rejected
}
```

**Note**: Any document type not listed in `ALLOWED_DOCUMENTS` will be rejected during capture, triggering the validation error with the "forbidden" icon and displaying the `not_allowed` notification message.

### Handling Unknown Documents

The system may detect an unknown document type when it cannot confidently classify the document. This is represented by the `'UNK'` (unknown) key for both document type and page ID.

**Important**: Unknown documents cannot be added to the `ALLOWED_DOCUMENTS` configuration as `'UNK'` is not a valid document type in the settings validation. When an unknown document is detected, it will automatically trigger the `onError` callback (see [onError](#onerror)) with a `not_allowed` error code. This is the intended behavior to ensure only recognized and valid document types are processed by the SDK.

## Output

The SDK will output in the following structure:

```json
{
    "image": ["...base64_img"],
    "meta": [
        {
            "angle": "...",
            "coordinates": [
                ["...", "..."],
                ["...", "..."],
                ["...", "..."],
                ["...", "..."]
            ],
            "force_capture": "...",
            "device": "..."
        }
    ],
    "token": "sdk_token"
}
```

Example:

```json
{
    "image": ["iVBORw0KGgoAAAANSUhEUgAAAysAAAS..."],
    "meta": [
        {
            "angle": 0,
            "coordinates": [
                [0, 0],
                [0, 100],
                [150, 100],
                [150, 0]
            ],
            "force_capture": false,
            "device": "Back Camera"
        }
    ],
    "token": "sdk_token"
}
```