# i18n-services

## Introduction

`@dialpad/i18n-services` contains all the tooling needed to manage the resources
for our i18n packages, i.e.
[`@dialpad/i18n-vue2` for legacy apps](../i18n-vue2/README.md) and
[`@dialpad/i18n` for modern apps](../i18n/README.md). this includes:

- Uploading new strings to a translation service.
- Downloading translated strings from a translation service.
- Loading bundled resources at runtime for a given locale.
- Generating a "Dialpadistan" locale from `en-US`.

## How it works?

There are essentially two main concepts behind the scenes:

### BaseLocaleManager

The BaseLocaleManager having the common logic that oversees internationalization
(i18n) in a Vue2.js and a Vue3.js application using the
[Fluent localization framework](https://projectfluent.org/). Its
responsibilities include:

- **Loading and caching translation bundles:** Manages the `.ftl` data, which
  maps keys to text for each locale.
- **Switching the current locale:** Provides methods to change the active
  language dynamically.
- **Handling bundle namespaces:** Manages potential duplicate bundle requests
  from multiple sources, though this feature is still being refined.
- **Initializing the Vue plugin:** Exposes translation methods (`$t`, `$ta`) to
  `Vue` templates, enabling localized content within the application.

### BundleSource

The second key concept is the BundleSource interface, which abstracts the
loading and management of localization resources. This interface has two primary
implementations: `RawBundleSource` and `HTTPBundleSource`, each designed for
different use cases.

- `RawBundleSource`: This implementation handles localization resources defined
  within the application. It organizes these resources defined directly in the
  application, and imported directly at runtime (for example, using
  `import(./[locale].ftl`). This approach is ideal for applications with mostly
  static content.

- `HTTPBundleSource`: This implementation fetches localization resources
  dynamically from a server. It caches these resources locally, allowing for
  efficient access while still being flexible enough to load translations on
  demand.

**Disclaimer:** _Both implementations are provisional and may change as we
better understand our needs. Currently, `RawBundleSource` is the preferred
option for most applications._

#### Synchronous Resource Loading with [builtResources](cci:1://file:///Users/daniellovero/src/web-clients/packages/i18n-services/bundle/bundle-source/raw.js:28:2-34:3)

While the
[dynamicResources](cci:1://file:///Users/daniellovero/src/web-clients/packages/i18n-services/bundle/bundle-source/raw.js:2:2-17:3)
method is used for asynchronously loading resources, the
[RawBundleSource](cci:2://file:///Users/daniellovero/src/web-clients/packages/i18n-services/bundle/bundle-source/raw.d.ts:14:0-30:1)
class also provides a
[builtResources](cci:1://file:///Users/daniellovero/src/web-clients/packages/i18n-services/bundle/bundle-source/raw.js:28:2-34:3)
method for synchronous resource loading. This is useful when you have the
localization resources available at compile time and prefer to import them
directly without dealing with promises.

##### How to use [builtResources](cci:1://file:///Users/daniellovero/src/web-clients/packages/i18n-services/bundle/bundle-source/raw.js:28:2-34:3)

Instead of dynamically importing `.ftl` files, you can import them directly and
use the
[builtResources](cci:1://file:///Users/daniellovero/src/web-clients/packages/i18n-services/bundle/bundle-source/raw.js:28:2-34:3)
method to process the resources. Here's an example:

```typescript
import { RawBundleSource } from '@dialpad/i18n-services';
// Import your .ftl files directly
import enUSResource from './locales/en-US.ftl';
import esResource from './locales/es.ftl';

// Create an array of BuiltResource objects
const builtResourcesArray: BuiltResource[] = [
  ['en-US', 'namespace', enUSResource],
  ['es', 'namespace', esResource],
  // ... add other locales and namespaces as needed
];

// Convert BuiltResource objects to Resource objects using builtResources
const resources: Resource[] =
  RawBundleSource.builtResources(builtResourcesArray);

// Now, resources can be used to instantiate RawBundleSource or for any other purpose
const bundleSource = new RawBundleSource({ resources });
```

**If you're looking for the 'Dialpadistan' translator, please
[skip to this section](#dialpadistan-translator).**

## Setup

### Installation

```sh
pnpm add @dialpad/i18n-services
```

### Configuration

You can find the needed configuration for using `$t` and `$ta` functions in your
application in the [i18n for vue2](../i18n-vue2/README.md) and
[i18n for vue3](../i18n/README.md) documentation. In both cases the needed
configuration should be the same.

## Usage

### Adding Resources at Runtime with `addSources`

#### API Method: `addSources`

The `addSources` method allows you to add multiple translation resources to the
`LocaleManager` at runtime. Once all resources are loaded, it triggers the
`ready` promise indicating that the locale manager is fully initialized and
ready to handle localization requests.

##### Parameters

- `sources`: An array of `BuiltResource` items where each item contains:
  - `locale`: The locale code for the translation resource.
  - `namespace`: The namespace associated with the resource.
  - `source`: The actual source string or a promise that resolves to the source
    string.

##### Example

```typescript
// Assuming you have a LocaleManager instance
const manager = new LocaleManager(...);

// Define your resources
const resourcesToAdd: BuiltResource[] = [
  // Array of [locale, namespace, source] tuples
  ['en-US', 'namespace1', sourceStringOrPromise],
  ['es-ES', 'namespace1', anotherSourceStringOrPromise],
  // Add more resources as needed
];

// Add the resources to the LocaleManager
manager.addSources(resourcesToAdd);
```

### I18n interface

The API for our i18n tools is the same either for the vue3 or vue2 supported
versions. You can find the more details about how to use the tool in the
[i18n for vue2](../i18n-vue2/README.md) and [i18n for vue3](../i18n/README.md)
documentation.

### Dialpadistan translator

This feature uses the `@fluent/syntax` parser and a set of regex and grammar
rules to convert an `en-US.ftl` file into an `dp-DP.ftl` inside the same
directory. The "dialpadistani" language is just the original language
transformed character-per-character to non-latin characters which is useful to
test the proper render of different fonts.

For example:

```ftl
# src/localization/en-US.ftl

MESSAGE_INPUT_LABEL = New message input
FEED_NEW_SEPARATOR = Beginning of unread items
```

Will translate to:

```ftl
# src/localization/dp-DP.ftl

MESSAGE_INPUT_LABEL = Ŋάŵ ɱάššëğά īŋþø†
FEED_NEW_SEPARATOR = Sάğīŋŋīŋğ ůƒ øŋřάëḍ ī†άɱš
```

To execute a Dialpadistan translation you should run the following script
pointing to your Fluent English file: _(Note: it can either be a relative path
or an absolute path)_

```sh
pnpm pnpm --filter=@dialpad/i18n-services run translate:dialpadistan /your/english/fluent/file/en-US.ftl
```

When succeeding translating, you will notice a message on your terminal stating
the folder where the output file will be placed. This folder is _the same as
your English Fluent file_.

e.g. in this case you will see
`File transformed and saved as /your/english/fluent/file/dp-DP.ftl`.

#### Commit hook

This translator includes a post-commit that automatically translates english
files to dialpadistani. This hook checks the commit for changed `en-US.ftl`
files and will only run the translator against those files that had changes in
the commit.

If this script finally makes any change to `dp-DP.ftl` files, it will
automatically commit those changes with the message:
`chore(auto-generated): translate en-US.ftl files to dialpadistani`

#### Known issues

There is one specific scenario where this translator is failing, and we don't
consider it to be harmful. This is because of a bug on `@fluent/syntax` parser.

And this is when having a comment right in the middle between an equal sign and
a value, and that comment has at least one placeholder, whatever's after the
text of the first placeholder will be translated (and if the comment it's
multi-lined every comment line will be squashed and inlined after that
placeholder as well)

e.g.

```ftl
MY_KEY =
    # My awesome comment
    # And here the {$placeholder} is present
        .aria-label = { $name } ({ $muted ->
        [true] Muted{" "}
        *[other] {""}
    })
```

will be translated to:

```ftl
MY_KEY =
    # My awesome comment
    # And here the {$placeholder} os sintirp
        .aria-label = { $name } ({ $muted ->
        [true] Muted{" "}
        *[other] {""}
    })
```

This is because the way`@fluent/syntax` parser works at the current version. In
order to fix this it would require an amount of effort that is probably not
worth it. It will imply either to patch the parser behavior somehow or extra
usage of regex pre and post processing files, which will not only increase
complexity of the solution but also reduce the robustness of it. And given the
fact that is just a comment, we prefer not to solve it in-house and wait until
they fix or improve this behavior on upcoming versions.

**But keep in mind that whenever adding comments, it should be before the whole
KEY = VALUE and not in between.**

## Context screenshots

Context screenshots are, as their name suggest, screenshots that provide context
to recognize where each translation is used in each app.

Each screenshot should contain visual information of where the translation can
be found in the app with enough context to be easily located. If other dev looks
at the screenshot they should be able to locate it fairly easily in your app.

There should be one screenshot per each translation key on each `.ftl` file. You
can reuse the same screenshot for multiple keys, but each file will correspond
to one key, so you must make one copy for each translation key. The screenshot
filename will be `{translationKeyHere}.png`. We only support `.png` images for
now.

There is a PR check which will fail if any translation key is missing its
corresponding screenshot. There is an exception list for paths that should be
ignored by this check. Avoid using it unless it's completely necessary. If you
strictly need to use it, remember to mention in your commit message why you are
using it.

These screenshots will be located in a directory called `context-screenshots`
sibling to each `.ftl` file in your project. If you have an `.ftl` file, it
should have a sibling screenshots directory. Each screenshot has a file size
limit of 20MB.

The screenshots are uploaded to smartling along with the CSV/FTL files through a
github action when a PR to main is merged.

## Resources

- [Fluent docs](https://github.com/projectfluent/fluent/wiki)
- [I18n best practices](https://localization.blog/2022/05/16/i18n-best-practices-keep-it-together/)
- [i18n for vue2 docs](../i18n-vue2/README.md)
- [i18n for vue3 docs](../i18n/README.md)

## Translation Workflows

The following GitHub workflows handle various aspects of the translation
process:

### 1. Sync Translations Workflow

**Name**: Pull translations from third party translation service

**Purpose**: This workflow synchronizes translations by pulling them from a
third-party translation service (Smartling) and automatically creates a pull
request if there are any changes.

**Triggers**:

- Push to the `main` branch
- Daily scheduled run at midnight (cron: `0 0 * * *`)
- Manual trigger (workflow_dispatch)

**Configuration**:

- Uses the `pull-translations.yml` workflow from `dialpad/wcf-sdk@v0.6.0`
- Smartling project ID is retrieved from repository variables
- Working directory is set to "./packages/i18n"
- Node version is determined from the ".nvmrc" file
- Requires Smartling API credentials stored in GitHub secrets

### 2. Force Pull Translations Workflow

**Name**: Manual Force Pull Translations

**Purpose**: This workflow allows for manual forced pulls of translations from
the translation service, with configurable options.

**Triggers**:

- Manual trigger only (workflow_dispatch)

**Input Parameters**:

- `USE_FTL_FLOW`: Option to use FTL flow (boolean, default: true)
- `SKIP_UNAVAILABLE_LOCALES`: Option to skip unavailable locales without failing
  (boolean, default: true)
- `WORKING_DIRECTORY`: Specifies the working directory (string, default: ".")
- `NODE_VERSION_FILE`: Path to the Node version file (string, optional)

**Configuration**:

- Uses the `force-pull-translations.yml` workflow from `dialpad/wcf-sdk@v0.6.0`
- Smartling project ID is retrieved from repository variables
- Requires Smartling API credentials stored in GitHub secrets

### 3. Translation Screenshots Check Workflow

**Name**: Check translations screenshots

**Purpose**: This workflow verifies if the screenshots for translations are
up-to-date with their respective FTL files.

**Triggers**:

- Pull requests targeting the `main` branch

**Configuration**:

- Uses the `screenshots-translations.yml` workflow from `dialpad/wcf-sdk@v0.6.0`
- Smartling project ID is retrieved from repository variables
- Working directory is set to "./packages/i18n"
- Node version is determined from the ".nvmrc" file
- Requires Smartling API credentials stored in GitHub secrets

### 4. Upload to Translation Service Workflow

**Name**: Push en-US.ftl to third party translation service

**Purpose**: This workflow pushes the English (en-US.ftl) source files to the
third-party translation service (Smartling) for translation.

**Triggers**:

- Push to the `main` branch

**Configuration**:

- Uses the `push-translations.yml` workflow from `dialpad/wcf-sdk@v0.6.0`
- Smartling project ID is retrieved from repository variables
- Working directory is set to "./packages/i18n"
- Node version is determined from the ".nvmrc" file
- Requires Smartling API credentials stored in GitHub secrets

All workflows use Smartling as the translation service and require proper API
credentials (`SMARTLING_API_USER` and `SMARTLING_API_SECRET`) stored as GitHub
secrets.
