# Strapi plugin strapi-plugin-elasticsearch

A plugin to enable integrating Elasticsearch with Strapi CMS.

## Installation

via npm:

```
npm i @geeky-biz/strapi-plugin-elasticsearch
```

via yarn:

```
yarn add @geeky-biz/strapi-plugin-elasticsearch
```

## Supported Strapi version

The latest version of this plugin works for Strapi v5. For Strapi v4, please install the version 0.0.8 of this plugin.

## Supported ElasticSearch version
The plugin is expected to work with ElasticSearch v8. It may or may not function as expected with ElasticSearch v7 or v9. Please share your findings if you have tried the plugin with these ElasticSearch versions.

## Plugin Configuration

Within your Strapi project's `config/plugin.js`, enable the plugin and provide the configuration details:

```
module.exports = {
    // ...
    'elasticsearch': {
      enabled: true,
      config: {
        indexingCronSchedule: "<cron schedule>",
        searchConnector: {
          host: "<hostname for Elasticsearch>",
          username: "<username for Elasticsearch>",
          password: "<password for Elasticsearch>",
          certificate: "<path to the certificate required to connect to Elasticsearch>"
        },
        indexAliasName: "<Alias name for the Elasticsearch index>"
      }  
    },
    // ...
  }
```

Example plugin configuration (with adequate `.env` variables set up):
```
module.exports = {
    // ...
    'elasticsearch': {
      enabled: true,
      config: {
        indexingCronSchedule: "00 23 * * *", //run daily at 11:00 PM
        searchConnector: {
          host: process.env.ELASTIC_HOST,
          username: process.env.ELASTIC_USERNAME,
          password: process.env.ELASTIC_PASSWORD,
          certificate: path.join(__dirname, process.env.ELASTIC_CERT_NAME)
        },
        indexAliasName: process.env.ELASTIC_ALIAS_NAME
      }  
    },
    // ...
  }
```
## Ensuring connection to Elasticsearch
When connected to Elasticsearch, the `Connected` field within the `Setup Information` screen shall display `true`.

![image](https://github.com/geeky-biz/strapi-plugin-elasticsearch/assets/17068206/a0fe0d6c-95e9-4c3c-95e1-46209db113c7)

## Configuring collections & attributes to be indexed
The `Configure Collections` view displays the collections and the fields setup to be indexed.

![image](https://github.com/geeky-biz/strapi-plugin-elasticsearch/assets/17068206/13ad3a24-02a6-4c86-8da2-e015ba9c18ea)

From this view, individual collection can be selected to modify configuration:

![image](https://github.com/geeky-biz/strapi-plugin-elasticsearch/assets/17068206/bdc1d674-a74f-4534-9b48-ad0e0eebaeea)

## Configuring indexing for dynamic zone or component attributes
To enable indexing content for attributes of type `component` or `dynamiczone`, additional information needs to be provided via JSON in the following format:

```
{
  "subfields": [
        {
          "component": "<component name within schema.json>",
          "field": "<field name from within that component>"
        },
        {...},
        {...}
      ]
}
```
### Example 1:
If we have an attribute called `seo_details` of type `component` like the following within our collection `schema.json`:
```
    "seo_details": {
      "type": "component",
      "repeatable": false,
      "component": "metainfo.seo"
    },
```
And, if we seek to index the contents of the `meta_description` field belonging to the component `seo`, our `subfields` configuration should be:
```
{
  "subfields": [
        {
          "component": "metainfo.seo",
          "field": "meta_description"
        }
      ]
}
```
![image](https://github.com/geeky-biz/strapi-plugin-elasticsearch/assets/17068206/df1f7dba-2aa1-410e-a567-1de73156a020)

### Example 2:
If we have an attribute called `sections` of type `dynamiczone` within our collection `schema.json`:
```
    "sections": {
      "type": "dynamiczone",
      "components": [
        "content.footer",
        "content.paragraph",
        "content.separator",
        "content.heading"
      ]
    },
...
```
And, if we seek to index the contents of the fields `title` for `content.heading` and `details` as well as `subtext` for `content.paragraph`, our `subfields` configuration should be:
```
{
  "subfields": [
        {
          "component": "content.paragraph",
          "field": "details"
        },
        {
          "component": "content.paragraph",
          "field": "subtext"
        },
        {
          "component": "content.heading",
          "field": "title"
        }
      ]
}
```
The subfields JSON also supports multiple level of nesting:
```
{
  "subfields": [
        {
          "component": "content.footer",
          "field": "footer_link",
          "subfields": [
            {
              "component": "content.link",
              "field": "display_text"
            }
          ]
        }
      ]
}
```
Note: Indexing of `relations` attributes isn't yet supported.

## Indexing of metadata fields
By default, the plugin excludes indexing the following metadata fields:
```
['createdAt', 'createdBy', 'publishedAt', 'publishedBy', 'updatedAt', 'updatedBy']
```
This means, these metadata fields do not appear in the `Configure Collections` view to set them up for indexing. To allow indexing these fields, `allowIndexingMetadataFields : true` needs to be specified in the plugin configuration.

```
module.exports = {
    // ...
    'elasticsearch': {
      enabled: true,
      config: {
          // ...
          allowIndexingMetadataFields: true,
      }  
    },
    // ...
  }
```

With the above plugin configuration, the metadata fields become available to be selected for indexing.

## Transforming data before indexing
To modify the data stored within Strapi before it is submitted by the plugin for indexing, transformer functions can be setup.

1. Define all the transformer functions within `src/extensions/elasticsearch/strapi-server.ts` as exhibited in the following example (it is vital for the transformer function to return the transformed data):

```JavaScript
export default (plugin) => {
    plugin.config.transformers = {
        capitalize: (value: string) => {
            return value.charAt(0).toUpperCase() + value.slice(1);
        },
        renderMarkdown: (value: string) => {
            return markdownToTxt(value);
        },
        roundFloatValue: (value: number) => {
            return Number(value.toFixed(2));
        }
    };
    return plugin;
}
```
2. Once defined, a drop-down to select the transformer function shall be available with the list of function names (as in the screenshot below). Select the transformer function for specific fields.
![image](https://github.com/user-attachments/assets/fa21c228-1d2f-48cc-b6d3-c921bf6a0b5e)

3. With the above setup in place:
- Your transformer function will execute every time before submitting the data for indexing.
- The data returned from the transformer function will be submitted for indexing.

## Exporting and Importing indexing configuration
To enable backing up the indexing configuration or transferring it between various environments, these can be Exported / Imported from the `Configure Collections` view.

![image](https://github.com/geeky-biz/strapi-plugin-elasticsearch/assets/17068206/6e099392-499e-4101-8f51-85b7eff8aa38)

## Scheduling Indexing
Once the collection attributes are configured for indexing, any changes to the respective collections & attributes is marked for indexing. The cron job (configured via `indexingCronSchedule`) makes actual indexing requests to the connected Elasticsearch instance. 

- `Trigger Indexing` triggers the cron job immediately to perform the pending indexing tasks without waiting for the next scheduled run.
- `Rebuild Indexing` completely rebuilds the index. It may be used if the Elasticsearch appears to be out of sync from the data within Strapi.

![image](https://github.com/geeky-biz/strapi-plugin-elasticsearch/assets/17068206/71df02a9-8513-4a91-8e23-2b5f34495c20)

Whenever a collection is configured for indexing, it may already have data that needs to be indexed. To facilitate indexing of the past data, a collection can be scheduled for indexing in the next cron run from the `Configure Collections` view:

![image](https://github.com/geeky-biz/strapi-plugin-elasticsearch/assets/17068206/7f37453a-dc87-406a-8de0-0391018b7fb5)

### Triggering Indexing Externally
To trigger indexing from an external event:
- From `Users & Permissions Plugin` -> `Roles` -> Select a Role -> `ElasticSearch`, select `triggerIndexing`
- Now, for that role, invoking the route `/api/elasticsearch/trigger-indexing/` shall trigger indexing to index all the data that is pending to be indexed (same behavior as that when clicking the `Trigger Indexing` button from the Strapi Admin UI Elasticsearch Plugin screen).

## Searching
You may directly use the Elasticsearch search API or you may use the Search API exposed by the plugin (at `/api/elasticsearch/search`). The plugin search API is just a wrapper around the Elasticsearch search API that passes the query parameter to the Elasticsearch search API and returns the results coming from Elasticsearch:

For example, the below API call would result into the Elasticsearch search API being triggered with the query 
```
`/api/elasticsearch/search?query=query%5Bbool%5D%5Bshould%5D%5B0%5D%5Bmatch%5D%5Bcity%5D=atlanta`
```
would result into the Elasticsearch search API being triggered with query
```
query[bool][should][0][match][city]=atlanta
```
The plugin's API would return the response from the Elasticsearch search API.

Note: To use the `search` API (at `/api/elasticsearch/search`), you will have to provide access via `Settings` -> `Users & Permissions Plugin` -> `Roles` -> (Select adequate role) -> `Elasticsearch` -> `search`.

### Extending Search API
The recommended was to enhance the Search API is to write your own route and controller. Below is an example of how this can be achieved (this example adds pagination capability to the search API):

- Within your setup, create `src/extensions/elasticsearch/strapi-server.js` with the following contents:

```
const { Client } = require('@elastic/elasticsearch')
const qs = require('qs');

let client = null;

module.exports = (plugin) => {

    client = new Client({
        node: plugin.config.searchConnector.host,
        auth: {
          username: plugin.config.searchConnector.username,
          password: plugin.config.searchConnector.password
        },
        tls: {
          ca: plugin.config.searchConnector.certificate,
          rejectUnauthorized: false
        }
    });

    plugin.controllers['performSearch'].enhancedSearch = async (ctx) => {
            try
            {
                const params = qs.parse(ctx.request.query)
                const query = params.search;
                const pagesize = params.pagesize;
                const from = params.from;
                const result= await client.search({
                    index: plugin.config.indexAliasName,
                    query: { "bool" : { "should" : [ { "match": { "content": "dummy"} } ] } },
                    size: pagesize,
                    from: from 
                });
                return result;
            }
            catch(err)
            {
                console.log('Search : elasticClient.enhancedSearch : Error encountered while making a search request to ElasticSearch.')
                throw err;
            }
    }

    plugin.routes['search'].routes.push({
        method: 'GET',
        path: '/enhanced-search',
        handler: 'performSearch.enhancedSearch',
      });
  
    
    return plugin;
};

```

- This will create a new route `/api/elasticsearch/enhanced-search` being served by the function defined above.
- You can add / modify the routes and controllers as necessary.

## Bugs
For any bugs, please create an issue [here](https://github.com/geeky-biz/strapi-plugin-elasticsearch/issues).

## About
- This plugin is created by [Punit Sethi](https://punits.dev).
- I'm an independent developer working on Strapi migrations, customizations, configuration projects (see [here](https://punits.dev/strapi-customizations/)).
- For any Strapi implementation requirement, write to me at `punit@tezify.com`.