---
title: "Pipeline & Plugins"
---

# Pipeline & Plugins

{% subtitle %}

Understanding the main concepts of Limedocs

{% endsubtitle %}

## The Limedocs Pipeline

Limedocs *sources* virtual-files, *transform* them, and finally output them. This whole process is called the *Pipeline*. The Pipeline consists of plugins of various types, among which:

- [Source plugins](#source-plugins)
- [Transformation plugins](#transformation-plugins)
- [Output plugins](#output-plugins)
- [Search plugins](#search-plugins)
- [Other plugins](#other-plugins)

## Virtual files

Limesdocs manipulates **virtual files**, meaning that these files do not necessarily exist on a filesystem. This is a great abstraction which allows Limedocs to source, transform, and output practically anything, everywhere. Once created, virtual files pass through the whole pipeline and can be modified without touching the filesystem.

A virtual file is defined by:

- its `path` (even if virtual)
- its `Content-type`
- its `contents` 
- and some meta data...

Learn more by reading the [VFile API](api/vfile.md).

## Plugins

### Using plugins

Plugins must be referenced in `limedocs.config.js` under:

- `sources` for source plugins
- `transformers` for transform plugins
- `outputs` for output plugins
- `search` for search plugins

You can either reference a plugin by **its path** or **npm package name**.

{% alert %}

When using a *npm package*, the package must be installed in your site directory (using `npm install`)
or globally (`npm install -g`).

{% endalert %}

#### Examples

```js title="Using filename and options"
module.exports = {
  // ...
  sources: [
    {
      // Plugin path
      resolve: "path/to/simple-sql-source-plugin.js",
      // Plugin options (optional)
      options: {
        table: "my_table"
      }
    }
  ]
  // ...
}
```

```js title="Using a npm package and options"
module.exports = {
  // ...
  sources: [
    {
      // NPM package name
      resolve: "my-npm-package",
      // Plugin options (optional)
      options: {
        table: "my_table"
      }
    }
  ]
  // ...
}
```

```js title="Simplified version without options"
module.exports = {
  // ...
  sources: [
    // Without options, you can simply specify NPM package or filename in a string
    "my-npm-package",
  ]
  // ...
}
```

***

### Source plugins

These plugins allows you to pull your **website sources**. They are executed at the beginning of the pipeline.

Usually you will use `@limedocs/source-filesystem` plugin to generate a website from markdown files, but you may 
also want to develop your own *source plugins* to pull data from various sources.

#### What source plugins can do

- Pull sources from your filesystem or from an object storage services like AWS S3 ou Google Cloud Storage.
- Pull sources from a database.
- Pull sources from HTTP requests or APIs.
- Pull sources from `Promises` results.

{% alert %}

Of course, multiple sources plugins can be used at the same time!

{% endalert %}

Check out the [Source Plugin API](api/source-plugins.md) to learn how easily develop such plugins.

***

### Transformation plugins

The job of transformation plugins is to convert virtual files from/to various content types.

#### Usage examples

There is no limit to what you can transform! Here are some usage examples:
- Converting markdown to HTML
- Converting HTML to PDF
- Minify javascript
- Compile less/sass/etc
- Convert lines of sales from a CSV file to PDF invoices
- etc...

#### Multiple Transformations

A virtual-file passes from a *transformation-plugin* to another one depending on its *content type*.

For example, a text file could be first processed by a *text-to-markdown* plugin, then by a *markdown-to-html* plugin, and finaly by a *html-to-pdf* plugin.

Please note that a virtual-file can be processed by multiple transform-plugins during the pipeline lifecycle, but only once per plugin.

Check out the [Transformation Plugin API](api/transformation-plugins.md) to learn how easily develop such plugins.

***

### Output plugins

After having your sources transformed, Limedocs will output your site somewhere, usually to your disk. This is done thanks to *output plugins* like `@limedocs/output-filesystem`.

#### Usage examples

- Output files to your disk (using the plugin `@limedocs/output-filesystem`)
- Output files to some object sorage service like AWS S3 or Google storage
- Even call some REST APIs or a GraphQL server
- etc...

Check out the [Output Plugin API](api/output-plugins.md) to learn how easily develop such plugins.

***

### Search plugins

Search plugins can add search capabilities to your website. Usually you will use only one search plugin, but you may add more if needed. Limedocs supports the `@limedocs/search-lunr` plugin
out of the box which is suitable for small websites and does not require a backend.

Check out the [Search Plugin API](api/search-plugins.md) to learn how easily develop such plugins.

***

### Other plugins

Other non-specific plugins can be used by Limedocs, for example to generate sitemaps or manifests,
or do anything you may want to do.