# Migrations

You will find hereafter all necessary documentation to apply MongoDb migrations
to your project.

<!-- TOC depthFrom:2 -->

- [Purpose](#purpose)
- [Requirements](#requirements)
- [Models](#models)
- [Validation](#validation)
- [Examples](#examples)
  - [Creating a new collection](#creating-a-new-collection)
  - [Creating a new index](#creating-a-new-index)
  - [Removing an existing index](#removing-an-existing-index)
  - [Updating an existing index](#updating-an-existing-index)
  - [Updating a document validator](#updating-a-document-validator)
  - [Removing a collection](#removing-a-collection)
  - [Real-life example](#real-life-example)
- [References](#references)

<!-- /TOC -->

> For data migration, please go the [data-migration.md](./data-migration.md)

## Purpose

Before adding a migration, you have to understand why we need one. MongoDb is
a Document Oriented database and therefore, no constraint is applied to your
schema. For this reason, the source code is in charge to maintain the quality
of documents stored into the database as well as the coherence of the schema
over time to database structure.

A migration in this article is a modification of the database structure and
contract. The structure is given by the names and types of the collection, the
name, the keys and the types of the indexes and, starting with MongoDb 3.2, the
validation schema you want to apply to your documents.

## Requirements

## Models

> [Full documentation](./models.md)

Models are composed of two files: a definition file and a module. The
definition file has the following structure:

```js
/**
 * CONFIGURATION
 */
const DATABASE = 'database';
const COLLECTION = 'collection';
const COLLECTION_OPTIONS = {};
const VALIDATOR_SCHEMA = {};
const VALIDATOR_OPTIONS = {
  validationLevel: 'off'
};

/**
 * @see http://mongodb.github.io/node-mongodb-native/2.2/api/Collection.html#createIndex
 */
const INDEXES = [
  // [{ KEYS }, { OPTIONS }]
  // ie. [{ _id: 1 }, { name: '_id_' }]
];

module.exports = {
  DATABASE,
  COLLECTION,
  COLLECTION_OPTIONS,
  INDEXES,
  VALIDATOR_SCHEMA,
  VALIDATOR_OPTIONS
};

```

The module has the following minimal structure:

```js
const { DATABASE, COLLECTION } = require('./definition');

/**
 * Return the model collection
 *
 * @returns {Object}
 */
function collection(mongodb) {
  return mongodb.db(DATABASE).collection(COLLECTION);
}

module.exports = {
  collection
};

```

## Validation

In order to validate the migrations present into your project, you need to
check that:

- the application of all your migrations lead to the definition given by your
  models.

- any migration script must apply a *single atomic* modification onto the
  database structure.

> One exception to this rule is the `0.*` migration scripts initializing a non
> already production deployed database. In this case, and **only in this case**,
> because no data is present in the collection and no traffic is landing on the
> service, a single script initializing all collections and indexes is
> accepted.

- any migration script must be *rollbackable*.

in order to spare time on writing valid migrations scripts, most of these steps
have been tested automatically with `mocha` by applying the following test
code ([source](../test/migrations/checks-example.test.js)):

```js
describe('MongoDb migrations validation', () => {
  let mongodb;

  beforeEach(async () => {
    // Initialize the MongoDb Connector with models and definitions:
    mongodb = new MongoDbConnector(config, models, migrations);

    await mongodb.connect();
  });

  afterEach(async () => {
    await mongodb.disconnect();
  });

  it('applies migrations to the same result as models init', async () => {
    await mongodb.checkDangerouslyMigrationsAndModelsEquality();
  });

  it('validates that each migration is rollbackable', async () => {
    await mongodb.checkDangerouslyMigrationsRollbackability();
  });
});

```

## Examples

### Creating a new collection

> [source code `create-collection.js`](../src/migrations/create-collection.js)

You can create a new collection by using the following script example:

```js
/**
 * Downgrade the state to previous version
 * @param {object} mongodb
 * @return {Promise<boolean>} Status of the migration
 */
async function rollback(mongodb) {
  await mongodb
    .getMigrationHelper('createCollection')
    .rollback('my-database', 'my-collection');

  return true;
}

/**
 * Upgrade the state to version x.x.x
 * @param {object} mongodb
 * @return {Promise<boolean>} Status of the migration
 */
async function update(mongodb) {
  await mongodb
    .getMigrationHelper('createCollection')
    .update('my-database', 'my-collection');

  return true;
}

module.exports = { rollback, update };

```

### Creating a new index

> [source code `create-index.js`](../src/migrations/create-index.js)

You can create an index onto a collection by using the following script example:

```js
/**
 * Downgrade the state to previous version
 * @param {object} mongodb
 * @return {Promise<boolean>} Status of the migration
 */
async function rollback(mongodb) {
  await mongodb
    .getMigrationHelper('createIndex')
    .rollback('my-database', 'my-collection', 'my-index', { keys: 1 }, { options: 1 });

  return true;
}

/**
 * Upgrade the state to version x.x.x
 * @param {object} mongodb
 * @return {Promise<boolean>} Status of the migration
 */
async function update(mongodb) {
  await mongodb
    .getMigrationHelper('createIndex')
    .update('my-database', 'my-collection', 'my-index', { keys: 1 }, { options: 1 });

  return true;
}

module.exports = { rollback, update };

```

### Removing an existing index

> [source code `drop-index.js`](../src/migrations/drop-index.js)

You can drop an index from a collection by using the following script example:

```js
/**
 * Downgrade the state to previous version
 * @param {object} mongodb
 * @return {Promise<boolean>} Status of the migration
 */
async function rollback(mongodb) {
  await mongodb
    .getMigrationHelper('dropIndex')
    .rollback('my-database', 'my-collection', 'my-index', { key: 1 }, {});

  return true;
}

/**
 * Upgrade the state to version x.x.x
 * @param {object} mongodb
 * @return {Promise<boolean>} Status of the migration
 */
async function update(mongodb) {
  await mongodb
    .getMigrationHelper('dropIndex')
    .update('my-database', 'my-collection', 'my-index', { key: 1 }, {});

  return true;
}

module.exports = { rollback, update };

```

### Updating an existing index

The update of an existing index must be splitted into 2 separated migrations:
a `drop` then a `create`.

### Updating a document validator

> [source code `update-validator.js`](../src/migrations/update-validator.js)

You can create an document validator onto a collection by using the following script example:

```js
/**
 * Downgrade the state to previous version
 * @param {object} mongodb
 * @return {Promise<boolean>} Status of the migration
 */
async function rollback(mongodb) {
  await mongodb
    .getMigrationHelper('updateValidator')
    .rollback('my-database', 'my-collection');

  return true;
}

/**
 * Upgrade the state to version x.x.x
 * @param {object} mongodb
 * @return {Promise<boolean>} Status of the migration
 */
async function update(mongodb) {
  await mongodb
    .getMigrationHelper('updateValidator')
    .update('my-database', 'my-collection', {
      key: 'string'
    }, {
      validationLevel: 'strict'
    })

  return true;
}

module.exports = { rollback, update };

```

### Removing a collection

> [source code `drop-collection.js`](../src/migrations/drop-collection.js)

You can remove a collection safely by using the following script example:

```js
/**
 * Downgrade the state to previous version
 * @param {object} mongodb
 * @return {Promise<boolean>} Status of the migration
 */
async function rollback(mongodb) {
  await mongodb
    .getMigrationHelper('dropCollection')
    .rollback('my-database', 'my-collection');

  return true;
}

/**
 * Upgrade the state to version x.x.x
 * @param {object} mongodb
 * @return {Promise<boolean>} Status of the migration
 */
async function update(mongodb) {
  await mongodb
    .getMigrationHelper('dropCollection')
    .update('my-database', 'my-collection');

  return true;
}

module.exports = { rollback, update };

```

### Real-life example

- [rider-app-state](https://github.com/transcovo/rider-app-state/tree/master/src/migrations)
- [gonzales](https://github.com/transcovo/gonzales)
- [kenny](https://github.com/transcovo/kenny)
- [pegasus](https://github.com/transcovo/pegasus/tree/master/server/migrations)

## References

- [Blog post about the MongoDb migration pattern](https://blog.staging.chpr.fr/post/2017-11-16-database-migration/)
- [Slides about the MongoDb migration process](https://docs.google.com/presentation/d/18M_g_kaHeWENwiO3TsV3LSqdIA2WQeazWjO4gY1B86I/edit#slide=id.p)