# schema-mapper-spec

[![build status](https://travis-ci.org/schema-mapper/spec.svg)](https://travis-ci.org/schema-mapper/spec)
[![Dependencies](https://david-dm.org/schema-mapper/spec.svg)](https://david-dm.org/schema-mapper/spec)
[![License](http://img.shields.io/badge/license-MIT-blue.svg?style=flat)](LICENSE)

Schema mapper defines a specification for data and is specifically designed to make storing data in different data stores less painful
(especially for the ones that need a schema to be more useful).

It provides:
- A definition for a [Project](#project) containing a name, a version and schemas.
- A definition for a [Schema](#schema) that is easy to write by humans but also easy to parse for computers.
- A definitions for [Changes](#changes) that describe modifications made to a project.
- A definition for [DataInfo](#datainfo) which describes a transport format for data and includes information about the project and schema.

A **Project** groups a bunch of schemas and versions them.

A **Schema** describes the type of the data and the columns it has.

**Changes** represent individual modifications to the schema.
These changes can be used to transform a [DataInfo](#datainfo) object from a old version to a newer one
and because we can easily invert changes because they contain the previous value,
you can transform new data to data that is compatible with older versions of the schema as well.

A **DataInfo** object describes data and includes schema, project and version information.

# Using the specifications

There are a couple of ways to use this specification.

- Validate your schemas, data and types by using the [JSON-schemas](https://github.com/schema-mapper/spec/tree/master/json-schemas) for the different types using tools like:
  - [tv4](https://www.npmjs.com/package/tv4)
  - [is-my-json-valid](https://www.npmjs.com/package/is-my-json-valid)
  - JSON-schema validators are implemented in other languages as well
- Validate your schemas, data and types by using the validator provider by this package.
  - [Node API](#node-api)
  - [CLI](#cli)
- Automatically get changes by comparing 2 [Project](#project)s using [schema-mapper-differ](https://www.npmjs.com/package/schema-mapper-differ)
- Transform [DataInfo](#datainfo) objects by applying changes by using [schema-mapper-tranformer](https://www.npmjs.com/package/schema-mapper-transformer)

# Validation

```shell
npm install --save schema-mapper-validator
```

Usage:
```js
var validator = require('schema-mapper-validator');

var dataInfoValidationResult = validator.validateDataInfo(dataInfo);
var schemaValidationResult = validator.validateProject(project);
var schemaValidationResult = validator.validateSchema(schema);
var changesValidationResult = validator.validateChanges(changes);
console.log(dataInfoValidationResult, schemaValidationResult, changesValidationResult);
```

## Types

Below is a specification of the types

### ProjectCollection

A ProjectCollection groups projects together.
It is an object where the key is the id of the project and the value is a [Project](#project).

### Project

The project is an object that contains:
- A name property, that will for example be used for determining the name of a database.
- A version property, that will for example be used for determining the name of a database.
- A schemas property that contains the schemas of the project.

<table>
  <tr><th>Name</th><th>Type</th><th>Description</th></tr>
  <tr><td>name</td><td>String</td><td>The name of the project</td></tr>
  <tr><td>version</td><td>Number</td><td>The version of the project</td></tr>
  <tr><td>schemas</td><td>Object</td><td>An object of schemas where the key is the unique id for the schema and the value is an object of type <a href="#schema">Schema</a></td></tr>
</table>

### SchemaCollection

A SchemaCollection groups schemas together.
It is an object where the key is the id of the schema and the value is a [Schema](#schema).

### Schema

The schema is an object that contains:
- A name property, that will for example be used for determining the name of a database table.
- A columns property that describe the properties of the data.

<table>
  <tr><th>Name</th><th>Type</th><th>Description</th></tr>
  <tr><td>name</td><td>String</td><td>The name of the schema</td></tr>
  <tr><td>columns</td><td>Object</td><td>An object of columns where the key is the unique id for the column and the value is an object of type <a href="#column">Column</a></td></tr>
</table>

Example:
```json
{
  "name": "users",
  "columns": {
    "1": {
      "name": "id",
      "type": "uuid"
    },
    "2": {
      "name": "name",
      "type": "string"
    },
    "3": {
      "name": "email",
      "type": "string"
    },
    "4": {
      "name": "location",
      "type": "point"
    }
  }
}
```

#### Columns

Columns describe all the properties of data.
It is an object where the key is the id of the column and the value is a [Column](#column).

#### Column

A column describes a single property of data.
<table>
  <tr><th>Name</th><th>Type</th><th>Description</th></tr>
  <tr><td>name</td><td>String</td><td>The name of the column</td></tr>
  <tr>
    <td>type</td>
    <td>String</td>
    <td>
      The type of the column, which can be any <a href="#columntype">ColumnType</a>
    </td>
  </tr>
</table>

Example:
```json
{
  "name": "id",
  "type": "uuid"
}
```

#### ColumnType

The column type is a string indicating the type of the value in the data.
Here is a table to see what a the different types are and what their data format looks like.

<table>
  <tr>
    <th>Column type</th>
    <th>JSON type</th>
    <th>Example</th>
  </tr>
  <tr>
    <th>uuid</th>
    <td>String</td>
    <td>"583814cd-b90a-4341-8825-ab42d1125666"</td>
  </tr>
  <tr>
    <th>uuid[]</th>
    <td>Array</td>
    <td>["583814cd-b90a-4341-8825-ab42d1125666"]</td>
  </tr>
  <tr>
    <th>string</th>
    <td>String</td>
    <td>"Hello"</td>
  </tr>
  <tr>
    <th>string[]</th>
    <td>Array</td>
    <td>["Hello", "World"]</td>
  </tr>
  <tr>
    <th>text</th>
    <td>String</td>
    <td>"I can be very long"</td>
  </tr>
  <tr>
    <th>text[]</th>
    <td>Array</td>
    <td>["It's just text", "Yo!"]</td>
  </tr>
  <tr>
    <th>point</th>
    <td>Object (GeoJSON Point)</td>
    <td>{ "type": "Point", "coordinates": [50, 5] }</td>
  </tr>
  <tr>
    <th>point[]</th>
    <td>Object (GeoJSON MultiPoint)</td>
    <td>{ "type": "MultiPoint", "coordinates": [ [100.0, 0.0], [101.0, 1.0] ] }</td>
  </tr>
  <tr>
    <th>linestring</th>
    <td>Object (GeoJSON LineString)</td>
    <td>{ "type": "LineString", "coordinates": [ [100.0, 0.0], [101.0, 1.0] ] }</td>
  </tr>
  <tr>
    <th>linestring[]</th>
    <td>Object (GeoJSON MultiLineString)</td>
    <td>{ "type": "MultiLineString", "coordinates": [ [ [100.0, 0.0], [101.0, 1.0] ], [ [102.0, 2.0], [103.0, 3.0] ] ] }</td>
  </tr>
  <tr>
    <th>polygon</th>
    <td>Object (GeoJSON Polygon)</td>
    <td>{ "type": "Polygon", "coordinates": [ [ [100.0, 0.0], [101.0, 0.0], [101.0, 1.0], [100.0, 1.0], [100.0, 0.0] ] ] }</td>
  </tr>
  <tr>
    <th>polygon[]</th>
    <td>Object (GeoJSON MultiPolygon)</td>
    <td>{ "type": "MultiPolygon", "coordinates": [ [[[102.0, 2.0], [103.0, 2.0], [103.0, 3.0], [102.0, 3.0], [102.0, 2.0]]], [[[100.0, 0.0], [101.0, 0.0], [101.0, 1.0], [100.0, 1.0], [100.0, 0.0]], [[100.2, 0.2], [100.8, 0.2], [100.8, 0.8], [100.2, 0.8], [100.2, 0.2]]] ] }</td>
  </tr>
  <tr>
    <th>date</th>
    <td>String</td>
    <td>"2015-08-03"</td>
  </tr>
  <tr>
    <th>date[]</th>
    <td>Array</td>
    <td>["2015-08-03", "2016-09-04"]</td>
  </tr>
  <tr>
    <th>datetime</th>
    <td>String</td>
    <td>"2015-08-03 12:42:33"</td>
  </tr>
  <tr>
    <th>datetime[]</th>
    <td>Array</td>
    <td>["2015-08-03 12:42:33", "2015-08-03 12:42:49"]</td>
  </tr>
  <tr>
    <th>float</th>
    <td>Number</td>
    <td>15.04</td>
  </tr>
  <tr>
    <th>float[]</th>
    <td>Array</td>
    <td>[14.04, 15.04]</td>
  </tr>
  <tr>
    <th>integer</th>
    <td>Number</td>
    <td>420</td>
  </tr>
  <tr>
    <th>integer[]</th>
    <td>Array</td>
    <td>[1, 2, 3]</td>
  </tr>
  <tr>
    <th>boolean</th>
    <td>Boolean</td>
    <td>true</td>
  </tr>
  <tr>
    <th>boolean[]</th>
    <td>Array</td>
    <td>[ true, false ]</td>
  </tr>
  <tr>
    <th>json</th>
    <td>Object</td>
    <td>{ test: true, number: 1 }</td>
  </tr>
  <tr>
    <th>json[]</th>
    <td>Array</td>
    <td>[{ test: true }, {test: false}]</td>
  </tr>
</table>

## DataInfo

An object describing a piece of data.
This object is used as input for the transformer.
The transformer will apply changes to this object and modify the name, version and data if needed.

<table>
  <tr><th>Name</th><th>Type</th><th>Description</th></tr>
  <tr><td>projectId</td><td>String</td><td>A string indicating the id of the project</td></tr>
  <tr><td>projectName</td><td>String</td><td>A string indicating the name of the project</td></tr>
  <tr><td>projectVersion</td><td>Number</td><td>A string indicating the version of the project</td></tr>
  <tr><td>schemaId</td><td>String</td><td>A string indicating the id of the schema</td></tr>
  <tr><td>schemaName</td><td>String</td><td>A string indicating the name of the schema</td></tr>
  <tr><td>item</td><td>Object</td><td>The item that conforms to the schema</td></tr>
</table>

Example:
```json
{
  "projectId": "1",
  "projectName": "demo",
  "projectVersion": 3,
  "schemaId": "1",
  "schemaName": "users",
  "data": {
    "id": "088ea6d5-fd14-4460-aed0-1a4b0ceed555",
    "name": "Koen"
  }
}
```

### Changes

Changes describe a set of changes, changes are usually grouped a versioning mechachnism that spans across / locks the individual schema versions.
The data format is a simple array, containing changes as described below.

#### ProjectCreateChange

TODO

#### ProjectTagChange

TODO

#### ProjectRenameChange

TODO

#### ProjectRemoveChange

TODO

#### SchemaCreateChange

This change indicates that a schema is created.

<table>
  <tr><th>Name</th><th>Type</th><th>Description</th></tr>
  <tr><td>change</td><td>String</td><td>A string indicating the type of change</td></tr>
  <tr><td>projectId</td><td>String</td><td>The id of the project</td></tr>
  <tr><td>schemaId</td><td>String</td><td>The id of the schema</td></tr>
  <tr><td>schema</td><td>String</td><td>A <a href="#schema">Schema</a></td></tr>
</table>

Example:
```json
{
  "change": "schema.create",
  "projectId": "1",
  "schemaId": "1",
  "schema": {
    "name": "users",
    "columns": {
      "1": {
        "name": "id",
        "type": "uuid"
      }
    }
  }
}
```

#### SchemaRemoveChange

This change indicates that a schema is removed.

<table>
  <tr><th>Name</th><th>Type</th><th>Description</th></tr>
  <tr><td>change</td><td>String</td><td>A string indicating the type of change</td></tr>
  <tr><td>projectId</td><td>String</td><td>The id of the project</td></tr>
  <tr><td>schemaId</td><td>String</td><td>The id of the schema</td></tr>
  <tr><td>oldSchema</td><td>String</td><td>The old schema</a></td></tr>
</table>

Example:
```json
{
  "change": "schema.remove",
  "schemaId": "1",
  "oldSchema": {
    "name": "users",
    "columns": {}
  }
}
```

#### SchemaRenameChange

This change indicates that a schema is renamed.

<table>
  <tr><th>Name</th><th>Type</th><th>Description</th></tr>
  <tr><td>change</td><td>String</td><td>A string indicating the type of change</td></tr>
  <tr><td>schemaId</td><td>String</td><td>The id of the schema</td></tr>
  <tr><td>name</td><td>String</td><td>The name of the schema</a></td></tr>
  <tr><td>oldName</td><td>String</td><td>The old name of the schema</a></td></tr>
</table>

Example:
```json
{
  "change": "schema.rename",
  "schemaId": "1",
  "name": "users",
  "oldName": "user"
}
```

#### ColumnCreateChange

This change indicates that a column is added to a schema.

<table>
  <tr><th>Name</th><th>Type</th><th>Description</th></tr>
  <tr><td>change</td><td>String</td><td>A string indicating the type of change</td></tr>
  <tr><td>schemaId</td><td>String</td><td>The id of the schema</td></tr>
  <tr><td>columnId</td><td>String</td><td>The id for the column</td></tr>
  <tr><td>column</td><td>String</td><td>The properties of the <a href="#column">Column</a></td></tr>
</table>

Example:
```json
{
  "change": "column.create",
  "schemaId": "1",
  "columnId": "1",
  "column": {
    "name": "id",
    "type": "uuid"
  }
}
```

#### ColumnRemoveChange

This change indicates that a column is removed from a schema.

<table>
  <tr><th>Name</th><th>Type</th><th>Description</th></tr>
  <tr><td>change</td><td>String</td><td>A string indicating the type of change</td></tr>
  <tr><td>schemaId</td><td>String</td><td>The id of the schema</td></tr>
  <tr><td>columnId</td><td>String</td><td>The id for the column</td></tr>
  <tr><td>oldColumn</td><td>Object</td><td>The properties of the old <a href="#column">Column</a></td></tr>
</table>

Example:
```json
{
  "change": "column.remove",
  "schemaId": "1",
  "columnId": "2",
  "oldColumn": {
    "name": "name",
    "type": "string"
  }
}
```

#### ColumnRenameChange

This change indicates that a column is removed from a schema.

<table>
  <tr><th>Name</th><th>Type</th><th>Description</th></tr>
  <tr><td>schemaId</td><td>String</td><td>The id of the schema</td></tr>
  <tr><td>change</td><td>String</td><td>A string indicating the type of change</td></tr>
  <tr><td>columnId</td><td>String</td><td>The id for the column</td></tr>
  <tr><td>name</td><td>String</td><td>The new name of the column</td></tr>
  <tr><td>oldName</td><td>String</td><td>The old name of the column</td></tr>
</table>

Example:
```json
{
  "change": "column.rename",
  "schemaId": "1",
  "columnId": "1",
  "name": "id",
  "oldName": "user_id"
}
```

#### ColumnTypechangeChange

This change indicates that the type of a column is changed.

<table>
  <tr><th>Name</th><th>Type</th><th>Description</th></tr>
  <tr><td>change</td><td>String</td><td>A string indicating the type of change</td></tr>
  <tr><td>schemaId</td><td>String</td><td>The id of the schema</td></tr>
  <tr><td>columnId</td><td>String</td><td>The id for the column</td></tr>
  <tr><td>columnName</td><td>String</td><td>The name of the column</td></tr>
  <tr><td>type</td><td>String</td><td>The new type of the column</td></tr>
  <tr><td>oldType</td><td>String</td><td>The old type of the column</td></tr>
</table>

Example:
```json
{
  "change": "column.typechange",
  "schemaId": "1",
  "columnId": "1",
  "type": "integer",
  "oldType": "uuid"
}
```

# API docs
[API Docs](https://schema-mapper.github.io/spec)

# Licence
MIT