env

<div id="top" align="center">
  <img
	alt="logo"
	src="https://nodejs.org/static/images/logo.svg"
	width="256px"
  />

  </br>

  <h1 align="center"><b>env</b></h1>
  <h4 align="center">¡Environment variables made easy!</h4>
</div>

<br />

<p align="center">
  <img
	src="https://img.shields.io/badge/version-1.0.0-blue?style=flat-square"
	alt="version"
  />
  &nbsp;
  <img
	src="https://img.shields.io/badge/TypeScript-007ACC?style=flat-square&logo=typescript&logoColor=white"
	alt="typescript"
  />
  &nbsp;
  <img
	src="https://img.shields.io/badge/nodejs-~14.0.0_||_^16.14.2-darkgreen?style=flat-square"
	alt="nodejs engine"
  />
  &nbsp;
  <img
	src="https://img.shields.io/badge/npm->=7.5.6-darkgreen?style=flat-square"
	alt="npm engine"
  />
</p>

<br />

<!-- ABOUT THE PROJECT -->

## 📖 **About**

Eases NodeJS <b>environment variable handling</b>, like [env-cmd](https://www.npmjs.com/package/env-cmd) or [dotenv](https://www.npmjs.com/package/dotenv), but with <b>powerfull features and extensibility</b> for adding custom providers (as plugins) for <u>load</u>, <u>pull</u> and <u>push</u> the variables from different stores.

<p align="right">(<a href="#top">back to top</a>)</p>

<!-- REQUIREMENTS -->

## 📌 **Requirements**

First, [download](https://nodejs.org/) and install **NodeJS**. Version `16` or higher is required.

Validate installed versions of node and npm with:

```bash
> node -v
v16.14.2

> npm -v
8.3.0
```

You can initialize a new npm project using:

```bash
> npm init
```

<p align="right">(<a href="#top">back to top</a>)</p>

<!-- QUICK START -->

## ⚡️ **Quick start**

> 🔔 Make sure that you have [NodeJS 14+](https://nodejs.org/) installed on your computer.

-   Installs the package:

```bash
> npm install @achs/env

  added 1 packages, and audited 1 packages in 1s

  found 0 vulnerabilities
> _
```

-   Executes binary directly:

```bash
> node_modules/.bin/env --help

  Usage: env [command] [options..] [: subcmd [:]] [options..]

  Commands:
	env [options..] [: <subcmd> :]
	env pull [options..]
	env push [options..]
	env schema [options..]
> _
```

```bash
> npx env --help

  Usage: env [command] [options..] [: subcmd [:]] [options..]

  Commands:
	env [options..] [: <subcmd> :]
	env pull [options..]
	env push [options..]
	env schema [options..]
> _
```

-   Or add desired commands in your **npm script** in `package.json`:

```javascript
{
  ...,
  "scripts": {
	// starts project injecting "dev" environment variables and debug log level
	"start:dev": "env -e dev -m debug : node dist/main.js : --log debug",
	// starts project injecting "prod" environment variables
	"start:prod": "env -e prod -m debug : node dist/main.js",
	...,
	// builds project injecting "prod" environment variables
	"build:prod": "env -e prod -m build : tsc",
	...,
	"env:schema": "env schema -e dev --ci",
	// uploads environment "dev" variables
	"env:push:dev": "env push -e dev",
	// downloads environment "dev" variables
	"env:pull:dev": "env pull -e dev"
  },
  ...
}
```

-   Execs your command:

**file**: _dist/main.js_

```javascript
console.log(`My environment loaded is: ${process.env.ENV}`);
```

```bash
> npm run start:dev

  13:31:59.865 INFO  loading dev environment in debug mode
  13:31:59.911 DEBUG using package-json provider
  13:31:59.912 DEBUG using app-settings provider
  13:31:59.914 DEBUG using secrets provider
  13:32:00.109 DEBUG environment loaded:
  {
	NODE_ENV: 'development',
	ENV: 'dev',
	VERSION: '1.0.0',
	NAME: '@my-app',
	VAR1: true,
	VAR2: true,
	GROUP1__VAR1: 'G1V2',
	ARR1: '1,val,true',
	SECRET: '***'
  }
  13:32:00.116 INFO  executing command > node dist/main.js
  My environment loaded is: dev
  13:32:00.232 INFO  process finished successfully
> _
```

<p align="right">(<a href="#top">back to top</a>)</p>

## ⛩ **Structure**

```bash
├── src/
│   ├── commands/ # lib commands handlers
│   │   ├── env.command.ts
│   │   ├── export.command.ts
│   │   ├── pull.command.ts
│   │   ├── push.command.ts
│   │   └── schema.command.ts
│   ├── interfaces/ # provider interfaces
│   ├── providers/ # integrated providers
│   │   ├── package-json.provider.ts
│   │   ├── app-settings.provider.ts
│   │   ├── local.provider.ts
│   │   └── azure-key-vault.provider.ts
│   ├── utils/
│   │   ├── command.util.ts
│   │   ├── interpolate.util.ts
│   │   ├── json.util.ts
│   │   ├── normalize.util.ts
│   │   ├── schema.util.ts
│   │   └── logger.ts
│   ├── arguments.ts # global arguments
│   ├── exec.ts # initialization logic (load config, commands, etc.)
│   └── main.ts
├── tests/ # integration tests
├── .eslintrc.json
├── jest.config.json
├── tsconfig.build.json
└── tsconfig.json
```

<p align="right">(<a href="#top">back to top</a>)</p>

<!-- COMMANDS AND OPTIONS -->

## ⚙️ **Commands & Options**

Options handling has the ability of **replace arguments itself**, using `[[` and `]]` as delimiters.
So, in example for define your config file path, you must use your _root_ argument,
supposing root has the value of "config", this definition _`[[root]]/any-config-file.json`_ will be
_`config/any-config-file.json`_, or if your _env_ argument is "dev", this definition
_`[[root]]/config-file.[[env]].json`_ will be _`config/config-file.dev.json`_.

<div align="center">
  <span style="font-size:20px;font-weight:bold" align="center">Options</span>
</div>

### Global options

| Option							 | Description									 | Type	   | Default | Required? |
| ---------------------------------- | ----------------------------------------------- | ---------- | ------- | --------- |
| `--help`						   | Shows help									  | `boolean`  |		 | No		|
| `--e, --env`					   | Environment for load							| `string`   |		 | Yes	   |
| `-m, --modes`					  | Execution modes								 | `string[]` | `[]`	| No		|
| `--nd, --nestingDelimiter`		 | Nesting level delimiter for flatten			 | `string`   | `__`	| No		|
| `--arrDesc, --arrayDescomposition` | Whether serialize or break down arrays		  | `boolean`  | `false` | No		|
| `-x, --expand`					 | Interpolates environment variables using itself | `boolean`  | `false` | No		|
| `-ci`							  | Continuous Integration mode					 | `boolean`  | `false` | No		|

</br>

### Workspace options

| Option			 | Description					   | Type	 | Default						   | Required? |
| ------------------ | --------------------------------- | -------- | --------------------------------- | --------- |
| `--root`		   | Default environment folder path   | `string` | `env`							 | No		|
| `-c, --configFile` | Config JSON file path			 | `string` | `[[root]]/settings/settings.json` | No		|
| `-s, --schemaFile` | Environment Schema JSON file path | `string` | `[[root]]/settings/schema.json`   | No		|

### JSON Schema options

| Option				 | Description												| Type			  | Default | Required? |
| ---------------------- | ---------------------------------------------------------- | ----------------- | ------- | --------- |
| `-r, --resolve`		| Whether merges new schema or override					  | `merge, override` | `merge` | No		|
| `--null, --nullable`   | Whether variables are nullable by default				  | `boolean`		 | `true`  | No		|
| `--df, --detectFormat` | Whether format of strings variables are included in schema | `boolean`		 | `true`  | No		|

### Logger options

| Option			  | Description | Type									 | Default | Required? |
| ------------------- | ----------- | ---------------------------------------- | ------- | --------- |
| `--log, --logLevel` | Log level   | `silly, trace, debug, info, warn, error` | `info`  | No		|

<div align="center">
  <span style="font-size:20px;font-weight:bold" align="center">Commands</span>
</div>

-   ## **`env`**

Inject your environment variables into `process.env` and executes a command.

```bash
env -e [env] [options..] [: subcmd [:]] [options..]
```

Examples:

```bash
> env -e dev -m test unit : npm test
```

```bash
> env -e dev -m debug : npm start : -c [[root]]/[[env]].env.json
```

```bash
> env -e prod -m build optimize : npm build
```

-   ## **`pull`**

Pulls environment variables from providers stores.

```bash
env pull -e [env] [options..]
```

| Option			| Description			   | Type	  | Default | Required? |
| ----------------- | ------------------------- | --------- | ------- | --------- |
| `-o, --overwrite` | Overwrite local variables | `boolean` | `false` | No		|

Examples:

```bash
> env pull -e dev
```

-   ## **`push`**

Pushes environment variables to providers stores.

```bash
env push -e [env] [options..]
```

| Option		| Description						  | Type	  | Default | Required? |
| ------------- | ------------------------------------ | --------- | ------- | --------- |
| `-f, --force` | Force push for secrets (replace all) | `boolean` | `false` | No		|

Examples:

```bash
> env push -e dev
```

-   ## **`schema`**

Generates validation schema from providers output variables.

```bash
env schema -e [env] -m [modes] [options..]
```

Examples:

```bash
> env schema -e dev -m build
```

-   ## **`export`**

Export unified environment variables to a file from providers.

```bash
env export -e [env] -m [modes] [options..]
```

| Option		  | Description						| Type	 | Default  | Required? |
| --------------- | ---------------------------------- | -------- | -------- | --------- |
| `-u, -p, --uri` | Uri for export file with variables | `string` | `.env`   | No		|
| `-f, --format`  | Format for export variables		| `string` | `dotenv` | No		|

Examples:

```bash
> env export -e dev -m build -f json --uri [[env]].env.json
```

<p align="right">(<a href="#top">back to top</a>)</p>

<!-- PROVIDERS -->

## 📡 **Providers**

Main feature of this library is using providers for get and set environment variables.
So, you can define your own provider, but lib came with 3 integrated providers:

-   ## **`package-json`**

Load some info from your project `package.json`.

Info read is:

```json
{
	"version": "1.0.0",
	"project": "project-name",
	"name": "@package-name",
	"title": "app-name",
	"description": "any description"
}
```

| Option			  | Description				 | Type	 | Default | Required? |
| ------------------- | --------------------------- | -------- | ------- | --------- |
| `--vp, --varPrefix` | Prefix for loaded variables | `string` | `""`	| No		|

Examples:

```bash
> env -e dev -m build : react-script build : --vp REACT_APP_
```

</br>

-   ## **`app-settings`**

Non secrets loader for `appsettings.json`.

`appsettings.json` file has the format below:

```json
{
	"|DEFAULT|": {},
	"|MODE|": {},
	"|ENV|": {}
}
```

In example:

```json
{
	"|DEFAULT|": {
		"VAR1": "v1_default"
	},
	"|MODE|": {
		"build": {
			"NODE_ENV": "production"
		},
		"debug": {
			"NODE_ENV": "development"
		},
		"test": {
			"NODE_ENV": "test"
		}
	},
	"|ENV|": {
		"dev": {
			"C1": "V1",
			"C2": "V2",
			"C3": 3,
			"GROUP1": {
				"VAR1": null,
				"VAR2": "G1V2",
				"VAR3": true,
				"GROUP2": {
					"VAR1": "G1G2V1"
				}
			},
			"C4": "23"
		}
	}
}
```

| Option				  | Description						  | Type	 | Default					 | Required? |
| ----------------------- | ------------------------------------ | -------- | --------------------------- | --------- |
| `--ef, --envFile`	   | Environment variables file path	  | `string` | `[[root]]/appsettings.json` | No		|
| `--sp, --sectionPrefix` | Prefix for env and modes in env file | `string` | `::`						| No		|

</br>

-   ## **`package-json`**

Load some info from your project `package.json`.

Info read is:

```json
{
	"version": "1.0.0",
	"project": "project-name",
	"name": "@package-name",
	"title": "app-name",
	"description": "any description"
}
```

| Option			  | Description				 | Type	 | Default | Required? |
| ------------------- | --------------------------- | -------- | ------- | --------- |
| `--vp, --varPrefix` | Prefix for loaded variables | `string` | `""`	| No		|

Examples:

```bash
> env -e dev -m build : react-script build : --vp REACT_APP_
```

-   ## **`azure-key-vault`**

Azure Key Vault provider, allows to load secrets from vault store to `env/secrets/[[env]].env.json` per environment.
Also, handles `env/secrets/[[env]].local.env.json` for load local variables with precedence over base.

| Option								  | Description									| Type	   | Default								   | Required? |
| --------------------------------------- | ---------------------------------------------- | ---------- | ----------------------------------------- | --------- |
| `--secretFolder`						| Secret variables folder path				   | `string`   | `[[root]]/secrets`						| No		|
| `--secretFile`						  | Secret variables file path					 | `string`   | `[[secretFolder]]/[[env]].env.json`	   | No		|
| `--localSecretFile`					 | Local secret variables file path			   | `string`   | `[[secretFolder]]/[[env]].local.env.json` | No		|
| `-k, --keys, --keysFile`				| Azure Key Vault SPN credentials files paths	| `string[]` | `['../keys.json', '[[root]]/keys.json']`  | No		|
| `--url, --vaultUrl`					 | Azure Key Vault server URL					 | `string`   |										   | Yes	   |
| `--spn, --clientId, --id`			   | SPN Client ID								  | `string`   |										   | Yes	   |
| `-p --password, --pass, --clientSecret` | SPN Client Secret Password					 | `string`   |										   | Yes	   |
| `-t, --tenant`						  | Azure Tenant ID								| `string`   |										   | Yes	   |
| `--mock`								| Mocks Azure Key Vault client (testing purpose) | `string`   | `false`								   | No		|

<p align="right">(<a href="#top">back to top</a>)</p>

<!-- PROVIDERS -->

## ✒ **Creating Custom Providers**

You can create your custom providers, in two ways:

-   **Local Script**: you must create a JavaScript file (.js), exporting by default your "provider" following standard interface exported by this lib.
-   **NPM Package**: you must create your custom NPM library and export by default your "provider" using standard interface exported by this lib.

How to load your provider is shown in Config Section.

In example, a provider exported by your NPM package written in TypeScript should be like:

```typescript
import { CommandArguments, EnvProvider } from '@achs/env';
import { logger, readJson, writeJson } from '@achs/env/utils';

const KEY = 'my-unique-provider-key';

interface MyProviderCommandArguments extends CommandArguments {
	anyExtraOption: boolean;
}

export const MyProvider: EnvProvider<MyProviderCommandArguments> = {
	// unique identifier for provider
	key: KEY,

	// (optional) allows to provider adds new arguments/options
	// to commands using yargs for builder
	builder: (builder) => {
		builder.options({
			anyExtraOption: {
				group: KEY,
				alias: ['a', 'aeo'],
				type: 'boolean',
				default: false,
				describe: 'Any option description',
			},
		});
	},

	// call on environment variables loading,
	// may be a Promise
	load: ({ env, modes, ...options }) => {
		if (env === 'dev')
			return {
				NODE_ENV: 'development',
			};

		// you can return a list of JSON environment variables for merge
		return [
			{
				NODE_ENV: 'production',
			},
			{
				ANY_VAR: 'ANY_VALUE',
				ANY_GROUP: {
					INNER_VAR: 12,
				},
			},
		];
	},

	// (optional) call on pulling variables from provider store,
	// config may pass in your config file
	pull: ({ env, modes, ...options }, config) => {
		// anyway you want for pulling variables to cache
	},

	// (optional) call on pushing/updating variables to provider store,
	// config may pass in your config file
	push: ({ env, modes, ...options }, config) => {
		// anyway you should do for pushing or updating your variables
	},
};
```

<p align="right">(<a href="#top">back to top</a>)</p>

<!-- CONFIG -->

## 📥 **Config**

You can configure any config argument inside you config file, but commonly providers are designed for this purpose.

```javascript
{
	"log": "silly",
	// will hide values of keys SECRET and MY_API_KEY in logging
	"logMaskValuesOfKeys": ["SECRET", "MY_API_KEY"],
	// integrated providers and custom providers together
	"providers": [
		{
			"path": "package-json"
		},
		{
			"path": "app-settings"
		},
		{
			"path": "azure-key-vault",
			"config": {
				"dev": {
					"vaultUrl": "https://kv-desa-ittec-sti.vault.azure.net"
				},
				"qa": {
					"vaultUrl": "https://kv-qa-ittec-sti.vault.azure.net"
				}
			}
		},
		{
			"path": "local"
		},
		{
			// custom NPM package
			"path": "@npm-package",
			"type": "module",
			"config": {
				"any-config": "any value"
			}
		},
		{
			// custom script inside project
			"path": "scripts/custom-loader.js",
			"type": "script"
		}
	]
}
```

<!-- AZURE KEY VAULT -->

## 💽 **Azure Key Vault**

Allows you to store your secrets in Azure Key Vault.

#### 1.2. NPM Scripts

For load desired environment, add you npm script like **`env -e <env> -m <mode1[ mode2]> : <your-command>`**.

-   **mode**: (build|debug|test) execution mode base variables.
-   **env**: (dev|qa|stg|prod) environment variables.

_In example: `env -e dev -m debug : npm start`_

## 2. Structure

#### 2.1. Environments

Your `env/secrets` folder will contain files below:

-   **dev.env.json**: development environment.
-   **dev.local.env.json**: local development environment (takes precedence).
-   **qa.env.json**: quality assurance environment.
-   **qa.local.env.json**: local qa environment (takes precedence).
-   **prod.env.json**: production environment.
-   **prod.local.env.json**: local production environment (takes precedence).

_This folder should contains environment variables files for system environments._

#### 2.2. Keys (env/keys.json)

Your `keys.json` file should contains you Azure Key Vault SPN credentials per environment:

```json
{
	"<env-name>": {
		"vaultUrl": "<azure-key-vault-url>", // you can skip this var if present in config
		"clientId": "<spn-client-id>",
		"clientSecret": "<spn-secret-password>",
		"tenantId": "<tenant-id>"
	},
	...
}
```

In example:

```json
{
	"dev": {
		"clientId": "f176a774-239e-4cd3-8551-88fd9fb9b441",
		"clientSecret": "WyBwkmcL8rGQe9B2fvRLDrqDuannE4Ku",
		"tenantId": "6d4bbe0a-5654-4c69-a682-bf7dcdaed8e7"
	},
	"qa": {
		"clientId": "5dcd9f45-7067-4387-94d8-e5e7066ba630",
		"clientSecret": "60ec5e16430a46eba70dfea80d721b66",
		"tenantId": "6d4bbe0a-5654-4c69-a682-bf7dcdaed8e7"
	}
}
```

Your secrets will be grouped, using your "name" and "project" variables from `package.json` file.

_This file allows to load environment files locally first run time._

## 3. Commands

You can use two command scripts for refresh your local env files
or publish/updates env files in the azure key vault from your local files.

-   **Pulls Secrets File**: `env pull -e <env> [-o]`. (-o forces to replace your local file).
-   **Pushes/Publishes Secrets File**: `env push -e <env>`.

#### 3.1. Environment Variables for Credentials

You can set your credentials variables from node environment variables.

In example:

```bash
user@machine:/mnt/c/Users/user$ AZURE_VAULT_URL=https://kv-desa-ittec-sti.vault.azure.net \
								AZURE_CLIENT_ID=f176a774-239e-4cd3-8551-88fd9fb9b441 \
								AZURE_CLIENT_SECRET=WyBwkmcL8rGQe9B2fvRLDrqDuannE4Ku \
								AZURE_TENANT_ID=6d4bbe0a-5654-4c69-a682-bf7dcdaed8e7 \
								node_modules/.bin/env pull -e dev -o
```

or

```bash
user@machine:/mnt/c/Users/user$ npx pull -e dev -o \
								--vaultUrl https://kv-desa-ittec-sti.vault.azure.net \
								--spn f176a774-239e-4cd3-8551-88fd9fb9b441 \
								--password WyBwkmcL8rGQe9B2fvRLDrqDuannE4Ku \
								--tenant 6d4bbe0a-5654-4c69-a682-bf7dcdaed8e7
```

## 4. Schema

This lib uses JSON schema for validate and retrieve secrets from store.

For each property in the file, loader will retrieve the value from Azure Key
Vault.

When you push a new variable from any of your environment secrets
file, `env.schema.json` will be updated automatically.

If you want to ignore to load some variable without delete it, you can remove
the variable from `env.schema.json`.

## 5. Shared and Nested Keys

You can organize your keys in nested objects,
and declare project shared secrets (skips group
separation) prefixing with '\$' like:

```json
{
	// .dev.env.json
	"$SHARED": "sharedValue",
	"GROUP1": {
		"$SHARED": "sharedValue2",
		"VAR": "anyValue1",
		...
	},
	"GROUP2": {
		"VAR": "anyValue2",
		"SUBGROUP1": {
			"VAR": "anyValue1",
			...
		},
		...
	},
	"VAR3": "anyValue3",
	...
}
```

So, in your application you can use the variables as shown below:

```javascript
const myVar1 = process.env.GROUP1__VAR;
const myVar2 = process.env.GROUP2__VAR;
const myVar2 = process.env.GROUP2__SUBGROUP1_VAR;
const myVar3 = process.env.VAR3;
// shared vars will load in every project
const mySharedVar1 = process.env.SHARED;
const mySharedNestedVar2 = process.env.GROUP1__SHARED;
```

## 6. Priority

### From lowest to highest.

-   `(dev|qa|prod).env.json`
-   `(dev|qa|prod).local.env.json` (takes precedence over previous)

<p align="right">(<a href="#top">back to top</a>)</p>

<!-- LINTING -->

## 🧿 **Linting**

Project uses ESLint, for code formatting and code styling normalizing.

-   **eslint**: linter integrated with TypeScript.

For correct interpretation of linters, is recommended to use [Visual Studio Code](https://code.visualstudio.com/) as IDE and install the plugins in .vscode folder at 'extensions.json', as well as use the config provided in 'settings.json'

<p align="right">(<a href="#top">back to top</a>)</p>

<!-- CHANGELOG -->

## 📋 Changelog

For last changes see [CHANGELOG.md](CHANGELOG.md) file for details.

<p align="right">(<a href="#top">back to top</a>)</p>

<!-- BUILT WITH -->

## 🛠️ Built with

-   [yargs](http://yargs.js.org/)
-   [tslog](https://tslog.js.org/#/)
-   [subslate](https://github.com/josh-hemphill/subslate)
-   [merge-deep](https://github.com/jonschlinkert/merge-deep)
-   [ajv](https://ajv.js.org/)
-   [to-json-schema](https://www.npmjs.com/package/to-json-schema)

<p align="right">(<a href="#top">back to top</a>)</p>

---

<br />

<p align="center">
  <img
	width="15%"
	src="https://upload.wikimedia.org/wikipedia/commons/0/09/Logo_ACHS.svg"
  />
  <h2 align="center">ASOCIACIÓN CHILENA DE SEGURIDAD</h2>
  <h3 align="center">Tranformación Digital ▪ Equipo de Desarrollo</h3>
</p>