# @oceanstream/cli
## About
[OceanStream](https://oceanstream.io/) is an integrated cloud data platform used for hydroacoustic data collection and analysis in real-time. It is built on [Azure IoT Edge](https://azure.microsoft.com/en-us/products/iot-edge) infrastructure.
## OceanStream CLI
Install from NPM with:
(Add `-g` to install globally):
```sh
npm i @oceanstream/cli
```
## Usage
Run the tool without any command-line arguments to see a top-level commands description.
```
npx oceanstream
```
or shorter: `npx os`
Output will be:
```
Usage: oceanstream [options] [command]
Operations on OceanStream Edge modules
Options:
--version / -V output the version number
--help / -h display help for command
Commands:
create Initialize a new OceanStream project
delete Delete an OceanStream project
app Operations on Azure applications
env Manage project environments with different settings and configurations
device Operations on OceanStream edge devices
module Manage OceanStream Edge modules for specific tasks on devices
set Set current environment, container registry, and device
help display help for command
```
You can also see help on any of top-level commands, for example: `oceanstream app help`.
## Concepts
The CLI fullfills the following roles:
- manage the IoT Edge related infrastructure in Azure which is used by the OceanStream platform
- emulate IoT Edge devices using Azure VMs or local Docker containers
- build IoT Edge modules and deploy them on IoT Edge devices
**Application** is a representation of an Azure _App registration_. Consists of `tenantId`, `clientId`, `clientSecret`.
**Environment** is a predefined set of Azure infrastructure entities, which allows to operate on such set in a more convenient way. Currently environment consists of `appName` (reference to a specific application), `subscriptionId`, `location`, `resourceGroupName`, `iotHubName`.
**Device** is an Azure IoT Edge device.
**Module** is an Azure IoT Edge device-compatible module which essential wraps a Docker container.
### Azure setup
First of all, you'll need to create an Azure _app registration_ and assign it proper permissions. This is the only step which requires interacting with Azure web console (_TODO_: simplify using CLI commands). In Azure, create a new _app registration_. Create a new client secret for an _app registration_ (_Certificates & secrets_ section), save the _Client Secret ID_ somewhere safe. Also make a note of _Application (client) ID_ and _Directory (tenant) ID_ (can be seen in an _Overview_ section). If you don't have an active subscription, you also need to create one. Go to the subscription, open _Access control (IAM)_ section, go to _Roles_ and create a custom role named `IoT Contributor` using the following permissions:
```json
{
"actions": [
"*",
"Microsoft.Storage/storageAccounts/blobServices/containers/delete",
"Microsoft.Storage/storageAccounts/blobServices/containers/read",
"Microsoft.Storage/storageAccounts/blobServices/containers/write",
"Microsoft.Storage/storageAccounts/blobServices/generateUserDelegationKey/action"
],
"notActions": [
"Microsoft.Authorization/*/Delete",
"Microsoft.Authorization/*/Write",
"Microsoft.Authorization/elevateAccess/Action",
"Microsoft.Blueprint/blueprintAssignments/write",
"Microsoft.Blueprint/blueprintAssignments/delete",
"Microsoft.Compute/galleries/share/action"
],
"dataActions": [
"Microsoft.Devices/IotHubs/*",
"Microsoft.Devices/IotHubs/*/read",
"Microsoft.Devices/IotHubs/fileUpload/notifications/action",
"Microsoft.Devices/IotHubs/devices/*",
"Microsoft.Devices/IotHubs/twins/*",
"Microsoft.Storage/storageAccounts/blobServices/containers/blobs/delete",
"Microsoft.Storage/storageAccounts/blobServices/containers/blobs/read",
"Microsoft.Storage/storageAccounts/blobServices/containers/blobs/write",
"Microsoft.Storage/storageAccounts/blobServices/containers/blobs/move/action",
"Microsoft.Storage/storageAccounts/blobServices/containers/blobs/add/action"
],
"notDataActions": []
}
```
Assign this role to the previously created app registration.
### Initial project setup
To create a new project, use the `oceanstream create` command. If you're already in a directory with a `package.json` file, the tool will use the existing folder and the project name from the package.json file. If you're not in a directory with a package.json file, you'll be prompted to enter a project name, and a new sub-directory with the provided name will be created in the current directory.
### Environment
Now, create a new environment by running `oceanstream env add`. You will need to enter the environment name, select an application to use with an environment, select Azure Subscription, pick the location for the resources, create new or select existing Azure resource group, create new or select existing Azure IoT Hub (to reduce the costs, if it's the only IoT Hub you have, upon creation you can select `F1` tier and capacity of `1`). Note that if IoT Hub with such name exists anywhere globally, you will need to pick a different name. To see the list of created environments you can use `oceanstream env ls`.
### IoT Device
To create an IoT Device, just run `oceanstream device add`. Select an environment and enter the device name. To create a device emulator in the cloud, use `oceanstream device emulate`. Select an environment, pick the device you want to emulate, create new or select an existing security group (should allow SSH connection), create new or select an existing virtual network, pick the names for the subnet / public IP / network interface, enter the VM name, VM administrator name and the administrator password. After the VM has been created, the SSH command to connect to it will be displayed in the console. The VM might not be available right way as it usually requires a few minutes to be fully provisioned.
### Modules
To create an IoT Edge module, use `oceanstream module create`. Pick the name of the module and select the language to use. To build the module, use `oceanstream module build`. You will need to select an environment and for each of the following resource select existing or create new: container registry (to store image in), storage account (to upload module sources to), storage container (to store module sources in). After that, you'll need to select which module you want to build, pick the OS & architecture to build an image for and enter the image tag.
### Deploy modules on IoT Device
To apply the modules on the IoT Device, you need to create the manifest first. In a `manifests` subdirectory of a project, create a `.yaml` file with the following contents (just an example):
```yaml
modules:
jsmodule:
tag: 'v1'
csmodule:
tag: 'v2'
```
Here the `jsmodule` and `csmodule` are the names of the modules and `tag` is a tag to use when building the Docker image.
Now, execute the `oceanstream device apply`. Select the environment, IoT device to apply modules to, a container registry to pull modules from and the device manifest.
## Installation
To install the CLI, follow these steps:
1. Navigate to the project folder if you already have it, or download the project.
2. Install npm packages by running the command: `npm install`
3. Create a `.env` file and specify the environment variable: `NODE_ENV=development`
4. Execute the command: `npm link`
After completing these steps, try running `oceanstream help` to see a list of available commands and ensure the installation was successful.
## Commands
## Project
### create
Initialize a new IoT Edge project.
oceanstream create
| Parameter | Alias | Description |
| :-------- | :---- | :----------- |
| `--name` | `-n` | Project name |
Example:
If you want to create a new IoT Edge project in a new sub-directory, use the following command:
```bash
oceanstream create -n my-iot-project
```
Or
```bash
oceanstream create
```
You will be prompted to enter the project name:
```bash
Enter the project name: my-iot-project
```
After successful execution, you will receive the following output:
```bash
Creating the project my-iot-project
```
Now you can see your project folder in the directory from which the create command was executed.
```bash
oceanstream create
```
You will be prompted to enter the project name:
```bash
Enter the project name: my-iot-project
```
After successful execution, you will receive the following output:
```bash
Creating the project my-iot-project
```
Now you can see your project folder in the directory from which the create command was executed.
If you want to create a new IoT Edge project in the current directory using the project name from the package.json file, use the following command:
```bash
oceanstream create
```
After successful execution, you will receive the following output:
```bash
Creating the project
```
Now your current directory has been initialized as an IoT Edge project.
### delete
Delete an existing IoT Edge project.
```bash
oceanstream delete
```
This command will list all available IoT Edge projects in the current working directory and prompt you to select one to delete.
Example:
To delete an existing IoT Edge project, use the following command:
```bash
oceanstream delete
```
You will be prompted to select a project from the list of available projects:
```bash
? Select a project to delete: (Use arrow keys)
❯ my-iot-project
another-iot-project
```
After selecting the project and confirming the deletion, you will receive the following output:
```bash
Deleting the project my-iot-project
```
The selected project folder will be removed from the current working directory.
You can also delete the IoT Edge project located in the current directory.
Example:
```bash
oceanstream delete
```
You will be prompted to select a project to delete. Choose "Current Directory":
```bash
? Select a project to delete: Current Directory
```
After confirming the deletion, you will receive the following output:
```bash
Deleting the project Current Directory
Project Current Directory deleted.
```
The current directory, which is the selected project folder, will be removed along with all its contents.
Note: Be cautious when using this command, as it will permanently delete the chosen project directory and all its contents. Make sure you have backed up any important data before proceeding.
## App
The `app` command is used for managing Azure application credentials. To use this command, you need to navigate to your project's directory:
```sh
cd
```
Replace with the name of your project folder.
### add
Add new Azure application credentials.
```sh
oceanstream app add
```
Parameters:
- `Application name`: name of the Azure application
- `Directory (tenant) ID`: the tenant ID for your Azure Active Directory
- `Application (client) ID`: the client ID for your Azure application
- `Client secret`: the client secret for your Azure application
Example:
```sh
cd my-iot-project
```
then
```sh
oceanstream app add
```
This command will prompt you for the required information:
```sh
? Application name name-of-app
? Directory (tenant) ID my-id
? Application (client) ID my-client
? Client secret [hidden] enter your secret here
```
Fill in the details as prompted.
### list
List Azure application credentials.
```sh
oceanstream app list
```
or
```sh
oceanstream app ls
```
Example:
```sh
cd my-iot-project
oceanstream app ls
```
Output:
```sh
name-of-app
Directory (tenant) ID: my-id
Application (client) ID: my-client
```
### remove
Remove an Azure application from the configuration file.
```sh
oceanstream app remove
```
or
```sh
oceanstream app rm
```
Example:
```sh
cd my-iot-project
oceanstream app remove
```
Output:
```sh
? Select application to remove from the configuration file (Use arrow keys)
❯ name-of-app
```
This command will prompt you to select the Azure application you want to remove from the configuration file.
## Env
The env command is used for managing environments in your project. To use this command, you need to navigate to your project's directory:
```sh
cd
```
Replace with the name of your project folder.
### add
Add a new environment.
```sh
oceanstream env add
```
Prompts:
Environment name
- Select the application to use with the environment
- Azure Subscription ID
- Select the Azure location to create resources within the environment
- Resource group
- IoT Hub name
- Container registry (will be used to store module container images)
- Storage account (will be used to store module sources)
- Storage container (will be used to store module sources)
Example:
```sh
oceanstream env add
```
Output:
```sh
? Environment name my-env
? Select the application to use with the environment my-app
✔ Connected to Azure API
? Azure Subscription ID my-sub
? Select the Azure location to create resources within the environment westeurope
? Resource group my-resource-group
? IoT Hub my-env-iot-hub
? Container registry (will be used to store module container images) myenviotcr
? Storage account (will be used to store module sources) myenviotsa
? Storage container (will be used to store module sources) Create new
? Storage container name my-env-iot-sc
✔ Storage container my-env-iot-sc has been created
Environment my-env has been added
```
After executing the oceanstream env add command, you will be prompted to provide the necessary information to set up a new environment, including the name of the environment, the Azure application to use, the subscription ID, the location to create resources within the environment, and other configuration options. Once you have provided all the required information, the environment will be created and added to the configuration file. You can then use the oceanstream env list command to see a list of all the environments currently configured for your project.
### list
The env list command is used to display a list of all environments that have been added to your project. To use this command, navigate to your project's directory:
```sh
cd
```
Then, run the env list command:
```sh
oceanstream env ls
```
This will display a list of all the environments that have been added to your project, along with their associated configuration details, such as the application used with the environment, the subscription ID, resource group, IoT hub, container registry, and storage account.
For example:
```bash
my-env
Application: my-app
Subscription ID: sub-id
Resource group: my-resource-group
IoT Hub: my-env-iot-hub
Container registry: myenviotcr
Storage account: myenviotsa
Storage container: my-env-iot-sc
```
This can be useful for quickly reviewing the configuration details of each environment and ensuring that they are correct.
### remove
To remove an environment, you can use the `rm` or `remove` command. This command will remove the environment from the configuration file.
For example:
```sh
oceanstream env rm
```
After running this command, you will be prompted to select the environment to remove from the configuration file.
Please note that removing an environment will not delete the resources created in Azure. If you want to delete the Azure resources associated with an environment, you will need to do so manually using the Azure portal or Azure CLI.
## Device
The device command is used to manage Azure IoT Edge devices. Before using this command, make sure you navigate to your project's directory:
```sh
cd
```
Replace with the name of your project folder.
### add
Add a new (or existing) Azure IoT Edge device.
| Parameter | Alias | Description |
| :-------------- | :---- | :------------------------------------------- |
| `--environment` | `-e` | Select the environment to create a device in |
| `--device-name` | `-n` | Name of the IoT Edge device |
Example:
If you want to add a new Azure IoT Edge device to a specific environment and provide the device name with the command, use the following command:
```bash
oceanstream device add -e my-environment -n my-iot-device
```
If you want to add a new Azure IoT Edge device and provide the environment and device name interactively, use the following command without the --environment and --device-name options:
```sh
oceanstream device add
```
You will be prompted to select the environment to create a device in, and enter a name for the device. Once the device is successfully created, you will see a message confirming that the IoT Edge device has been created:
```sh
IoT Edge Device has been created
```
### emulate
Emulate an IoT Edge device.
Options:
| Option | Alias | Description |
|---------------------------|-------|-----------------------------------------------|
| --environment | -e | Select the environment to emulate |
| --strategy | -s | Select the strategy to emulate: [Cloud, Local]|
| --device | -n | Name of the IoT Edge device |
| --security-group | -g | Name of the security group |
| --virtual-network-name | -v | Name of the virtual network |
| --public-ip | -p | Name of the public IP |
| --domain-name | -d | Domain name |
| --network-interface-name | -i | Name of the network interface |
| --virtual-machine | -m | Name of the virtual machine |
| --username | -u | Name of the admin |
| --password | -w | Password of the username |
Example:
To emulate a device with specific settings, run the following command:
```bash
oceanstream device emulate -e my-environment -n my-iot-device -s Cloud|Local -g my-security-group -v my-virtual-network -p my-public-ip -d my-domain-name -i my-network-interface -m my-virtual-machine -u my-admin-username -w my-admin-password
```
To emulate a device interactively select options, run the following command without providing additional options:
```sh
oceanstream device emulate
```
To emulate a device, run the device emulate command and select the environment where you want to emulate the device. The command will connect to the Azure API and prompt you to select the device you want to emulate. Then, you will be prompted to choose an emulation strategy. You can choose to emulate the device locally or in the cloud using an Azure virtual machine.
If you choose to emulate the device in the cloud, the command will prompt you to select a security group to enable SSH access to the virtual machine and a virtual network to use for the virtual machine. Then, you will be prompted to enter an administrator username and password for the virtual machine. Once the virtual machine is created, you will be provided with instructions on how to connect to it using SSH.
### apply
The device apply command is used to apply module configurations to an Azure IoT Edge device. Before using this command, you should already have created an IoT Edge device and modules to apply to it.
Options:
| Option | Alias | Description |
| ------------- | ----- | --------------------------------------------- |
| --environment | -e | Select the environment to apply the config to |
| --device | -n | Name of the IoT Edge device |
| --registry | -r | Select the registry to apply |
| --manifest | -m | Select the manifest to apply |
First, create a new manifest file in the manifests directory of your project:
```sh
cd my-iot-project
mkdir manifests
cd manifests
touch example-manifest.yaml
```
Then, open the example-manifest.yaml file and add the content(just example):
```sh
modules:
tmt:
tag: "tag-name"
```
Save and close the file.
Finally, run the oceanstream device apply command to apply the modules configuration to the IoT device:
```sh
oceanstream device apply [options]
```
Here's an example of using the oceanstream device apply command along with the contents of a manifest file:
First, create a new manifest file in the manifests directory of your project:
```sh
Copy code
cd my-iot-project
mkdir manifests
cd manifests
touch example-manifest.yaml
```
Then, open the example-manifest.yaml file and add the following content:
Copy code
```yaml
modules:
tmt:
tag: 'v1'
```
Save and close the file.
Finally, run the oceanstream device apply command to apply the modules configuration to the IoT device:
```sh
oceanstream device apply
```
You will be prompted to select the environment to deploy the modules in and to choose the IoT device from the IoT Hub to apply the modules configuration to. You will also need to select the container registry to pull modules from and the device manifest to apply. Once you have made these selections, the modules from the manifest will be applied to the IoT device.
## Module
The module command is used for managing Azure IoT Edge modules. You can perform various operations with this command such as adding, listing, removing, and building modules.
### add
To add a new IoT Edge module, use the following command:
Options:
| Option | Alias | Description |
| ------------- | ----- | ------------------------------------------------ |
| --module-name | -n | Name of the IoT Edge module |
| --language | -l | Language of the IoT Edge module. [Cs, Js, Py, C] |
| --source | -s | Source of the IoT Edge module |
| --path | -p | Path of the IoT Edge module. For local modules |
| --username | -u | Github username. For Github modules |
| --repo | -r | Github repo name. For Github modules |
| --token | -t | Github token. For Github modules |
```sh
oceanstream module add [options]
```
or
```sh
oceanstream module add
```
Then you will be prompted to provide the following information:
The name of the module: the name of the new module you want to create.
Module's programming language: the programming language used to write the module code.
Module's source: the source of the module, which can be one of the following options:
New module in a project directory: a new module will be created in your current project directory.
Existing module in a local filesystem directory: the module will be imported from a local directory.
Existing module in a GitHub repository: the module will be imported from a GitHub repository.
After providing the required information, the new module will be created and added to your project.
For example, if you enter the command oceanstream module add, and then provide the name "mtm", select the programming language "C#", and choose to create a new module in a project directory, the output will be:
```sh
Created new module mtm
```
This means that a new module named "mtm" has been created in your project directory.
### list
List all the IoT Edge modules in the project.
```
oceanstream module list
```
or
```sh
oceanstream module ls
```
### remove
Remove an IoT Edge module from the project.
Options:
| Option | Alias | Description |
| ------ | ----- | --------------------------- |
| --name | -n | Name of the IoT Edge module |
```sh
oceanstream module remove
```
or
```sh
oceanstream module rm
```
During the execution of the command, if the --name option is not provided, you will be asked to select the module to remove from the project. After the selection or if the --name option is provided, the specified module will be removed from the project.
### build
The module build command is used for building an IoT Edge module for a specific environment.
Options:
| Option | Alias | Description |
| -------------- | ----- | ------------------------------------ |
| --environment | -e | Select the environment to build in |
| --module | -m | Select the module to build |
| --os | -o | Select the OS to build for |
| --architecture | -a | Select the architecture to build for |
| --tag | -t | Select the tag to build |
```sh
oceanstream module build
```
Once you're in the project directory, you can run the module build command to build a module for a specific environment. If options are not provided, this command will prompt you to select the environment, module, operating system, architecture, and image tag for the module you want to build.
After you've answered all of the prompts or provided options, the command will build the module and create a container with the specified tag.
## Set
Command for use current configuration
### set
Set the current environment, registry, and device for the project.
```bash
oceanstream set
```
Options:
| Option | Alias | Description |
| :----------- | :---- | :-------------------------- |
| `--env` | `-e` | Set the current environment |
| `--registry` | `-r` | Set the current registry |
| `--device` | `-d` | Set the current device |
Example:
To set the current environment, registry, and device, use the following command:
```bash
oceanstream set --env production --registry my-registry --device my-device
```
After successful execution, you will receive the following output:
```bash
Environment production is set as current
Registry my-registry is set as current
Device my-device is set as current
[WARNING]: The specified environment, registry, and device will be automatically applied for future actions. If not specified, the current settings will be used. To change them, simply provide the new values again.
```
To set only the current environment, use the following command:
```bash
oceanstream set --env production
```
After successful execution, you will receive the following output:
```bash
Environment production is set as current
[WARNING]: The specified environment, registry, and device will be automatically applied for future actions. If not specified, the current settings will be used. To change them, simply provide the new values again.
```
Similarly, you can set only the current registry or device by using the --registry or --device option respectively.
## Unit tests
Unit tests for the code base were written in Nightwatch using the `@nightwatch/apitesting` plugin. More info on [API testing](https://nightwatchjs.org/guide/writing-tests/api-testing.html) with Nightwatch.
Run tests with:
```sh
npm test
```
## Licence
MIT