# syvain-metrics

Command line tool for [Syvain Metrics](https://metrics.041.io/). It is built for
agents and scripts: every command prints newline-delimited JSON on stdout,
errors go to stderr with exit code 1, and experiments and folders can be
addressed by slug or path instead of ids.

## Install

```sh
npm install -g syvain-metrics
```

The package installs the `syvain-metrics` binary.

## Authenticate

Sign in with a browser, or use an organization API key:

```sh
syvain-metrics auth login            # choose browser sign-in or an API key
syvain-metrics auth login --device   # browser sign-in without the prompt
printf '%s' "$KEY" | syvain-metrics auth login --stdin
syvain-metrics auth status
syvain-metrics auth logout

export SYVAIN_METRICS_API_KEY=ak_...   # skips the saved login
export SYVAIN_METRICS_HOST=https://metrics.041.io       # optional override
```

Browser sign-in prints a link and a short code. Open the link in a browser on
any device, for example your laptop when the CLI runs over SSH. There you can
sign in, create an account, and pick the organization to act in. The CLI waits
for your approval, then saves the login and refreshes it as needed. Only a
browser sign-in can manage organizations, members, and API keys.

The DuckDB extension and the Python API client read only an API key from the
saved login, so a browser sign-in also saves a personal API key of its active
organization, named after you and this machine. `org switch` and `org create`
replace it with a key of the new organization, and `auth logout` revokes it. It
works only while you are a member of the organization. If the key cannot be
revoked, `auth logout` keeps the login so you can retry; `--force` deletes it
anyway and names the key to revoke with `api-key revoke`.

An API key acts as its organization. Create one with
`syvain-metrics api-key create` or on the app's API keys page,
https://metrics.041.io/app/settings/api-keys. Agents and CI usually get one
through `SYVAIN_METRICS_API_KEY`.

The CLI talks to `https://metrics.041.io` unless `auth login --host` or
`SYVAIN_METRICS_HOST` names another origin; the host is saved with the login.

## Organizations, members, and API keys

These commands need a browser sign-in. Member and API key changes need the admin
role.

```sh
syvain-metrics org list                   # your organizations; current marks the active one
syvain-metrics org switch my-lab          # id, slug, or name
syvain-metrics org create "My Lab"        # you become admin; switches to it

syvain-metrics member list
syvain-metrics member role ada@example.com admin     # admin, member, or a role key
syvain-metrics member remove ada@example.com

syvain-metrics invitation create ada@example.com --role member
syvain-metrics invitation list
syvain-metrics invitation revoke orginv_...

syvain-metrics api-key create ci --expires-in-days 90 | jq -r .secret
syvain-metrics api-key list
syvain-metrics api-key revoke ak_...
```

An invitation emails a link that signs the person in or up and joins them to the
organization. `api-key create` prints the secret once; store it right away.

## Addressing things

- An **experiment** is its id or its slug: `experiment get mamba-run-001`.
- A **folder** is its absolute path or its id: `/models/mamba`. The root is `/`.
  Paths are derived from the organization tree, so names with spaces work when
  quoted.

## Folders

```sh
syvain-metrics folder list                 # every folder with its path
syvain-metrics folder list /models         # direct children of one folder
syvain-metrics folder create /models/mamba # creates missing parents; idempotent
syvain-metrics folder move /scratch/run-1 /models
syvain-metrics folder move /scratch/run-1 /      # to the root
syvain-metrics folder rename /models/mamba mamba-v2
```

`folder create` prints the leaf folder with `created: true` when it was made and
`created: false` when the path already existed.

## Experiments

```sh
syvain-metrics experiment list --search mamba      # experimentId, slug, displayName, folderId, folderPath
syvain-metrics experiment list --folder /models    # direct members only
syvain-metrics experiment get mamba-run-001        # status, timestamps, meta, folderPath, url
syvain-metrics experiment move mamba-run-001 /models/mamba
syvain-metrics experiment rename mamba-run-001 "Mamba, lr 3e-4"   # "" clears the display name
syvain-metrics experiment catalog mamba-run-001    # series names with metadata keys and values
syvain-metrics experiment annotations mamba-run-001 --limit 20
```

## Views

A view is a saved app workspace for the organization: which runs, which charts,
which axes. `view link` prints an app link to a state without saving it. The
state is JSON where every field is optional; its fields are in the app docs
under views and links.

```sh
syvain-metrics view list                              # viewId, name, url
syvain-metrics view get "LR sweep"                    # with the full state
syvain-metrics view create "LR sweep" --folder /mamba/lr-sweep --state '{"defaults":{"logY":true}}'
syvain-metrics view update "LR sweep" --name "LR sweep, log y"
syvain-metrics view delete "LR sweep, log y"
syvain-metrics view link --experiment run-a --experiment run-b --state '{"groupBy":"experiment"}'
```

## Series

`series query` prints one line per experiment, series, and metadata partition:
`{experimentId, seriesName, metadata, data: {step, timestampMs, value}}`.
Without `--name` it reads every series in the experiment's catalog.

```sh
syvain-metrics series query mamba-run-001 --name loss --filter split=valid
syvain-metrics series query mamba-run-001 > run-001.jsonl
syvain-metrics series query run-a run-b --name loss --x-axis timestamp
```

For analysis across many experiments prefer the
[DuckDB extension](../../libs/metrics_duckdb/README.md); the CLI is the quick
way to pipe one experiment into a file.

`series render` charts the same selections as a PNG through the public Metrics
Renderer. Every experiment, series, and metadata partition becomes one line, so
use `--filter` to narrow what is drawn.

```sh
syvain-metrics series render run-a run-b --name loss --filter split=valid -o loss.png
syvain-metrics series render run-a --name loss --display-mode subplots -o loss.png
```

Run `syvain-metrics --help` or `syvain-metrics <command> --help` for every flag.

For preview or local rendering, set `SYVAIN_METRICS_RENDERER_URL` to the
renderer's full `/render/metric-series.png` URL. The default is the production
renderer.