# AI Runtime — a hands-on guide

A start-to-finish guide to **installing and using `ai-runtime-engine`**. It's written to be *studied*: read
top to bottom, run the snippets as you go, and you'll understand both **how it works** and **how to use it**.

You do **not** need an API key to follow along — a built-in mock provider lets you run everything offline.

**Naming, once:** the **product** is *AI Runtime*, the **npm package** is `ai-runtime-engine`, and the
**command-line tool** is `ai-runtime`. In code you use two classes: **`Runtime`** (the high-level API you'll
use most) and **`AI`** (the low-level router underneath it).

---

## Contents

1. [What you're installing](#1-what-youre-installing)
2. [Install](#2-install)
3. [Your first run (offline, no keys)](#3-your-first-run-offline-no-keys)
4. [How it works — the mental model](#4-how-it-works--the-mental-model)
5. [Using real providers](#5-using-real-providers)
6. [How to use it — worked examples](#6-how-to-use-it--worked-examples)
7. [The interactive terminal](#7-the-interactive-terminal)
8. [Configuration](#8-configuration)
9. [Safety model](#9-safety-model)
10. [Where to go next](#10-where-to-go-next)

---

## 1. What you're installing

`ai-runtime-engine` is two things in one package:

- **A library** — `import { Runtime, AI } from 'ai-runtime-engine'` to embed the runtime in your own code.
- **A CLI** — the `ai-runtime` command, for an interactive terminal and one-shot runs.

The core idea: you configure whatever AI providers you have (cloud APIs, a local model, an internal
endpoint), and the runtime **routes each request to the best available model** and runs it — so you write
*what you want*, not *which provider to call*. Around that it adds modes, memory, skills, tools, planning,
orchestration, comparison, and learning.

Requirements: **Node.js 22+**.

---

## 2. Install

As a dependency in your project (gives you the library and `npx ai-runtime`):

```bash
npm install ai-runtime-engine
```

Or globally, to get the `ai-runtime` command on your PATH:

```bash
npm install -g ai-runtime-engine
ai-runtime --help
```

---

## 3. Your first run (offline, no keys)

You can run the whole pipeline with a **mock provider** — no credentials, no network. Create `first.mjs`:

```js
import { Runtime, MockProvider, makeModel } from 'ai-runtime-engine';

// A fresh runtime in stateless mode, so this demo writes nothing to disk.
// (Persistence is ON by default; pass { persistence: 'disabled' } to turn it off.)
const runtime = new Runtime(undefined, { persistence: 'disabled' });

// Register one fake provider with one fake model that can "reason" and output "text".
runtime.ai.registerProvider(new MockProvider({
  id: 'demo',
  models: [makeModel({
    providerId: 'demo',
    id: 'demo-1',
    caps: { intelligence: ['reasoning'], output: ['text'] },
  })],
}));

const result = await runtime.run({ input: 'Say hello and explain what you are.' });

console.log('ok:', result.ok);
console.log('text:', result.response?.text);
console.log('chose:', result.routing?.selected);        // which provider/model was picked
console.log('why:', result.routing?.ranked?.length, 'candidate(s) considered');
```

```bash
node first.mjs
```

You'll see `ok: true`, a mock response, and a **routing trace** showing the decision. That trace is always
there — every run tells you *what it did and why*.

---

## 4. How it works — the mental model

A single `run()` flows through one path. You supply the capabilities; the runtime decides how to use them.

```
runtime.run({ input })
   │
   ▼  1. Pick a MODE — auto-detected, or you set it: chat · plan · execute · orchestrate · agent · compare · debug
   │
   ▼  2. Build the CONTEXT under a token budget (a one-line workspace summary, relevant memory, your request)
   │
   ▼  3. ROUTE to a model  ──►  the ONE router:
   │        filter (capability · health · privacy · cost)  →  score  →  pick  →  execute  →  validate
   │        →  fall back to the next-best model on failure
   │
   ▼  4. For multi-step work: PLAN → run tools/skills → succeed a step only on EVIDENCE → replan on failure
   │
   ▼  5. RESULT { ok, response, routing trace, + plan / comparison / memory as relevant }
```

Five ideas are worth internalizing:

- **Providers are replaceable; capabilities are durable.** You route by *what a task needs* (reasoning,
  JSON output, a big context window…), not by a provider name. Swap providers freely.
- **One router.** *Every* model call — chat, planning, comparison, a skill's own call — goes through the
  same router. There is never a second, hidden selection engine.
- **`Runtime` vs `AI`.** `AI` is the low-level router (`ai.run({ task, input })`). `Runtime` wraps it and
  adds modes, memory, skills, orchestration, etc. (`runtime.run({ input })`). Use `Runtime`; reach for `AI`
  only when you want the bare router.
- **Modes = what kind of work.** A quick answer is `chat`; a multi-step job is `orchestrate`; comparing
  models is `compare`. `auto` (the default) guesses from your request.
- **Evidence over claims.** A step "succeeds" only when there's proof — a tool returned ok, a validator
  passed — never because a model said "done."

---

## 5. Using real providers

To use a real model, name a provider and put its key in `.env`. The **fastest** way is the CLI:

```bash
ai-runtime setup     # detects your project, writes .ai-runtime/config.yaml + .env.example (asks first)
ai-runtime doctor    # tells you which providers are configured and reachable
```

Then add a key to `.env`. Recommended starters:

| Provider | Type | Key env var | Get a key |
|---|---|---|---|
| Google Gemini | Cloud | `GEMINI_API_KEY` | https://aistudio.google.com/apikey |
| Groq | Cloud | `GROQ_API_KEY` | https://console.groq.com/keys |
| OpenRouter | Cloud (aggregator) | `OPENROUTER_API_KEY` | https://openrouter.ai/keys |
| Ollama | **Local, no key** | — | https://ollama.com |

A minimal config (env-var **names** only — never the value):

```yaml
# .ai-runtime/config.yaml
runtime:
  defaultMode: auto
providers:
  - id: gemini
    kind: gemini
    apiKeyEnv: GEMINI_API_KEY
  - id: ollama          # local; no key required
    kind: ollama
```

In code, real providers load from that config:

```js
import { Runtime } from 'ai-runtime-engine';

const runtime = await Runtime.load({ workspace: process.cwd() });   // reads config + .env
const r = await runtime.run({ input: 'Summarize what this project does.' });
console.log(r.response?.text);
```

---

## 6. How to use it — worked examples

The examples below use the offline mock so you can run them as-is; with real providers configured, drop the
`registerProvider` line and use `await Runtime.load()` instead.

```js
import { Runtime, MockProvider, makeModel } from 'ai-runtime-engine';

function demoRuntime() {
  const rt = new Runtime();
  rt.ai.registerProvider(new MockProvider({
    id: 'demo',
    models: [makeModel({ providerId: 'demo', id: 'demo-1', caps: { intelligence: ['reasoning', 'planning'], output: ['text', 'json'] } })],
  }));
  return rt;
}
```

### a. Chat — a single answer

```js
const rt = demoRuntime();
const r = await rt.run({ input: 'Explain what a race condition is.', mode: 'chat' });
console.log(r.response?.text);
```

### b. Ask for a plan (no execution)

```js
const r = await rt.run({ input: 'Migrate this project off deprecated APIs', mode: 'plan' });
console.log(r.plan);   // a structured plan; nothing is executed in `plan` mode
```

### c. Preview a multi-step task with dry-run (zero changes)

```js
const r = await rt.run({ input: 'Fix the failing tests', mode: 'orchestrate', dryRun: true });
console.log(r.response?.text);   // shows the plan + what it WOULD touch, with no mutations
```

To let it actually act (write files, run shell, etc.), you grant permissions in config — everything is
deny-by-default (see [Safety model](#9-safety-model)).

### d. Compare models

`compare` fans the same request across several models and reports where they agree, differ, and contradict
each other:

```js
const rt = new Runtime();
rt.ai.registerProvider(new MockProvider({ id: 'a', models: [makeModel({ providerId: 'a', id: 'a-1', caps: { intelligence: ['reasoning'], output: ['text'] } })], behavior: { kind: 'ok', text: 'use PostgreSQL' } }));
rt.ai.registerProvider(new MockProvider({ id: 'b', models: [makeModel({ providerId: 'b', id: 'b-1', caps: { intelligence: ['reasoning'], output: ['text'] } })], behavior: { kind: 'ok', text: 'use MongoDB' } }));

const cmp = await rt.compare({ goal: 'which database?', input: 'Recommend a database', pins: [{ provider: 'a' }, { provider: 'b' }] });
console.log(cmp.analysis.unanimous, cmp.analysis.differences);
```

If you supply a `validate` function (e.g. run the tests), the answer that *passes* wins — evidence beats a
judge model.

### e. Remember facts across runs

With persistence on (the default when you `Runtime.load()`), memory is durable and scoped:

```js
const rt = await Runtime.load();
rt.memory.remember({ text: 'This project uses PostgreSQL', scope: 'project' });
console.log(rt.memory.search('database'));   // finds the fact; it's injected into future chats
```

You can also just say it in chat: *"Remember that we deploy on Fridays."*

### f. Add your own skill (no code)

Drop a manifest into `.ai-runtime/skills/` and it's auto-loaded. A skill composes the built-in tools:

```yaml
# .ai-runtime/skills/read-readme.skill.yaml
id: read-readme
version: 1.0.0
name: Read README
description: read the project README
steps:
  - tool: filesystem
    input: { op: read, path: README.md }
```

Run `ai-runtime skills` to see it loaded, or `ai-runtime skills --discover` to find skill files elsewhere in
your repo (reported, not auto-loaded). Skills can also be shipped as npm packages — name them under
`runtime.skills.packages` in config. (The example below names `ai-runtime-developer-skills`, which is built
in-tree in this workspace and is **not published to npm** — resolve it from the workspace or a local path.)

```yaml
# .ai-runtime/config.yaml
runtime:
  skills:
    packages:
      - ai-runtime-developer-skills
```

### g. Steer which models get used

- **Exclude** = a hard filter (never selected). **Prefer** = a soft nudge. **Neither** = normal routing.

```js
const r = await rt.run({ input: 'anything', routing: { excludeProviders: ['a'], preferProviders: ['b'] } });
```

Or via environment variables (`AI_EXCLUDE_PROVIDERS`, `AI_PREFER_MODELS`, …). Learning can nudge preferences
from real outcomes, but **it can never override an exclusion**.

### h. Action capabilities (2.4.0+)

The runtime reasons about *what a task needs to be able to do*, not just which provider to call. Ask what
a goal would require, or list the curated action vocabulary:

```bash
ai-runtime capabilities --actions           # the curated capability catalog
ai-runtime capabilities "refactor the auth module"   # the actions this goal implies
```

Config (`runtime.capabilities`): `catalog` enriches the planner's catalog and is **ON by default since
3.0.0** (set `catalog: false` to remove it); `planning` (default OFF) derives a goal's required
capabilities before planning and reports gaps — it is advisory and never blocks a run or grants anything.
In the terminal, `/capabilities <goal>` does the same.

### i. MCP servers (2.5.0+)

Attach tools from external [MCP](https://modelcontextprotocol.io) servers. They are added locally (your
config file is never edited), credentials are named by env var (never inlined), and each server's tools
are permission-gated like any other:

```bash
ai-runtime mcp add docs --command "npx -y @some/docs-mcp" --token-env DOCS_TOKEN
ai-runtime mcp test docs        # connect, handshake, list tools, ping
ai-runtime mcp                  # list servers;  mcp show <id> for one in detail
```

```yaml
# .ai-runtime/config.yaml
mcp:
  servers:
    docs: { transport: stdio, command: "npx -y @some/docs-mcp", tokenEnv: DOCS_TOKEN }
permissions:
  mcp: { servers: { docs: read } }   # off | read | full
```

Then pass `mcp: true` on a run (needs a tool-calling model). Server-supplied text is treated as untrusted
data, never instructions. In the terminal: `/mcp` and `/mcp <id>`.

### j. Agents (2.7.0+)

A goal can delegate bounded, read-shaped sub-work to an agent (a third kind of plan step). Agents run
inside an explicit envelope — a ceiling of tools, permissions, and call/time budgets — and their findings
are admitted only after passing an output-contract and a validation gate.

```yaml
# .ai-runtime/config.yaml
runtime:
  agents:
    enabled: true       # default OFF
    decompose: true     # allow a goal to delegate to a derived, read-only agent nobody configured (default OFF)
```

A derived agent's permissions are a **ceiling, never a default**, and no model-authored string ever
becomes a tool id, objective, or permission (the `auto_` id prefix is reserved). In the terminal,
`/agents` lists agent tasks with progress and `/agents stop <id>` stops one.

### k. Execution groups & evaluate (3.2.0+)

Run many model tasks as one bounded, resumable unit — with bounded concurrency, per-task timeout, retries,
optional quorum, budgets, and cancellation. Every task runs through the one router; the Runtime provides the
*mechanics*, your code provides the *meaning* (roles, prompts, verdicts). A "Council" is entirely a caller
concept — the Runtime never sees it:

```ts
import { Runtime } from 'ai-runtime-engine';
const rt = await Runtime.load();

// Round 1 — fan out advisors. "advisor" exists only here, as an opaque taskId.
const advisors = await rt.executeGroup({
  groupId: 'council-42:advisors',
  tasks: COUNCIL.advisors.map((a) => ({ taskId: `advisor:${a.name}`, request: { task: 'chat', input: { text: a.prompt(question) }, provider: a.provider, model: a.model } })),
  concurrency: 3,
  timeoutMs: 60_000,
  retry: { maxAttempts: 2, backoffMs: 500 },
});

// Quorum: the Runtime signals; the CALLER decides to stop the stragglers.
const off = rt.on((e) => {
  if (e.type === 'execution_group.quorum_reached' && e.groupId === 'council-42:reviews') rt.cancelExecutionGroup(e.groupId);
});
const done = advisors.results.filter((r) => r.status === 'completed');
const reviews = await rt.executeGroup({ groupId: 'council-42:reviews', quorum: 2, tasks: done.map((r) => ({ taskId: `review:${r.taskId}`, request: COUNCIL.reviewRequest(question, r.value) })) });
off();

// A chairman is just one more run; verification is the generic evaluation primitive.
const chair = await rt.run({ input: COUNCIL.chairmanPrompt(question, done, reviews.results), mode: 'chat' });
const verdict = await rt.evaluate({ candidate: chair.response?.text ?? '', rubric: 'Addresses the question; reconciles ≥2 advisor positions; no contradiction.' });
```

Crashed mid-way? `rt.resumeExecutionGroup('council-42:advisors')` re-runs only the tasks that did not
complete. Config caps live under `runtime.executionGroups` (`maxConcurrency`, `defaultTimeoutMs`,
`defaultRetryAttempts`, `maxGroupTasks`) — a caller may tighten these, never relax them.

---

## 7. The interactive terminal

The most fun way to explore. From a project that installed the package (or globally):

```bash
ai-runtime            # opens the interactive terminal
```

Type a request in plain language, or use a slash command:

```
/help                 list everything
/info  /status        what's configured
/providers            providers + access state
/providers/all models  every provider's models
/mode plan            set the mode
/memory               list remembered facts
/skills discover      find skill files in the repo
/compare <request>    compare across providers
/dry-run              toggle "plan only, no changes"
/doctor  /cleanup     diagnostics · maintenance
/exit
```

For scripts and CI, the one-shot form runs a single request:

```bash
ai-runtime run "Explain this error: TypeError: x is not a function" --json
```

---

## 8. Configuration

Three places, and only these:

| Path | What | Commit? |
|---|---|---|
| `.ai-runtime/config.yaml` | Project config — settings + providers (env-var **names**). | ✅ yes |
| `.env` | The secret **values** for those env vars. | ❌ no (gitignored) |
| `~/.ai-runtime/` | Local state — conversations, memory, executions, cache, learning. | ❌ never |

Handy optional environment variables: `AI_DEFAULT_MODE`, `AI_CONTEXT_MAX_TOKENS`, `AI_MAX_COST_USD`,
`AI_MAX_CALLS`, `AI_MEMORY_ENABLED`, `AI_LEARNING_ENABLED`, `AI_EXCLUDE_*` / `AI_PREFER_*`,
`AI_*_RETENTION_DAYS`, `AI_RUNTIME_HOME`. All optional; unset means the default.

For the full router configuration (routing strategies, scoring weights, privacy posture, self-declared
providers, the `run()`/`AI` reference), see [router.md](router.md).

---

## 9. Safety model

The runtime is built to be safe to point at your real files and shell:

- **Deny-by-default permissions.** Reads are allowed and **jailed to your workspace**; writes, shell, git
  commit/push, and network are **off** until you grant them under `permissions:` in config. Destructive
  shell commands always ask.
- **Untrusted data.** Anything read from files, tools, the web, or MCP is treated as *data, never
  instructions* — it can't change permissions, obtain secrets, or run commands.
- **Secrets** are named by env var only; an inline key is rejected, and every value is redacted from all
  output (including the terminal).

Details in [security.md](security.md).

---

## 10. Where to go next

- **[../README.md](../README.md)** — the concise reference (install, CLI, config at a glance).
- **[router.md](router.md)** — the engine: routing strategies, scoring, providers, the `run()`/`AI` API.
- **[architecture.md](architecture.md)** — how the router and Runtime layers fit together.
- **[security.md](security.md)** — the full security model.

Happy building. Remember the through-line: **describe what you want; the runtime picks the model and proves
the result.**
