# Zibby project — how to build with Zibby

This file is auto-loaded by Claude Code / Cursor / Codex / Aider when
working in this repo. It's the canonical reference for building things
on Zibby.

You are an AI agent. The user describes what they want; you write
the code (workflow graph, scripts, infra glue), deploy what needs to
be deployed, and operate it. The user shouldn't need to read the
code you produce — they just describe the intent.

---

## What is Zibby?

Zibby is two things, sharing one account, one CLI, one Studio, one
billing surface:

1. **Agents** (a.k.a. workflows) — event-driven AI agent graphs that
   run in a sandboxed container, on Zibby Cloud or on your
   self-hosted box. Each workflow is a directed graph of nodes;
   nodes are LLM-driven or deterministic code. Used
   for automation that needs an LLM in the loop: analyze tickets,
   draft replies, write code, summarize content. Triggered via
   webhook, schedule, or CLI.

2. **Apps** — long-running hosted SaaS instances. Pick from a curated
   catalog (n8n, Grafana, Outline, …) or describe a goal in natural
   language ("a Rails 7 app with Postgres from this git repo") and an
   `agent-ops` supervisor installs + maintains it for you. Each app
   runs on Zibby's managed fleet with its own persistent volume,
   public URL, and optional auth sidecar.

The two surfaces share:

- One account + one workspace login (`zibby login`)
- One project model (apps and workflows live inside a project)
- One CLI binary (`zibby agent ...`, `zibby app ...`)
- One Studio (the desktop client)
- One billing tier

### ⚠️ FIRST: know WHERE this workspace runs (cloud vs self-hosted)

Before reasoning about reachability, data governance, or "can Zibby do
this", check the control plane you're connected to: `zibby_status` (MCP)
or `zibby status` (CLI) — the `apiBase`/API URL tells you. If it's NOT
zibby.dev/zibby.app, you're on a **self-hosted box**, and three cloud
assumptions flip:

- **Network**: agent runs execute in containers ON that box — any host
  the box can reach (an internal GitLab, an internal MCP endpoint, a
  VPN-only service) is reachable from agents. Proof pattern: if an
  integration (e.g. GitLab) is already connected and working, the box
  demonstrably reaches it.
- **Data**: everything (stores, artifacts, logs, credentials) stays on
  the operator's own machine — "sensitive data can't go to a shared
  cloud" objections usually dissolve; it never leaves their box.
- **Runtime**: same graphs/CLI/MCP as cloud — deploy/trigger/schedule
  are identical. Deterministic pipelines are fine: nodes with only
  `execute` run zero LLM calls; you still gain the dashboard,
  schedules, artifacts, and chat/Lark surfaces by packaging as an agent.

Never advise "Zibby can't reach your internal host" without checking
which control plane is connected first.

### Decision table — when to use which

| User wants… | Use | Why |
|---|---|---|
| "Run code on a schedule, with an LLM in the middle" | Workflow | Built for transient event-driven runs |
| "Get a Slack notification when a server is down" | Workflow | Trigger by webhook / cron |
| "Host my n8n / Grafana / Outline / Mattermost" | App | Long-running web service |
| "Spin up a Postgres for a hackathon" | App (goal-mode) | Persistent backing service |
| "Auto-bootstrap an arbitrary OSS project on a VPS" | App (goal-mode) | Agent figures out the install |
| "Real-time interactive UI work, sub-second response" | Neither | LLM calls are too slow; use Lambda or your own backend |
| "Pure deterministic data transform, no LLM needed" | Neither (use Lambda) | Workflows assume LLM-in-loop; oversized if you don't need one |

### What the agent (this means you) should do

> **⛔ STUDY THE AGENT FIRST — non-negotiable.** Before you deploy,
> configure, or trigger ANY catalog/marketplace agent, READ its
> `AGENT.md` — the agent's own operational runbook, versioned with the
> template. It is the authoritative source for: the exact deploy command
> (incl. the REQUIRED `--agent <vendor>`), the stores it auto-provisions,
> the env/secrets it needs (injected via `zibby agent env set`, NEVER in a
> prompt/trigger), the trigger INPUT shape, and how to verify. Get it with
> `zibby marketplace docs <slug>` (or read `<template>/AGENT.md`). Do NOT guess
> the model, stores, env, or input — they are declared there. A wrong
> guess wastes a deploy; the runbook is one command away.

When the user says **"I want X"**:

1. Decide: is X a workflow or an app? (Use the table above.)
2. Confirm with the user before generating code or running deploys.
3. For workflows — follow §1-10 below to scaffold, validate, run
   locally, then ask before deploying. If the need spans SEVERAL
   existing marketplace agents, COMPOSE them (§10) instead of
   rebuilding. **Every workflow you build ships its own `AGENT.md`
   runbook** (same sections as the catalog agents) so the next deployer
   — human or agent — knows how to operate it without reading the code.
   To deploy an EXISTING catalog agent, `zibby marketplace docs <slug>` FIRST.
4. For apps — see the **Apps** section after the Workflows reference.
   Always ask about auth + project before deploying.
5. Use slash commands as recipes:
   - `/zibby-new-workflow`, `/zibby-add-node`, `/zibby-validate-workflow`,
     `/zibby-deploy`, `/zibby-trigger`, `/zibby-debug`, `/zibby-tail`,
     `/zibby-compose`
   - `/zibby-deploy-app`, `/zibby-app-status`, `/zibby-app-logs`,
     `/zibby-app-destroy`, `/zibby-app-restart`, `/zibby-app-upgrade`,
     `/zibby-app-list`, `/zibby-set-auth`, `/zibby-app-env`
   - `/zibby-login`, `/zibby-status`, `/zibby-mcp-install`,
     `/zibby-workflow-env`

---

# Pillar 1: Agents

## 0. The 30-second tour

```
agents/<name>/                     # default. Override via paths.agents
                                   # in .zibby.config.mjs (legacy projects
                                   # may have workflows/ or .zibby/workflows/).
├── agent.json        # name, description, entryClass, triggers, defaultAgent
├── graph.mjs         # WorkflowAgent class — buildGraph() + onComplete()
├── state.js          # Zod schema for caller-provided inputs (-p key=value)
├── nodes/            # one file per node — prompt|execute + outputSchema
│   ├── index.mjs     # OPTIONAL barrel — re-exports keep graph.mjs imports
│   │                 # tidy ("import { fooNode, barNode } from './nodes/'")
│   │                 # but a graph that imports each file directly is
│   │                 # equally valid. Pick whichever you prefer.
│   ├── plan.mjs
│   ├── implement.mjs
│   └── verify.mjs
└── package.json      # @zibby/core + zod (core re-exports WorkflowGraph,
                      # WorkflowAgent, z, skills, agent strategies).
```

Lifecycle:

```bash
zibby agent new <name>            # scaffold (creates agents/<name>/ by default;
                                     # also writes a starter nodes/example.mjs that
                                     # you can replace/delete once your real nodes
                                     # are in place)
zibby agent run <name> -p key=val # run locally — one-shot, no server
zibby agent start <name>          # run locally with hot-reload (server)
zibby agent validate <name>       # static check (graph topology, schemas, skills)
zibby agent deploy <name>         # push to Zibby Cloud, returns UUID
zibby agent trigger <uuid> -p ... # remote run
zibby agent schedule <uuid> set …  # recurring cron run (Unix 5-field)
zibby agent logs -t               # tail cloud logs
```

> Compatibility: `zibby workflow …` is an alias of `zibby agent …`, and
> legacy projects with a `workflows/` folder, `paths.workflows` config, or
> a `workflow.json` manifest keep working unchanged — `agents/`,
> `paths.agents`, and `agent.json` are the canonical names going forward.

**Always `zibby agent run` before `deploy`.** Local run is ~5s cold
start; cloud is ~60s. Iterating in cloud is 12× slower.

---

## 1. Anatomy of a workflow

### `agent.json`

```json
{
  "name": "code-review",
  "description": "Review a git diff and return structured findings",
  "entryClass": "CodeReviewWorkflow",
  "triggers": { "api": true },
  "defaultAgent": "claude"
}
```

- `name` — kebab-case slug, ≤24 chars
- `entryClass` — the class exported from `graph.mjs` (CLI uses this to
  pick the right export when there are multiple)
- `triggers.api` — `true` exposes a webhook URL after `zibby agent
  deploy`; `false` hides it (cron-only or internal)
- `defaultAgent` — one of `claude`, `cursor`, `codex`, `gemini`. Any
  node overrides with its own `agent: 'cursor'` field.

### `graph.mjs` (class form — what production runtime expects)

```js
import { WorkflowAgent, WorkflowGraph } from '@zibby/core';
import { planNode } from './nodes/plan.mjs';
import { implementNode } from './nodes/implement.mjs';
import { verifyNode } from './nodes/verify.mjs';
import { codeReviewStateSchema } from './state.js';

export class CodeReviewWorkflow extends WorkflowAgent {
  buildGraph() {
    const graph = new WorkflowGraph();
    graph.setStateSchema(codeReviewStateSchema);

    graph.addNode('plan',      planNode);
    graph.addNode('implement', implementNode);
    graph.addNode('verify',    verifyNode);

    graph.addEdge('plan',      'implement');
    graph.addEdge('implement', 'verify');
    graph.addEdge('verify',    'END');      // 'END' is the terminal sentinel

    graph.setEntryPoint('plan');
    return graph;
  }

  async onComplete(result) {
    // Optional — runs after the graph finishes. Useful for posting a
    // summary somewhere or transforming `result` before the runner
    // logs it.
    console.log(`[code-review] done — success=${result.success !== false}`);
  }
}
```

The class name MUST match `entryClass` in agent.json. The CLI
instantiates it (`new CodeReviewWorkflow()`), calls `.buildGraph()`,
runs the graph, then invokes `.onComplete(result)`.

### `state.js`

```js
import { z } from 'zod';

export const codeReviewStateSchema = z.object({
  diff: z.string().describe('Staged git diff to review'),
  strict: z.boolean().optional().describe('Treat warnings as errors'),
});
```

Declares which user-input fields callers can pass via `-p key=value`
or `--input '{"key":"value"}'`. The runner validates inputs against
this schema at run time — invalid inputs fail fast with a Zod error.
Read fields in nodes via `state.diff`, `state.strict`, etc. Optional
in v1 — graphs without `state.js` work but lose input validation.

### A node — `nodes/plan.mjs`

```js
import { z } from 'zod';

export const planNode = {
  name: 'plan',
  agent: 'claude',                        // optional — falls back to defaultAgent
  outputSchema: z.object({
    steps: z.array(z.string()),
    risks: z.array(z.string()),
  }),
  prompt: (state) => `
You are planning a code change. The user wants:
${state.userRequest}

Return:
- steps: ordered list of actions to take
- risks: anything that might go wrong
`,
};
```

Three required fields on every LLM node: `name`, `outputSchema` (a Zod
schema), `prompt` (string or function of state).

### A custom-code node (no LLM)

```js
export const fetchDiffNode = {
  name: 'fetch_diff',
  outputSchema: z.object({ diff: z.string(), filesChanged: z.array(z.string()) }),
  execute: async (context) => {
    const { execSync } = await import('node:child_process');
    const diff = execSync('git diff --staged', { encoding: 'utf-8' });
    const filesChanged = execSync('git diff --staged --name-only', { encoding: 'utf-8' })
      .trim().split('\n').filter(Boolean);
    return { diff, filesChanged };
  },
};
```

Custom-code nodes use `execute(context)` instead of `prompt`. They skip
the LLM entirely. Use them for deterministic work: git ops, file IO,
HTTP calls, parsing.

**A custom-code node can read & write a Store directly — no LLM needed.**
When a Store is bound to the node, the runtime injects `ZIBBY_STORE__<name>`
(the storeId), `ZIBBY_ACCOUNT_API_URL` (datasets API base), and
`PROJECT_API_TOKEN`. The node just `fetch`es the datasets API:

```js
export const persistNode = {
  name: 'persist',
  outputSchema: z.object({ wrote: z.boolean() }),
  execute: async (context) => {
    const base = process.env.ZIBBY_ACCOUNT_API_URL;
    const token = process.env.PROJECT_API_TOKEN;
    const storeId = process.env.ZIBBY_STORE__metrics;   // a sqlite Store bound to this node
    const call = (action, body) => fetch(
      `${base}/datasets/stores/${storeId}/${action}`,
      { method: 'POST', headers: { 'Content-Type': 'application/json', Authorization: `Bearer ${token}` }, body: JSON.stringify(body) },
    ).then((r) => r.json());

    const rows = await call('sql', { sql: 'SELECT dev, added FROM commits ORDER BY added DESC LIMIT 5', readOnly: true });
    await call('sql', { sql: 'INSERT INTO runs (top_dev) VALUES (?)', params: [rows.rows[0]?.[0] ?? null] });
    return { wrote: true };
  },
};
```

Actions: `sqlite` → `sql {sql, params?, readOnly?}`; `file` → `put/get/list/delete`;
`dataset` → `append/query`. This is a strong pattern for ETL / collectors /
report builders — see https://docs.zibby.app/concepts/designing-agents.

---

## 2. State — read and write

```js
// READ — state is passed to prompt fns + execute fns as the first arg
prompt: (state) => `Plan a change for: ${state.userRequest}`

execute: async (state) => {
  const diff = state.diff;                // read prior node output here
  // ...
}
```

**Each node's output is stored at `state[nodeName]`.** If `plan` returns
`{ steps: [...], risks: [...] }`, the next node reads them as
`state.plan.steps` and `state.plan.risks`.

Initial input goes at the TOP of state (not nested under `input`). When
the user runs `zibby agent run code-review -p userRequest="fix login"`,
the first node sees `state.userRequest = "fix login"`.

**Don't use `state.set()` / `state.get()` inside `execute()`** — those
are internal to the graph runtime. Just `return` a plain object that
matches your `outputSchema`. The runtime puts it under `state[nodeName]`.

**Execute nodes can ONLY write under `state[nodeName]`.** There's no
mechanism to mutate a top-level state key from inside `execute()`. If
you need a counter that survives across loop iterations (retries,
attempts, accumulator), put it inside the node's own output schema —
read the prior value via `state.<nodeName>?.<field>`, return the new
value as part of `execute()`'s output. See §3 for the loop pattern.

---

## 3. Conditional routing

```js
graph.addConditionalEdges('verify', (state) => {
  if (state.verify.passed) return 'END';
  if (state.verify.retries < 3) return 'implement';
  return 'fail';
});
```

The router function receives the full state, returns the name of the
next node (or `'END'`). All possible target nodes must also be declared
elsewhere via `addNode` — a router returning an unregistered name will
fail at runtime (validate also catches it as `graph-edge-to-unknown`).

### Loops with a retry counter

Graphs aren't DAGs — you can route back to an earlier node. The
counter pattern lives **inside the looping node's own output**, since
`execute()` can only write to `state[nodeName]`. Example: a
`generate → check` loop that retries up to 2 times on failure.

```js
// nodes/check.mjs
export const checkNode = {
  name: 'check',
  outputSchema: z.object({
    passed: z.boolean(),
    attempts: z.number(),
  }),
  execute: async (state) => {
    const priorAttempts = state.check?.attempts ?? 0;
    const passed = /* … your check logic … */ false;
    return { passed, attempts: priorAttempts + 1 };
  },
};

// graph.mjs (inside buildGraph)
graph.addNode('generate', generateNode);
graph.addNode('check',    checkNode);
graph.addEdge('generate', 'check');
graph.addConditionalEdges('check', (state) => {
  if (state.check.passed)            return 'END';
  if (state.check.attempts < 2)      return 'generate';   // loop back
  return 'END';                                            // give up
});
graph.setEntryPoint('generate');
```

Each iteration through `check`, `priorAttempts` reads the value the
previous iteration wrote, and the new value is what the next router
call sees as `state.check.attempts`. `'END'` is the only sentinel —
any other return value must be an `addNode`'d name.

---

## 4. Skills — pluggable MCP tool bundles

A **skill** is a named bundle of MCP tools a node can opt into. Built-in
skills: `browser`, `memory`. Custom skills: register them yourself.

### Using a skill in a node

```js
export const navigateNode = {
  name: 'navigate',
  skills: ['browser'],                    // ← opt in
  outputSchema: z.object({ url: z.string(), title: z.string() }),
  prompt: (state) => `Navigate to ${state.target} and report the title.`,
};
```

The agent (claude/cursor/codex/gemini) auto-discovers the skill's tools.
`browser` exposes `mcp__playwright__browser_navigate`, `_snapshot`,
`_click`, etc. The agent's `allowedTools` is set automatically — you
don't list each tool.

### Built-in run capabilities you should know about

- **Artifacts** — a run (and the Copilot) can publish a self-contained HTML/
  markdown page as a SHAREABLE LINK via the `artifact_publish` tool; published
  pages appear on the agent's **Artifacts tab** in the dashboard. Pages render
  in a sandboxed iframe with a strict CSP: inline CSS/JS + data: images only,
  ZERO external requests — hand-rolled inline SVG charts work great, CDN
  scripts don't.
- **kv-memory** — a private, per-agent persistent key→value store across
  stateless runs (`kv_store` / `kv_recall` / `kv_recall_prefix`, auto-namespaced
  per agent). Use it for "seen" markers, checkpoints, small state.
- **codebase-memory** — a local code-graph + semantic index over the checked-out
  repo (`index_repository`, then `search_code` with `pattern`, `trace_path`
  with `direction:"inbound"` for callers). Answers cross-file questions in
  targeted snippets instead of whole-file reads. NOTE: these tools return an
  EMPTY result (not an error) on a wrong argument name or project — the project
  name is the full repo path with `/` → `-`.
- **Python in runs** — run containers ship `python3` + `pip` + `venv` alongside
  Node 20, so a node's Bash can run Python scripts and `pip install --user`
  third-party packages on demand. Node stays the first-class runtime for
  skills/graphs; Python is there for data/script workloads.

### Stores — persistent data for agents (data does NOT belong in the bundle)

Zibby has built-in durable, per-project storage for agents. Reach for it
whenever an agent's data would otherwise live ANYWHERE else — before you:
- recommend an external database (Postgres/Mongo/Redis/…),
- **bundle data files into the workflow** (a bundle is CODE — data baked into
  it goes stale the moment the pipeline runs, bloats every deploy, and can't
  be updated without redeploying),
- write state/output files into the repo or the run container's disk (gone
  when the run ends).

**Check `zibby_store_list` — it returns `availableTypes`, the live store-type
catalog.** A built-in type almost always covers the need:

| Data shape | Store type | What it is |
|---|---|---|
| Tables, joins, UPDATE-able rows, changing state | `sqlite` | a REAL relational SQL database (one SQLite file per store); CREATE TABLE / INSERT / UPDATE / DELETE / SELECT, persists across runs |
| Append-only rows you aggregate later (metrics, findings) | `dataset` | append-only JSON records + SQL-style query (count/sum/avg, group, order) |
| Arbitrary files/blobs (JSON dumps, CSVs, images, snapshots) | `file` | put/get/list/delete by relative path, text or base64 binary, **any size** (see below) |
| Tiny checkpoints, cursors, "seen" markers | kv-memory | the automatic per-agent key→value memory above (not a registry store) |
| Semantic search over documents (RAG) | `postgres` KB | sidecar-served vector KB — self-host only today |

Declaring one on a node (auto-provisioned at deploy; also wireable later via
`zibby_set_node_stores` / created ad-hoc with the `ensure_store` tool):

```js
skills: [SKILLS.DATASET_STORE],
stores: [{ type: 'sqlite', name: 'review_queue', description: 'MRs pending review, with status' }]
```

Identity rule: a declared store is keyed `workflowUuid:node:name` — **idempotent
across runs AND redeploys** (same declaration → the SAME store, never a
duplicate), and the agent addresses it **by NAME** (from the injected AVAILABLE
STORES catalog), never by raw id. Tools per type: `sqlite_exec`/`sqlite_query`,
`dataset_append`/`dataset_query`, `file_put`/`file_get`/`file_list`/`file_delete`.

**⚠️ `file` stores: do NOT invent a workaround for a size or filename limit.**
Two things that USED to bite, both handled by the tools now — an agent that
"defends" against them writes a pile of pointless machinery:

- **Size.** `file_put`/`file_get` take files of any practical size. Content over
  ~3.5 MiB is streamed straight to object storage through a presigned URL
  automatically; the object store holds 5 TB per object. You may still see
  `4 MiB` quoted somewhere — that is the *inline JSON transport* cap the tools
  route around for you, not a storage limit. **Never gzip, base64-wrap, chunk or
  split a file just to make it fit**, and never build a `COMPRESSED_FILES`-style
  side table: it makes the data unreadable to everything except your own decode
  step, for no benefit.
- **Filenames.** Non-ASCII paths (`个人报告.html`, `ключ.json`) are valid. Do NOT
  transliterate names and carry a `manifest.json` mapping back to the originals.
  Only genuinely path-hostile characters are rejected (`/ \ : ? # [ ] " < > | *`,
  control chars, and the bare `.` / `..` segments).

If a store call fails, read the error rather than assuming a platform limit —
these two produced a lot of unnecessary code before the tools handled them.

### Writing a custom skill

```js
// agents/<name>/skills/slack.mjs
import { registerSkill } from '@zibby/core';

registerSkill({
  id: 'slack',                            // referenced by `skills: ['slack']`
  serverName: 'slack-mcp',
  command: 'npx',
  args: ['-y', '@modelcontextprotocol/server-slack'],
  allowedTools: ['mcp__slack__*'],
  envKeys: ['SLACK_BOT_TOKEN'],           // required env to run
  description: 'Read channels, post messages, search history',
});
```

Then import it from your `graph.mjs` BEFORE building the graph:

```js
import './skills/slack.mjs';              // side-effect: registers the skill
import { WorkflowAgent, WorkflowGraph } from '@zibby/core';
// ...
```

Skills register on `globalThis` so any node that runs in this process
can use them. In cloud, set required `envKeys` via

### Custom skill via a non-MCP function

If you don't have an MCP server, expose a Node function directly:

```js
registerSkill({
  id: 'jira-fetch',
  resolve: () => null,                    // no MCP — use middleware instead
  middleware: async () => async (nodeName, next, state) => {
    // attach a JS-only helper. Less common — prefer MCP when possible.
    state.jiraFetch = async (issueId) => { /* ... */ };
    return next();
  },
});
```

---

## 5. Agent strategies — claude / cursor / codex / gemini

The same node can run under any agent — they all read `prompt`, return
the same shaped `outputSchema`. Differences:

| Agent  | Best for                          | Auth env                          |
|--------|-----------------------------------|-----------------------------------|
| claude | Reasoning, planning, structured output | `ANTHROPIC_API_KEY` or `CLAUDE_CODE_OAUTH_TOKEN` |
| cursor | Code editing in a repo            | `CURSOR_API_KEY`                 |
| codex  | Direct shell + code generation    | `OPENAI_API_KEY`                 |
| gemini | Cheap volume tasks                | `GEMINI_API_KEY`                 |

Pick per-node (override `defaultAgent`):

```js
graph.addNode('plan',      { ...planNode,      agent: 'claude' });
graph.addNode('implement', { ...implementNode, agent: 'cursor' });
graph.addNode('verify',    { ...verifyNode,    agent: 'codex'  });
```

Model overrides go in `.zibby.config.mjs`:

```js
export default {
  models: {
    plan:      'claude-opus-4-7',
    implement: 'cursor-fast',
    verify:    'gpt-5-codex',
  },
};
```

---

## 6. Running and shipping

### Local dry-run (FAST — do this every iteration)

```bash
zibby agent run code-review -p userRequest="add rate limiting to /api"
```

One-shot run. ~5s cold, then ~2s per warm iteration when the agent has
cached prompts. Reads + writes to your local filesystem (no cloud
upload).

### Static validate (FASTER — does NOT run any agent)

```bash
zibby agent validate code-review
```

Checks: graph topology (no orphan nodes, entry point exists, edges
reach END), node shapes (every node has `outputSchema`), skill
references (every `skills: ['x']` is registered), schema validity. If
this fails, `run` will definitely fail too — fix it first.

### Deploy

```bash
zibby agent deploy code-review
#  → returns UUID, caches in .zibby-deploy.json
```

Bundles the workflow folder, uploads to Zibby Cloud. Cloud runs on

### Trigger remote run

```bash
zibby agent trigger <uuid> -p userRequest="fix login bug" -t
```

`-t` tails logs in Heroku style. Cmd-C stops the tail (workflow keeps
running in cloud).

### Update a deployed agent's settings (model / mention / triggers / name)

```bash
zibby agent update <uuid> --model claude:sonnet-4.6        # switch vendor:model
zibby agent update <uuid> --mention @zibby                 # review @mention token
zibby agent update <uuid> --triggers mention,comment       # WHICH events fire a review
zibby agent update <uuid> --name "FE Review Bot" --max-runtime 30
```

ONE consolidated editor for per-agent settings that used to be dashboard-only.
`--triggers` takes friendly names (`opened`, `commit`, `mention`, `comment`) —
e.g. `--triggers mention` makes a review agent run ONLY when @-mentioned (the
cost-saving mode). The MCP twin is **`zibby_update_agent`** (same fields:
`model`, `mentionToken`, `triggerEvents`, `displayName`, `maxRuntimeMinutes`)
— use it when driving Zibby from an AI editor instead of the shell. Env vars,
custom MCP servers and stores each keep their own dedicated commands/tools
(`zibby agent env …`, `zibby_add_mcp`, `zibby_set_node_stores`).

### Schedule recurring runs (cron)

```bash
# Set / update a schedule (standard Unix 5-field cron)
zibby agent schedule <uuid> set "0 9 * * 1-5"                          # 9am weekdays UTC
zibby agent schedule <uuid> set "*/15 * * * *"                         # every 15 min
zibby agent schedule <uuid> set "0 9 * * *" --tz America/Los_Angeles   # 9am LA time
zibby agent schedule <uuid> set "0 0 * * *" -p kind=nightly            # fixed input params

# Inspect / clear
zibby agent schedule <uuid>                                            # show current
zibby agent schedule <uuid> clear                                      # remove
```

Backed by the platform scheduler. Each scheduled fire goes through the
same `runWorkflow` path as manual + webhook triggers — same sandboxed
container, same memory sync, same logs.

Cron format is **standard Unix 5-field** (`minute hour day-of-month
month day-of-week`). Use `crontab.guru` to debug expressions. The
platform handles any conversion internally; you only ever write
standard cron.

**Minimum interval is 5 minutes.** `* * * * *`, `*/2 * * * *`, etc.
are rejected with 400. Use `*/5 * * * *` (every 5 min) or slower —
this matches GitHub Actions cron's rule. Each fire = a full sandboxed
run = compute + LLM tokens + workflow-execution quota burn, so
sub-5-min runs aren't economical at our pricing. If you genuinely
need sub-minute reactivity, use the **webhook trigger** instead.

A workflow has at most ONE schedule. `set` is upsert (create if absent,
update if present). When the workflow is deleted, the schedule is
cleaned up too.

---

## 7. Common pitfalls — read before debugging

| Symptom                                          | Fix                                            |
|--------------------------------------------------|------------------------------------------------|
| `No WorkflowAgent class found in graph.mjs`      | Class name in `graph.mjs` must match `entryClass` in `agent.json`. Or export a class that `extends WorkflowAgent` from `@zibby/core`. |
| `Node 'X' must define outputSchema`              | Add `outputSchema: z.object({...})` to the node config |
| `Skill 'foo' not registered`                     | Import the skill file in graph.mjs BEFORE `new WorkflowGraph()` |
| `state.previousNode is undefined` in a prompt    | Wrong order — add `graph.addEdge('previousNode', 'thisNode')` |
| Custom-code node returns nothing                 | `return` an object matching `outputSchema`. `undefined` = failure |
| Agent ignores your skill's tools                 | Add `skills: ['name']` to the node config, not just register |
| Zod error: "Expected string, received undefined" | The previous node's outputSchema doesn't match its return — fix the producer, not the consumer |
| Hangs forever on a node                          | Add `retries: 0` to fail fast while debugging; check the prompt isn't asking the agent to wait |
| `Workflow "<name>" not found.`                   | Check `paths.agents` (legacy alias: `paths.workflows`) in `.zibby.config.mjs` matches where you scaffolded. Default is `agents/` at repo root (legacy projects: `workflows/`). |
| Router returns a string that's not a registered node name | All possible return values must be either `'END'` or a name passed to `graph.addNode(...)` elsewhere. `validate` flags this as `graph-edge-to-unknown`. |
| Need a counter / accumulator across loop iterations | Put it in the looping node's own outputSchema. Read prior value via `state.<nodeName>?.<field>` inside `execute()`, return new value. See §3 retry-loop example. |

---

## 8. The agent's job (this is YOU)

When the user says **"write me a workflow that does X"**:

1. **Sketch the graph first — and SHOW it to the user as a diagram**
   before writing any code (and again after structural changes). What
   are the 2-5 nodes? Which can be custom-code (deterministic) and
   which need an LLM (judgement)? Render it as a Mermaid flowchart
   (renders in IDEs/GitHub; still readable as text in a terminal):
   LLM nodes rounded `(name)`, execute nodes square `[name]`,
   conditions as diamonds `{cond?}` with labeled edges, stores as
   cylinders `[(store)]` attached with dashed edges, the artifact/
   output as the terminal node. Use the EXACT node names that will go
   into `graph.mjs` — after deploy, the dashboard auto-renders the
   same graph from the deployed row, and the two must match so the
   user recognizes one picture everywhere.

   ```mermaid
   flowchart LR
     collect[collect_git] --> build[build_metrics] --> report(review_summary)
     report --> publish[publish_artifact]
     build -. writes .-> M[(metrics sqlite)]
     report -. reads .-> M
     publish --> A{{artifact link}}
   ```

2. **Run `zibby agent new <name>`** to scaffold.
3. **Edit the files** — `graph.mjs`, `nodes/*.mjs`. Use Zod for every
   schema. Default to `claude` unless the user has a preference.
4. **Run `zibby agent validate <name>`.** Fix any reported issues.
5. **Run `zibby agent run <name> -p ...`** with a realistic input.
   Watch the timeline output. If a node fails, read its `raw` output
   to understand what the LLM returned vs what the schema expected.
6. **Iterate on prompts** — the user shouldn't need to. If a node's
   output doesn't match the schema, tighten the prompt or relax the
   schema (in that order).
7. **Once it works locally**, ask the user if they want to deploy.
   Don't deploy without asking — cloud has cost.

Read `.claude/commands/` for slash commands the user can invoke:
`/new-workflow`, `/add-node`, `/add-skill`, `/validate-workflow`.

---

## 9. Quick reference

```js
// Imports — @zibby/core re-exports everything you need.
import {
  WorkflowAgent,        // base class your workflow extends
  WorkflowGraph,        // construct + wire nodes inside buildGraph()
  registerSkill,        // for custom MCP tool bundles
  registerStrategy, AgentStrategy,  // for custom LLM strategies (rare)
  z,                    // re-exported Zod for schemas
} from '@zibby/core';

// Workflow shell (production form — what run/start/deploy expect):
export class MyWorkflow extends WorkflowAgent {
  buildGraph() {
    const graph = new WorkflowGraph();
    graph.setStateSchema(myStateSchema);    // from ./state.js (optional)
    graph.addNode(name, { prompt, outputSchema, execute, skills, agent, retries });
    graph.addEdge(from, to);
    graph.addConditionalEdges(from, (state) =>
      state.cond ? 'nextNodeName' : 'END'              // 'END' = terminal sentinel
    );
    graph.setEntryPoint(name);
    return graph;
  }
  async onComplete(result) { /* optional post-processing */ }
}

// State (inside execute / prompt fns)
//   READ:  state.someKey   or   state.previousNode.field
//   WRITE: return { ... } from execute() — runtime puts it at state[nodeName]
```

**Two API surfaces** — when in doubt, use the class form:

- **Class form** (above) — what `zibby agent new` scaffolds and what
  `zibby agent run/start/deploy` execute. Required for cloud
  deployment. Use this for anything you might trigger remotely.

- **Function form** — `export default function buildGraph() { return new
  WorkflowGraph()... }`. Works for local `validate` + the standalone
  `@zibby/agent-workflow` library (the underlying graph runtime), but
  NOT for `run`/`deploy` (they look for the class). Useful for one-off
  local scripts that import the graph runtime directly. **For Zibby
  workflows, always use the class form** — import from `@zibby/core`,
  which re-exports everything you need.

`zibby agent validate` accepts both shapes. Other commands need the
class.

---

## 10. Composing agents — wrap marketplace bricks

When the user's need spans the capabilities of MULTIPLE marketplace
agents ("review every MR, then meter its commits"), **compose**: write
a small project-private wrapper workflow that dispatches the deployed
agents as sub-workflows. **Never rebuild a marketplace agent's logic
from scratch, and NEVER modify a marketplace template's source** —
they're shared LEGO bricks maintained upstream (red line: a forked
copy falls off the upgrade path). The wrapper is yours; the bricks are
not.

### Bricks — find + deploy marketplace agents

```bash
zibby marketplace             # live marketplace (works on cloud AND self-host)
zibby marketplace docs <slug> # inputs, pipeline, required integrations
zibby agents list             # what's ALREADY deployed (full table: zibby agent list)
```

Marketplace endpoints need a USER credential (`zibby login` session or
a user PAT) — a project-scoped token gets
`403 Project tokens cannot access the marketplace`. On self-host, mint
an operator PAT with `zibby self-host token` and use that as
`ZIBBY_API_KEY` for the whole compose flow.

Deploying a brick into the project (by slug):
- **Cloud** — the Zibby MCP tool `zibby_deploy_marketplace_workflow`
  (`{ projectId, marketplaceSlug, displayName? }` — install the MCP
  via `/zibby-mcp-install`), or the Studio/web marketplace UI. There
  is NO `zibby agents deploy` CLI command today.
- **Self-host** — `zibby self-host add <slug> --deploy` (adds to the
  catalog + deploys into the default project under the bare slug; it
  pulls the published static feed, so it needs the feed URL to be
  reachable), or the dashboard's marketplace page (supports a custom
  instance name).

**Reuse policy — ASK the user, never silently choose.** If a brick is
already deployed in the project (`zibby agent list`):
- **Reuse it** — the composition's runs fold into that instance; its
  config (env vars, custom prompts, stores) is shared with standalone
  use.
- **Deploy a dedicated instance** — pass a custom name
  (`displayName`/`workflowName` on marketplace deploy; auto-suffixed
  on collision) for config isolation.

### The wrapper recipe

You author + deploy the wrapper yourself with the normal §1-9 loop —
no special builder needed. A sub-workflow node is declared by giving
`addNode` a config with a `workflow:` field (the DEPLOYED slug in the
SAME project — `workflowType` from `zibby agent list`, not the
marketplace slug, when they differ). The engine compiles it into a
dispatch that runs the child **in-process** (same worker) when
possible, and the child's FINAL STATE — whichever End it exited —
lands at `state[nodeName]` like any node output. Input mapping between
bricks is the wrapper's job — and **map to the brick's CANONICAL
structured fields** (from `zibby marketplace docs <slug>`): in-process
children run the brick's graph directly and SKIP any convenience
normalization its class `run()` does (e.g. gitlab-code-review parses
`mrUrl` → `projectId`+`mrIid` only on cold runs — pass
`projectId`/`mrIid` yourself).

```js
// agents/review-then-meter/graph.mjs
import { WorkflowAgent, WorkflowGraph } from '@zibby/core';
import { z } from 'zod';

export class ReviewThenMeterWorkflow extends WorkflowAgent {
  buildGraph() {
    const graph = new WorkflowGraph();
    graph.setStateSchema(z.object({
      projectId: z.string().describe('GitLab project path, e.g. group/repo'),
      mrIid: z.number().describe('Merge request IID to review, then meter'),
    }));

    // Brick 1 — dispatch the deployed gitlab-code-review agent.
    graph.addNode('review', {
      workflow: 'gitlab-code-review',          // deployed slug in this project
      input: (state) => ({ projectId: state.projectId, mrIid: state.mrIid }),
      timeoutMs: 15 * 60 * 1000,
    });

    // Brick 2 — dispatch the deployed commit-meter agent.
    graph.addNode('meter', {
      workflow: 'commit-meter',
      input: (state) => ({ projectId: state.projectId, mrIid: state.mrIid, mode: 'mr' }),
    });

    // Bricks are full multi-exit graphs. Branch on the child's RETURNED
    // state — its node outputs are nested (`state.review.review.posted`
    // = the child's own `review` node output). Only meter when the
    // review was actually posted and it wasn't a comment-reply run.
    graph.addConditionalEdges('review', (state) =>
      (state.review?.review?.posted === true
        && state.review?.trigger !== 'comment_reply') ? 'meter' : 'END');

    graph.addEdge('meter', 'END');
    graph.setEntryPoint('review');
    return graph;
  }
}
export default ReviewThenMeterWorkflow;
```

Plus the usual `agent.json` (`entryClass: "ReviewThenMeterWorkflow"`)
and a `package.json` with `@zibby/core` (^0.5.0+ — older engines lack
in-process dispatch) + `zod`. Deploy with
`zibby agent deploy review-then-meter`, trigger with
`zibby agent trigger <uuid> -p projectId=group/repo -p mrIid=9` —
same as any workflow.

Chain conditions must render VISIBLY in the graph — never an unlabeled
dispatch→End edge. `addConditionalEdges` alone works (the engine
≥0.4.29 materializes the branch in the serialized graph); for an
explicit labeled **Condition diamond** add a router node —
`graph.addNode('gate', { description: '...' })` (no
execute/prompt/outputSchema) — and route with
`graph.addConditionalEdges('gate', routeFn, { labels })`.

Sub-workflow node options: `workflow` (required, deployed slug),
`input` (object, or `(state) => object`), `output` (dot-path string or
`(finalState) => any` to extract just what you need instead of the
whole child state), `async: true` (fire-and-forget → returns
`{ jobId }`), `timeoutMs`, `retries`. For fan-out (sync N children —
e.g. a multi-source ingest that syncs lark + gitlab + github in one
run), call `dispatchSubgraph(slug, { input, async: false })` (import
from `@zibby/agent-workflow`, add it to package.json) inside one custom
`execute` node, one child per source you actually have a target for.

> **dispatchSubgraph runs the child IN-PROCESS — on a current engine the
> platform auto-supplies the parent's agent shell + cancel signal, so a
> bare `dispatchSubgraph(slug, { input, async: false })` "just works".**
> On OLDER engines a hand-rolled dispatch that omitted them ran the child
> agent-less (LLM nodes fail) or fell back to the HTTP path, which on
> self-host makes the PARENT hang until it's reaped as "stale" ~5 min
> after the child already finished (silent: child exits 0, parent dies
> later). If you must support an older engine, pass them explicitly —
> they're right there in the execute context:
> ```js
> execute: async (context) => {
>   const state = context.state?.getAll?.() ?? context;
>   const r = await dispatchSubgraph('lark-kb-sync', {
>     input: { event: { source: 'lark', rootNodes: [...] } },
>     async: false, output: 'fetch',
>     parentAgent: context.agent, signal: state._signal, // ← belt-and-suspenders
>   });
> }
> ```
> **Prefer SEQUENTIAL dispatch (a `for` loop) over `Promise.allSettled`**
> for in-process children: they share the run process, and env-carrying
> children serialize anyway (above) — one-at-a-time avoids exec-context
> interleaving and isolates a single source's failure. Reserve
> `Promise.allSettled` for children that are all no-env + genuinely
> independent.

**Fan-out / multi-source ingest — three things that bite (all learned
the hard way building the knowledge-base agent):**
- **A trigger body maps to `state`, and a node reading `state.event.x`
  will NOT see a top-level `{ x }`.** Whether your input lands at the
  top of state or under `event` depends on how the node reads it — so
  make the node's normalizer accept BOTH shapes (`state.sources ??
  state.event ?? loose-top-level`), and ship a `triggerExample` (VALID
  JSON — JSON has no comments) in the template so the deploy UI
  pre-fills the correct shape. Don't hardcode the example in the UI;
  it's template-driven.
- **When several children feed ONE shared resource (a KB / a store /
  a brain), make exactly ONE node the WRITER.** Children only fetch +
  return their docs; a single downstream `ingest` node does every
  write. Concurrent writers to one store corrupt it or drop writes.
- **The engine runs nodes SEQUENTIALLY** — plain edges don't fan out in
  parallel; `dispatchSubgraph` is the ONLY parallelism primitive. If you
  expected two edges to run at once, they won't.

**Env: in-process children run with their OWN row's env**
(`@zibby/agent-workflow` ≥0.4.32 + matching backend). A brick's
per-workflow env (its Env tab / `envSecret`) is applied to its
in-process wrapped runs too — the child's value wins for its run, and
the WRAPPER's env remains the fallback for any key the brick doesn't
define. So a wrapper needs ZERO credential duplication: leave each
brick's creds (e.g. `CLAUDE_CODE_OAUTH_TOKEN`) on the brick itself.
One caveat: env-carrying children are SERIALIZED when dispatched in
parallel (`Promise.allSettled` fan-out still gets correct per-child
creds, just one-after-another; bricks with no env of their own keep
full parallelism). On older engines (<0.4.32) children inherit the
wrapper's env only — symptom: the brick "completes" with an
`orchestration error: authentication_failed` detail.
A brick's saved per-node CUSTOM PROMPTS
(`nodeConfigOverrides.<node>.extraPromptInstructions`, via UI /
`zibby agent prompt` / MCP) apply in-process since ≥0.4.30, and its
`stores` bindings ride along since ≥0.4.32 (the store env + allowlist
ship with the child's own env).

### Triggers for compositions — INHERIT the entry brick's events

The wrapper's trigger is AGENT-DRIVEN, never hardcoded: for
webhook-driven compositions the wrapper's `triggers.events` is a COPY
of whatever the ENTRY brick declares — read it, don't invent it.
Recipe:

1. Read the entry brick's declared events from its deployed row:
   `GET $ZIBBY_API_URL/projects/<projectId>/workflows/<entry-slug>`
   → `.triggers.events` (equivalently, the brick template's
   `agent.json` `triggers.events`).
2. Write that exact array into the WRAPPER's `agent.json`:
   `{ "triggers": { "events": [<the entry brick's events, verbatim>] } }`.
   `zibby agent deploy` stamps it onto the wrapper's row.

The platform automatically SUPPRESSES the wrapped members' own
subscriptions (any workflow listed in a deployed wrapper's
`composedOf` stops receiving webhook events standalone), so the same
event never double-fires the brick inside AND outside the wrapper.
Cron / manual / chat-triggered compositions need nothing special.

### Cloud vs self-host — identical flow

Everything above works unchanged against a self-hosted control plane:

```bash
export ZIBBY_API_URL=https://zibby.internal.example.com   # control-plane URL
export ZIBBY_API_KEY=zby_pat_xxx    # USER pat — `zibby self-host token` mints one
```

Self-host specifics (verified against a live self-host stack):
- **In-process children need a one-time bundle step.** Cloud builds
  child bundles at deploy; the self-host deploy stages sources only —
  until you run the self-host bundle-builder
  (`node backend/selfhosted/build-child-bundle.mjs --project <id>
  --workflow <child-slug>`, shipped with the stack) for each brick,
  every dispatch still WORKS but falls back to a second run container
  (`dispatchMode` on the child execution row shows the difference).
  Re-run it after re-deploying a brick.
- **Deploy from an external machine can fail at the source upload** if
  the stack presigns its own object store (MinIO) with a
  docker-internal hostname
  (`minio:9000`). Deploy from the stack host (or a container on its
  network), or have the operator set a host-reachable
  `S3_PUBLIC_ENDPOINT`.
- For self-hosts on self-signed HTTPS, the CLI uses Node's standard
  TLS (no `--insecure` flag exists): point Node at the CA with
  `export NODE_EXTRA_CA_CERTS=/path/to/self-host-ca.pem`. Plain-HTTP
  self-hosts need nothing.

## 11. BYO sidecars (self-host) — ship your own resident service/MCP

A **sidecar** is a long-lived container the box runs NEXT TO agents
(the platform's own vector-KB and OAuth-broker engines ship this way).
On a self-hosted box you can register **your own** sidecar image — the
standard way to give agents (or the chat Copilot) a custom resident
service, e.g. a thin **MCP server wrapping an internal REST API**:

**Before you build one: if the goal is "let my agents call this internal REST
API", deploy the marketplace agent `OpenAPI MCP Bridge` instead.** It carries
its own sidecar, so deploying it IS the installation — no image, no upload. Put
the spec URL in its **Env** tab and every operation becomes an MCP tool:

```json
OPENAPI_APIS = {"billing":{"specUrl":"https://internal/v2/api-docs","root":"https://internal"}}
```

Its MCP endpoint is `/mcp/agent/<uuid>` on the box origin (Bearer PAT, shown on
the agent's detail page) — point a client at it, or attach it to another agent
with `zibby_add_mcp`. Build your own image (below) only when you need something
the bridge can't express: a non-HTTP protocol, a stateful session, custom auth.

```bash
# 1. build + save your image (any language; must serve HTTP on a port)
docker save my-report-mcp:0.1.0 | gzip > report-mcp.tar.gz

# 2. push it to the box (operator PAT — a restricted token is refused)
zibby sidecar push report-mcp.tar.gz --name report-mcp --port 8080 \
  --health-path /health --warm
zibby sidecar list           # custom + built-in names
zibby sidecar remove <name>  # unregister + stop
```

Facts that matter:
- The tarball is **sha256-pinned server-side** and re-verified on every
  load; a tampered/corrupt image never starts. Built-in sidecar names
  (`gbrain`, `pingcode`, …) are reserved — pick another.
- The container runs on the box's **infra network** at
  `http://zibby-sidecar-<name>:<port>` — the control-plane (and the
  in-process Copilot) can reach it; hostile run containers cannot dial
  it directly. `--public-path` additionally exposes a prefix ANONYMOUSLY at
  `https://<box-origin>/sidecars/<name>/…` — most sidecars need none (the
  copilot uses the infra network), so it is refused unless the box sets
  `SIDECAR_BYO_PUBLIC_PATHS=1`. It exists for browser/OAuth-callback flows.
- To let the **chat Copilot** use a sidecar-served MCP: register the
  sidecar, then attach the endpoint with the Copilot's `zibby_add_mcp`
  (URL `http://zibby-sidecar-<name>:<port>/mcp`). The Copilot can also
  drive the whole flow itself via `zibby_add_sidecar` /
  `zibby_list_sidecars` / `zibby_remove_sidecar` (owner-only; the add
  tool takes a `url` + pinned `sha256` instead of a file upload).
- `--warm` keeps it always-on (else on-demand + idle-reap). `--env-key`
  forwards a box env var; `--request-config-key` declares per-agent
  config resolved from the calling agent's encrypted Env bag.
- **Cloud has no BYO sidecars today** — this is a self-host-only
  surface; on cloud, wrap external APIs as a remote MCP instead.

---

# Pillar 2: Apps

## A. The 30-second Apps tour

```bash
zibby app templates                                 # browse the catalog
zibby app deploy n8n --project <id>                 # catalog deploy (deterministic)
zibby app deploy --goal "<text>" --project <id>     # goal-mode (LLM bootstrap)
zibby app list                                      # what's running
zibby app status <instanceId>                       # one instance's state
zibby app logs <instanceId> -t                      # live tail (container + supervisor)
zibby app set-auth <instanceId> --auth-type basic --auth-user admin --auth-password ...
zibby app upgrade <instanceId> --version vX.Y.Z     # agent-ops base image bump
zibby app destroy <instanceId> --yes                # permanently delete (app data wiped)
```

A Managed App is a long-running web service on Zibby's managed fleet. Each
instance has:
- A managed task running `agent-ops` + (for catalog) the app image OR (for
  goal-mode) whatever the agent installed
- A pinned data volume for persistent state (DB files, uploads, config)
- A public `https://<id>.apps.zibby.app` URL
- An optional auth sidecar (basic-auth, bearer token, or none)
- An encrypted env-var bag

## B. Catalog vs goal-mode — which path

The two `app deploy` paths are mutually exclusive.

### Catalog: `zibby app deploy <appType>`

- The backend uses a baked task definition. No LLM runs to install.
- Cold start: ~2-3 minutes (image pull + first boot).
- 20+ catalog entries: `n8n`, `grafana`, `wordpress`, `outline`,
  `mattermost`, `gas-town`, `caddy-static`, `appsmith`, `flowise`,
  `code-server`, `chatwoot`, `vaultwarden`, `umami`, `gitea`,
  `nocodb`, `directus`, `posthog`, `metabase`, `langfuse`,
  `flagsmith`. Run `zibby app templates` for the live list and per-app
  `architecture` requirements.
- Predictable, supported, the right default for "I want X" when X is in
  the catalog.

### Goal-mode: `zibby app deploy --goal "<text>"`

- LLM bootstrap: an `agent-ops` task in the user's instance runs an
  autonomous install loop driven by the goal text.
- Cold start: 5-30 minutes depending on what's being installed.
- Use for **anything not in the catalog**: custom apps, a specific
  git repo, an OSS project not promoted yet, multi-service exotic
  stacks.
- License responsibility for whatever gets installed sits with the
  user, not Zibby (same shape as `apt install` on a generic VPS).

Goal-mode flags worth knowing:

| Flag | Default | When to override |
|---|---|---|
| `--provider claude\|codex` | `claude` | Pick the agent driving the install |
| `--model <id>` | known-cheap default | Pin a specific model (e.g. `claude-sonnet-4-6`) |
| `--anthropic-token sk-ant-...` | workspace-stored | Per-deploy token override; format `sk-ant-oat01-` (OAuth) or `sk-ant-api03-` (API) |
| `--max-turns N` | 25 | Heavy installs (n8n, OpenHands) need 60-100 |
| `--timeout-min N` | 20 | Heavy installs need 30-45 |
| `--arch x86_64\|arm64` | per-template | Override CPU arch (most catalog entries are arm64) |

## C. Auth — every app gets a public URL, lock it down

Without auth, ANYONE with the `https://<id>.apps.zibby.app` URL can hit
the app. For tools like n8n / Grafana / Outline that's a real risk —
the URL is guessable from the catalog.

Three auth modes on the auth sidecar:

| Mode | When to use | Set with |
|---|---|---|
| `basic` | Quick personal tools, dashboards | `--auth-type basic --auth-user admin --auth-password ...` |
| `token` | API-only apps, scripted callers | `--auth-type token --auth-token ...` |
| `none` | App has its own login (n8n, wordpress) | `--auth-type none` or omit |

Set at deploy time, change after deploy with `zibby app set-auth`. Use
`zibby app set-auth <instanceId> --off` to remove auth entirely (only
safe if the app has its own login).

**Rotation:** re-run `set-auth` with new credentials. Old credentials
stop working immediately when the auth-layer reload completes (~5s). No
container restart needed.

**Always generate credentials with `openssl rand -hex`** — never reuse
a user-typed password. Never log credentials. Save them once at deploy
time; they're not recoverable.

## D. Multi-service catalog entries

Some catalog entries run multiple containers in one task:
- `wordpress` → wordpress + mysql
- `mattermost` → mattermost + postgres
- `gas-town` → web + worker + scheduler

The instance has ONE status (whole-instance), ONE URL (the primary
service's), and a per-service log stream:

```bash
zibby app logs <instanceId> --service mysql
zibby app logs <instanceId> --service agent-ops    # the supervisor itself
```

`zibby app status` lists every service under `services[]`. The auth
sidecar fronts only the `mainService` (declared in the catalog manifest).

## E. The agent-ops supervisor

Every app runs an `agent-ops` sidecar — a small LLM-driven daemon
([github.com/zibbyhq/agent-ops](https://github.com/zibbyhq/agent-ops),
Apache-2.0).

What the supervisor does:
- **Goal-mode**: runs the install loop on first boot. Verifies on a
  schedule that the installed thing still works. Re-installs if it
  doesn't (within budget).
- **Catalog**: runs scheduled health checks per the catalog's recipe.
  Restarts misbehaving services. Notifies via webhook on
  unrecoverable failures.

What it can do: run `shell` commands inside its task's filesystem
(scoped to the instance's data volume + the task's egress proxy).

What it cannot do: touch other instances, your local machine, or any
Zibby control-plane resources. Sandbox by design.

To see the supervisor's trail: `zibby app logs <instanceId> --service agent-ops`.

## F. BYOH — agent-ops on your own VPS

Don't want to host on Zibby's fleet? Run `agent-ops` directly on a VPS
you own. Same daemon, same configs, you handle the host:

```bash
# Debian / Ubuntu
sudo install -d -m 0755 /etc/apt/keyrings
curl -fsSL https://dl.zibby.app/apt/key.gpg \
  | sudo gpg --dearmor -o /etc/apt/keyrings/zibby.gpg
echo "deb [signed-by=/etc/apt/keyrings/zibby.gpg] https://dl.zibby.app/apt stable main" \
  | sudo tee /etc/apt/sources.list.d/zibby.list
sudo apt update && sudo apt install agent-ops

# Register with your Zibby workspace (optional — lets the workspace see the host)
agent-ops register --pat zby_xxx
agent-ops init --template wordpress-multisite --yes
sudo agent-ops start
```

Full docs: https://docs.zibby.app/apps/agent-ops. macOS / Homebrew + Docker
install paths are documented there too.

## G. Apps lifecycle — the commands at a glance

| Action | Command | Slash command |
|---|---|---|
| List instances + browse catalog | `zibby app list` / `zibby app templates` | `/zibby-app-list` |
| Deploy (catalog) | `zibby app deploy <appType>` | `/zibby-deploy-app` |
| Deploy (goal-mode) | `zibby app deploy --goal "..."` | `/zibby-deploy-app` |
| Status | `zibby app status <instanceId>` | `/zibby-app-status` |
| Logs | `zibby app logs <instanceId> [-t]` | `/zibby-app-logs` |
| Upgrade agent-ops | `zibby app upgrade <instanceId> --version vX.Y.Z` | `/zibby-app-upgrade` |
| Auth (set / rotate / off) | `zibby app set-auth <instanceId> ...` | `/zibby-set-auth` |
| Destroy (irreversible) | `zibby app destroy <instanceId>` | `/zibby-app-destroy` |

## H. Apps — common pitfalls

| Symptom | Fix |
|---|---|
| Goal-mode times out at default `--timeout-min` | Heavy install. Retry with `--timeout-min 45 --max-turns 80` and a more specific goal |
| `--anthropic-token must start with sk-ant-oat01- or sk-ant-api03-` | User pasted an IP-bound interactive token. `claude setup-token` gives a long-lived one |
| 402 from `app deploy` | Workspace lacks an Apps subscription. Direct to https://zibby.dev/billing |
| `pending` status for >10 min | Image pull stuck or task crashing on boot. `app logs <id>` for stderr |
| URL 502s after restart | New task hadn't passed health check yet. Wait 60s |
| App config changes not picked up | Env vars require `app restart` after `app env set` |
| `app destroy` lost important data | App data is wiped on destroy. No backup. Tell users explicitly before destroying anything stateful |
| Wrong auth mode set | `app set-auth --off` then re-run with the right mode |

## I. The agent's job for Apps (this is YOU)

When the user says **"deploy me a hosted X"**:

1. **Catalog or goal?** Check `zibby app templates`. If X is listed,
   use catalog. If not, use goal-mode with a clear sentence.
2. **Which project?** Look at `zibby status` for the current project,
   or prompt with `zibby list`.
3. **What auth?** Ask the user before running deploy. Pick basic /
   token / none. Generate secure creds with `openssl rand -hex`.
4. **Run deploy.** Capture the `instanceId` from the output.
5. **Tail logs while it boots.** Background the tail.
6. **Verify status reaches `running`.** Then tell the user the URL +
   auth credentials.
7. **Save the credentials.** Tell the user to save them too — they're
   rotatable but not recoverable.

When the user says **"my app is broken"**:

1. `zibby app status <id>` — read the status field.
2. `zibby app logs <id>` — read the last 100 lines.
3. Diagnose. Restart, env-fix, or escalate.
4. Don't destroy unless the user confirms data loss.

---

# Cross-pillar reference

## Auth + login

`zibby login` — browser OAuth, writes `~/.zibby/session.json`. Token
lasts 30 days. For headless / CI, set `ZIBBY_API_KEY=zby_xxx` (PAT
from https://zibby.dev/settings/api-keys) — env var takes precedence
over the session file. `zibby status` shows current auth + project +
configured agent credentials.

## Project model

Apps and workflows both live inside a **project**. List projects with
`zibby list`. Switch with `zibby project use <id>`. Set a default in
`.zibby.config.mjs` (`workspace.defaultProject`). When deploying, the
CLI prompts interactively if `--project` isn't passed and no default
is configured.

## MCP — let the IDE agent talk directly to Zibby

`zibby mcp install` (interactive; or `--project` / `--agent
<claude-code|cursor|codex|…>` + `--yes` for scripts) validates your
control-plane URL + token live, then writes the MCP server entry —
`./.mcp.json` for project scope, or the agent's global config. Works
against Zibby cloud AND a self-hosted box (`--url https://<box>` or
`ZIBBY_API_URL`). After this, the IDE agent can call `zibby_workflow_*`
/ `zibby_app_*` tools directly without shelling out. Inside Claude
Code: run `/zibby-connect` to set it up conversationally. See
`/zibby-mcp-install`.

## Memory sync

Test memory (`.zibby/memory/.dolt/`) is local-first Dolt SQL.
`zibby memory remote add` / `zibby memory remote use --hosted` opts
into team sync — teammates auto-pull learnings on `zibby test` start
and auto-push on a passing test. Set `memorySync.remote` in
`.zibby.config.mjs` and `zibby init` wires the remote automatically
for the rest of the team.

## How to invoke the CLI

`zibby` should be on PATH (npm global). If not, every project ships
`./.zibby/bin/zibby` as a fallback shim. Don't fall back to
`npx @zibby/cli` — not always published.