# ZelPi — Backend Integration Phases

Connecting real backends incrementally. **Free / OSS now → paid migration later.**

| Phase | Goal | Free / OSS (now) | Paid (later) | Status |
|---|---|---|---|---|
| **1** | Authoritative realtime backend + persistence + multi-client | Node + `ws` + `tsx`, JSON-file store, engine runs server-side | Fly.io / Railway host · Neon Postgres | ✅ **done** |
| **2** | Real System 2 intent parsing | **Ollama** (local Llama/Qwen) + deterministic fallback | Anthropic Claude / Bedrock | ✅ **done** |
| **3** | Durable database | **SQLite** (Node built-in `node:sqlite`, no deps) | Neon / RDS Postgres | ✅ **done** |
| **4** | Real robot via simulator | **rosbridge protocol** + bundled sim-robot (ROS 2 + Gazebo drop-in) | managed sim / hardware | ✅ **done** |
| **5** | Real Foundry (registry/tracking) | **MLflow** REST (OSS) | SageMaker | ✅ **done** |
| **6** | Auth + observability | **JWT** (node:crypto) + **Prometheus** `/metrics` | Keycloak/Auth0/Cognito · Grafana Cloud | ✅ **done** |
| **7** | Hardware-ready native HAL | **Versioned JSON protocol v1** (no ROS) + e-stop/watchdog/heartbeat + `RobotDriver` adapter interface + `zelpi hal` CLI | Multi-robot orchestration · managed fleets · cloud-native bridge | ✅ **done** |

---

## Phase 1 — what was built ✅

The simulation engine is now **server-authoritative**. The browser became a thin
client; the offline in-browser mode still works (so the public Vercel demo is
unaffected).

**Components**
- `server/server.ts` — Node HTTP + WebSocket server. Runs `PiOsEngine` at ~40 Hz,
  broadcasts authoritative snapshots ~13 Hz, accepts commands, persists to
  `server/state.json` every 3 s (and on SIGINT/SIGTERM).
- `lib/engine.ts` — added `exportState()` / `importState()` for durable persistence.
- `lib/useEngine.ts` — split into `useLocalEngine` (offline rAF) and
  `useRemoteEngine` (WebSocket client). Selected by the `NEXT_PUBLIC_PIOS_WS_URL`
  env var at build time.
- `components/Deck.tsx` / `TopBar.tsx` — handle the pre-first-snapshot state and
  show a connection badge (`offline engine` / `backend live` / `reconnecting…`).

**Protocol (WebSocket JSON)**
- Server → client: `{ type: "snapshot", snap: WorldSnapshot }`
- Client → server: `{ type: "intent", text }`, `{ type: "resolveSafety", option }`,
  `{ type: "createModel", kind, name, embodiment }`, `{ type: "deployModel", id }`,
  `{ type: "enrollEmbodiment", spec }`, `{ type: "togglePause" }`, `{ type: "reset" }`

**Properties gained**
- Single source of truth — multiple browser tabs/operators see the same fleet.
- State survives reload **and** server restart (JSON persistence).
- Clean seam (`exportState`/`importState`, the WS message switch) to swap the
  store for Postgres (Phase 3) and the host for Fly/Railway.

### Run it (two terminals)
```bash
# terminal 1 — backend (always-on; not serverless)
npm run server            # → http://localhost:8787  (ws://localhost:8787)

# terminal 2 — frontend pointed at the backend
npm run dev:connected     # → http://localhost:3000  (badge: "backend live")
```
Offline demo (no backend) is still just `npm run dev`. Health check:
`curl http://localhost:8787/health`.

### Verified
- `/health` → 200, sim clock advancing.
- WS round-trip: client sends "Inventory Aisle 4" → server creates task T1
  (5 subtasks, assigned to Atlas-01) and streams it back.
- Persistence: T1 written to `server/state.json`, restored on restart.
- `npm run build` (offline path) compiles clean.

---

## Phase 2 — real intent parsing ✅

System 2 now parses intent with a local **Ollama** model, falling back to the
deterministic parser on any failure. Same `submitIntent` surface — no UI change.

**Components**
- `server/planner.ts` — `planIntent(text, items)` calls Ollama `/api/chat`
  (`format: json`, temperature 0.2) with a System-2 prompt + world context
  (aisles + items), then **validates/coerces** the JSON (agent/kind enums,
  resolves color→itemId, clamps aisle). Returns `{ parsed, source: "llm"|"fallback" }`.
  `probeLLM()` checks availability for `/health` + startup log.
- `lib/engine.ts` — `submitPlannedIntent(text, parsed, source)` shares one code
  path; `submitIntent` delegates with the deterministic plan.
- `server/server.ts` — `intent` handler awaits the planner, then dispatches;
  `/health` reports `llm` status; re-probes every 30 s (picks up Ollama if it
  comes online later).

**Providers** — `server/planner.ts` supports two, selected by `PIOS_LLM_PROVIDER`
(`openai` | `ollama` | `auto`; `auto` picks OpenAI when a key is present):
- **OpenAI** (`/chat/completions`, JSON mode): `OPENAI_API_KEY`, `OPENAI_MODEL`
  (default `gpt-4o-mini`), `OPENAI_BASE_URL` (default `https://api.openai.com/v1`).
- **Ollama** (local/free): `OLLAMA_URL`, `OLLAMA_MODEL` (default `llama3.2`).
- Shared: `OLLAMA_TIMEOUT_MS` (15000), `PIOS_LLM=off` forces the deterministic parser.

Secrets load from a git/Vercel-ignored **`.env.local`** via `server/loadenv.ts`
(imported first in `server.ts`). The key never reaches source or the browser.

**Config (env)**
- `OLLAMA_URL` (default `http://127.0.0.1:11434`), `OLLAMA_MODEL` (default
  `llama3.2`), `OLLAMA_TIMEOUT_MS` (15000), `PIOS_LLM=off` to force deterministic.

**Enable the real LLM (free):**
```bash
# install from https://ollama.com, then:
ollama serve              # starts the API on :11434
ollama pull llama3.2      # ~2 GB; or qwen2.5:3b for lighter
npm run server            # auto-detects → "System 2 planner: LLM ready"
```
Without Ollama the backend logs "LLM unreachable — using deterministic fallback"
and works exactly as before.

**Verified** (via a mock Ollama): LLM-supplied plan is applied and the feed shows
`VLA (model) parsed`; when the model returns invalid JSON the system falls back to
the deterministic parser. Build clean.

**Active provider: OpenRouter** (OpenAI-compatible) via the OpenAI provider +
`OPENAI_BASE_URL=https://openrouter.ai/api/v1`, model `nex-agi/nex-n2-pro:free`
(a free model). Verified with real calls: key valid (HTTP 200), JSON-mode plans
returned correctly; mock proves the backend applies them (source=llm). Free models
can rate-limit → graceful fallback handles it; for reliability add a little
OpenRouter credit and set `OPENAI_MODEL=openai/gpt-4o-mini` (or
`meta-llama/llama-3.3-70b-instruct`).
(History: a prior OpenAI key authenticated but the account had `insufficient_quota`
— OpenRouter sidesteps that.) ⚠️ Keys in `.env.local` were shared in chat — **rotate them**.

**Paid migration:** OpenAI is live; alternatively swap to Anthropic Claude
(Bedrock) behind the same `planIntent()` interface — add prompt caching there.

## Phase 3 — durable database ✅

Persistence moved from a single JSON blob to **SQLite** (Node's built-in
`node:sqlite` — no external dependency, no Docker, free).

**Components**
- `server/store.ts` — a `Store` interface with `SqliteStore` (default) and
  `JsonStore` (fallback). SQLite schema: a `state` row (authoritative serialized
  snapshot) plus **queryable mirror tables** `tasks`, `models`, `embodiments`,
  written transactionally on each save (WAL mode). DB at `server/pi-os.db`
  (git/Vercel-ignored).
- `server/server.ts` — uses the store for load/save; **one-time migration** from
  the legacy `state.json` on first boot; `/health` reports `store`; new
  read endpoints `/db/tasks`, `/db/models`, `/db/embodiments`.

**Verified**
- Boot 1: migrated `state.json` → SQLite. Boot 2: `restored durable state (sqlite)`
  (no re-migration). Seeded a task + embodiment + model, killed the server,
  restarted → **all survived** (`vla-phase3 v2`, `Forklift-09`, fleet=5).
- `curl /db/models` etc. return real rows. Build clean.

**Paid migration:** implement the same `Store` interface over Postgres (Neon/RDS)
— swap `createStore()` and keep everything else.

## Phase 4 — real robot via simulator ✅

The robot is now an **external component** driven over the standard **rosbridge v2
protocol** (advertise/subscribe/publish). The engine stops integrating motion and
instead publishes `geometry_msgs/Twist` to `/<id>/cmd_vel`, ingesting
`nav_msgs/Odometry` from `/<id>/odom` to drive pose. A real **ROS 2 + Gazebo**
robot drops in unchanged — just point `ROSBRIDGE_URL` at its `rosbridge_server`.

**Components**
- `lib/engine.ts` — `externalControl` flag (skips internal `moveToward`),
  `setRobotPose()` (odometry injection), `fleetGeneration` (re-seed trigger).
- `server/rosbridge.ts` — `RobotBridge`: connects to rosbridge, advertises/subscribes
  per-robot topics, seeds initial poses, ~15 Hz diff-drive `cmd_vel` toward each
  waypoint, parses odom quaternion → yaw → `setRobotPose`. Auto-reconnect; falls
  back to the internal model when down. Enabled by `PIOS_ROS=on` (`ROSBRIDGE_URL`).
- `server/sim-robot.mjs` — bundled mock `rosbridge_server` + differential-drive
  fleet so the full loop runs **without installing ROS** (`npm run sim:robot`).
- `server.ts` — `/health` reports `ros`; bridge started/stopped with the server.

**Run it (free, no ROS install):**
```bash
npm run sim:robot     # terminal A — mock rosbridge robots on :9090
npm run server:ros    # terminal B — backend with the ROS link (PIOS_ROS=on)
npm run dev:connected # terminal C — UI
# …or one command:  ROS=1 ./run.sh
```
With a real robot: launch `ros2 launch rosbridge_server rosbridge_websocket_launch.xml`
+ Gazebo, set `ROSBRIDGE_URL=ws://<host>:9090`, match topic names — no code change.

**Verified:** with the sim-robot, Atlas-01 was driven `(4,8) → (72,27)` (74 units)
**entirely via the odometry loop** (internal motion disabled), `/health` shows
`ros.connected:true`. Build clean.

## Phase 5 — Foundry → model registry (MLflow) ✅

The Foundry now mirrors every training run into an **MLflow** tracking server.
- `server/foundry.ts` — on `createModel` opens an MLflow run + logs params; streams
  the loss curve as `loss` metrics (1 Hz); on convergence logs `final_loss`/`quality`,
  finishes the run, and **registers a model version**; on `deployModel` transitions
  that version to **Production** (`archive_existing_versions`). Best-effort — the
  engine's local training still drives the UI if MLflow is down.
- Enable: `PIOS_MLFLOW=on MLFLOW_URL=http://127.0.0.1:5000 npm run server`
  (run any MLflow: `pip install mlflow && mlflow server`). `/health` reports `foundry`.
- **Verified** (mock MLflow): all calls made — `runs/create`, `log-batch`,
  `log-metric`, `runs/update`, `model-versions/create`, `transition-stage`.

### Real training (free path) ✅
The Foundry's "train" can run an **actual LeRobot ACT** job instead of the
simulated curve — genuine gradient descent on a public robot dataset, $0 on a
free GPU. No hardware needed (dataset from the HF Hub).
- `train/train_act.py` — real ACT training → checkpoint + MLflow run, prints
  JSON-line progress. `train/mock_train.mjs` — no-GPU stand-in (same protocol).
  `train/README.md` — Colab/local instructions.
- `server/trainer.ts` — when `PIOS_TRAIN=local`, `createModel` spawns the job and
  streams its real loss into the engine (`setModelTrainingProgress` /
  `completeModelTraining`); the model is flagged `external` so the simulator skips it.
- Run: `PIOS_TRAIN=local npm run server` (real), or
  `PIOS_TRAIN=local PIOS_TRAIN_CMD="node train/mock_train.mjs" npm run server` (demo).
- **Verified** (mock job): real subprocess drove the loss `0.61 → 0.17`, converged
  to `ready` (quality 83%), model `external:true`.
- Paid: SageMaker training jobs; rent an A100 (Modal/RunPod) to fine-tune π0/OpenVLA.

## Phase 6 — auth + observability ✅

- **Auth** (`server/auth.ts`) — JWT HS256 via `node:crypto` (no deps). When
  `PIOS_AUTH=on`, the WS command channel requires a valid token (`?token=` or
  `Authorization: Bearer`). `/auth/login` issues dev tokens; in production an IdP
  (Keycloak/Auth0/Cognito) issues JWTs and the same `verify()` validates them
  (`PIOS_AUTH_SECRET`). The UI sends `NEXT_PUBLIC_PIOS_TOKEN` if set.
  **Verified**: no/forged token → close 1008; valid token → connected.
- **Observability** (`server/metrics.ts`) — Prometheus exposition at **`/metrics`**:
  counters (commands, intents, llm vs fallback, safety halts, models trained/deployed,
  embodiments) + gauges (fleet, active tasks, sim clock, ws clients, cognitive
  bandwidth). Scrape with Prometheus → Grafana; ship logs to Loki.
  **Verified**: `/metrics` renders; counters increment on events.

### Run with everything on
```bash
PIOS_ROS=on PIOS_MLFLOW=on PIOS_AUTH=on npm run server   # + sim:robot, mlflow, token
```

### Deployment note
The console stays on Vercel. The **backend must run on always-on compute** —
Vercel serverless can't hold open WebSocket connections or the sim loop. First
paid step is hosting `server/` on Fly.io/Railway (free tiers exist) and setting
`NEXT_PUBLIC_PIOS_WS_URL` to its public `wss://` URL. SQLite persists to the
instance disk; switch the `Store` to Neon Postgres for multi-instance.

## Phase 7 — Hardware-ready native HAL ✅

A **versioned, validated protocol (v1)** plus a clean driver adapter interface
make real robots plug-and-play. No ROS 2, no rosbridge — just WebSocket JSON
between the controller and a tiny native agent that runs on the robot next to its
SDK.

**Components**
- `lib/hal/protocol.mjs` — the single source of truth. All ops (HELLO, WELCOME,
  POSE, CMD, SKILL, ESTOP, RELEASE, PING, TELEMETRY, SKILL_RESULT, FAULT, PONG),
  validated codec, versioned (`PROTOCOL_VERSION = 1`).
- `server/robot-agent.mjs` — the agent that runs on the robot. Tiny (~300 lines):
  WebSocket server, message loop, safety layer (e-stop latch, watchdog ~500 ms,
  heartbeat ~500 ms). Calls a pluggable `RobotDriver` interface.
- `RobotDriver` interface — implement this against your SDK (Unitree, xArm, AgileX,
  etc.). Methods: `capabilities()` (metadata), `applyCmd(v, w)` (motor control),
  `runSkill(skill, args)`, `readTelemetry()`, `seedPose()`, `estop()`, `release()`.
- `cli/hal.mjs` — controller side. `createRobotLink()` manages the WebSocket,
  handshake, heartbeat, link-health, e-stop/release. Plugs into the embedded
  backend so the entire 9-layer stack can drive real hardware.
- `cli/commands.mjs#hal` — CLI (`zelpi hal connect|disconnect|estop|release|skills|status`).
  Integrated with `embedded.mjs` so `PIOS_ROBOT_URL` auto-connects on boot.

**Safety guarantees**
- E-stop latch (motion refused until release).
- Watchdog (500 ms silence → safe-stop).
- Heartbeat liveness (1500 ms no pong → link unhealthy, best-effort safe-stop).
- Malformed-frame drop (no crash).
- Protocol version validation (reject mismatches at handshake).

**Deploy (trivial)**
Robot side = 3 files + `ws` npm package:
```bash
# On the robot
node server/robot-agent.mjs --driver unitree   # (or --driver xarm, etc.)

# On the controller
zelpi hal connect ws://192.168.1.50:9091
zelpi intent "patrol aisle 1"    # robot moves; pose from telemetry
```

**Verified**
- Simulator-based testing: diff-drive agent sends velocity, receives telemetry.
- Protocol validation: all ops round-trip; version mismatch → clean rejection.
- Safety: watchdog fires on silence; e-stop latches/releases correctly.
- Integration: controller's `setExternalControl(true)` on link-up yields pose to
  robot; `setRobotPose()` ingest telemetry → world model → policy → HAL → robot
  command (closed loop).

**Free (now)**
- Native agent + versioned protocol (in repo).
- `zelpi hal` CLI (embedded backend).
- Skeleton drivers for Unitree / xArm / AgileX (stubs to fill in against your SDK).

**Paid (later)**
- Multi-robot fleet coordination (currently single-link; extend to per-robot
  links + arbitration).
- Managed robot gateway service (handle nat/firewall; cloud-to-edge push).
- Hardware SDK integration packages (pre-wrapped Unitree, Boston Dynamics, UFACTORY SDKs).

**Alternatives**
- **Legacy rosbridge** (`server/rosbridge.ts`, `PIOS_ROS=on`) still works but adds
  ROS 2 complexity; native HAL is the recommended path for new deployments.
- **Custom transport** (MQTT, gRPC, CAN, etc.) — the protocol codec is transport-agnostic;
  swap the WebSocket for your transport by reimplementing `cli/hal.mjs`.

See [`docs/HARDWARE.md`](HARDWARE.md) for the complete integration guide: protocol
spec, wire-format tables, units/frame contract, troubleshooting, and step-by-step
driver implementation.