# ZelPi Hardware Integration — Native HAL Guide

**Last Updated:** 2026-06-20  
**Protocol Version:** 1  
**No ROS required · DIMOS-style native agent · WebSocket JSON over plain TCP**

---

## Overview

ZelPi's Hardware Abstraction Layer (HAL) is a **minimal, versioned wire protocol** that decouples the embedded controller from the robot. Instead of ROS 2 + rosbridge (which adds complexity and deployment friction), the native HAL speaks **pure JSON over WebSocket** between the controller (embedded backend, running on the operator's machine or an edge service) and a tiny agent that lives on the robot next to its SDK.

### Why native instead of ROS?

- **Deployment trivial**: robot side = 3 files (`server/robot-agent.mjs` + `lib/hal/protocol.mjs` + the `ws` npm package). No ROS 2 installation, no rosbridge_server, no DDS.
- **Fast control path**: no middleware overhead; the policy sends a velocity command and it lands in 10–50 ms over a local network.
- **Works offline**: point `zelpi hal connect ws://192.168.1.50:9091` and go; no central ROS master or rosbridge instance required.
- **Version-controlled**: the wire protocol is semantically versioned (`PROTOCOL_VERSION`); agents reject controllers they disagree with at the handshake.
- **Runs inside the embedded backend**: when a hardware link is up, the entire 9-layer stack (Layers 4–8: World → Policy → Perception → HAL) drives **real hardware** through the same seams it drives the software twin.

### System diagram

```
┌──────────────────────┐          ┌─────────────────────────────────────┐
│   ZelPi Controller   │          │   Robot Agent (on robot hardware)   │
│ (embedded backend)   │          │                                     │
├──────────────────────┤          ├─────────────────────────────────────┤
│ 4. World Model       │  ┌─────→ │ websocket server                    │
│ 5. Policy            │  │       │   (ws://127.0.0.1:9091)            │
│ 6. Perception        │  │       │                                     │
│ 7. HAL              │  │       ├─────────────────────────────────────┤
│   ├ createRobotLink  │  │       │ RobotDriver interface              │
│   └ estop/release    │  │       │   (implements your SDK)            │
└──────────────────────┘  │       │                                     │
         ↑ setRobotPose   │       │ ├ capabilities()  → robot metadata │
         ↓ cmd/skill      │       │ ├ applyCmd(v,w)   → motor control  │
  ws://robot:9091 ────────┴───→   │ ├ runSkill(…)     → high-level    │
                                  │ ├ readTelemetry() → odometry      │
     Protocol v1 JSON             │ ├ seedPose()      → pose init     │
   (validated & versioned)        │ ├ estop()         → latch motion  │
                                  │ └ release()       → resume motion │
                                  └─────────────────────────────────────┘
```

---

## Wire Protocol v1 — Complete Message Table

Every frame is a JSON object with envelope `{ v: 1, op: "…", … }`. All frames are validated by the `decode()` function on inbound; malformed input is silently dropped (never crashes).

### Protocol Constants

```javascript
PROTOCOL_VERSION = 1
```

### Operation Set

| Direction | Op | Shape | Purpose |
|-----------|----|----|---------|
| **Handshake** | | | |
| robot → ctrl | `hello` | `{ v, op, caps }` | Robot announces hardware capabilities, drive kind, skills, limits. |
| ctrl → robot | `welcome` | `{ v, op, accept, controller, reason? }` | Controller accepts or rejects (reason: protocol mismatch, etc.). Link is live if `accept: true`. |
| **Control** (ctrl → robot) | | | |
| | `pose` | `{ v, op, id, x, y, theta }` | Seed a robot's initial odometry pose. Sent once on link-up. |
| | `cmd` | `{ v, op, id, v_, w, seq }` | **Unicycle velocity.** `v_` is linear (m/s), `w` angular (rad/s), `seq` a monotonic counter. |
| | `skill` | `{ v, op, id, skill, args, reqId }` | High-level skill (e.g. `"grasp"`, `"dock"`). Returns `skillResult` with progress/error. |
| | `estop` | `{ v, op, reason }` | Emergency stop. Robot latches this and refuses motion until `release`. Reason is a string (e.g. "manual", "watchdog"). |
| | `release` | `{ v, op }` | Clear the e-stop latch. |
| | `ping` | `{ v, op, t }` | Heartbeat probe. `t` is a timestamp (ms). |
| **Feedback** (robot → ctrl) | | | |
| | `telemetry` | `{ v, op, id, x, y, theta, ts, vx?, w?, battery?, joints?, faults?, estop? }` | Live odometry & state. `x/y` in robot's frame units (m), `theta` in radians. Optional: `vx` (linear velocity), `w` (angular), `battery` (%), `joints` (float array), `faults` (string array), `estop` (boolean). |
| | `skillResult` | `{ v, op, id, reqId, status, progress?, error? }` | Skill progress/completion. `status` is one of `"accepted" \| "running" \| "done" \| "failed"`. Optional `progress` (0–1) and error message. |
| | `fault` | `{ v, op, id, code, message, severity }` | Out-of-band fault report. `severity` is one of `"info" \| "warning" \| "error" \| "critical"`. |
| | `pong` | `{ v, op, t }` | Heartbeat reply. Echoes the ping `t`. |

### Skill Status Lifecycle

```javascript
SKILL_STATUS = {
  ACCEPTED: "accepted",   // robot received the skill request
  RUNNING:  "running",    // skill is executing
  DONE:     "done",       // skill completed successfully
  FAILED:   "failed",     // skill failed; check error field
}
```

### Drive Kinds (advertised in capabilities)

```javascript
DRIVE_KINDS = ["diff", "omni", "holonomic", "arm", "legged"]
```

A robot's `hello.caps.drive` tells the controller its base kinematics. The Policy layer uses this to select the right velocity controller (unicycle for diff/omni, Cartesian for arms, unicycle+strafe for legged bipeds).

### Fault Severities (lowest → highest)

```javascript
SEVERITY = ["info", "warning", "error", "critical"]
```

### Example frames

```json
// 1. Robot handshake
{ "v": 1, "op": "hello", "caps": {
    "protocol": 1,
    "robotKind": "unitree-go2",
    "drive": "diff",
    "dof": 12,
    "skills": ["move", "stop", "navigate", "stand", "sit"],
    "units": { "len": "m", "ang": "rad" },
    "frame": "odom",
    "limits": { "vMax": 1.5, "wMax": 2.0 },
    "rateHz": 50
  }
}

// 2. Controller acceptance
{ "v": 1, "op": "welcome", "accept": true, "controller": "zelpi" }

// 3. Initial pose seeding
{ "v": 1, "op": "pose", "id": "go2-01", "x": 0, "y": 0, "theta": 0 }

// 4. Velocity command (linear 0.5 m/s, rotate 0.1 rad/s)
{ "v": 1, "op": "cmd", "id": "go2-01", "v_": 0.5, "w": 0.1, "seq": 42 }

// 5. Skill request
{ "v": 1, "op": "skill", "id": "go2-01", "skill": "navigate",
  "args": { "target": [10, 5] }, "reqId": "req-001" }

// 6. Telemetry (full)
{ "v": 1, "op": "telemetry", "id": "go2-01", "x": 0.45, "y": 0.12,
  "theta": 0.05, "vx": 0.5, "w": 0.1, "battery": 87, "estop": false, "ts": 1718869200000 }

// 7. Skill done
{ "v": 1, "op": "skillResult", "id": "go2-01", "reqId": "req-001",
  "status": "done", "progress": 1.0 }

// 8. E-stop
{ "v": 1, "op": "estop", "reason": "watchdog timeout" }

// 9. Release
{ "v": 1, "op": "release" }

// 10. Fault
{ "v": 1, "op": "fault", "id": "go2-01", "code": "motor_overheat",
  "message": "left front motor temp 85°C", "severity": "warning" }

// 11. Heartbeat (ping/pong)
{ "v": 1, "op": "ping", "t": 1718869200000 }
{ "v": 1, "op": "pong", "t": 1718869200000 }
```

---

## Units & Coordinate-Frame Contract

**CRITICAL for real hardware:** The protocol uses a single set of units and frame semantics, and **your driver must convert**.

### What the protocol specifies

When a robot sends `hello`, it includes:

```javascript
"units": { "len": "m", "ang": "rad" },
"frame": "odom"
```

- **len** (length unit) — the units for `x`, `y`, `vx` (e.g. `"m"` for meters).
- **ang** (angle unit) — the units for `theta`, `w` (e.g. `"rad"` for radians).
- **frame** (coordinate frame name) — typically `"odom"` (odometry frame, inertial relative to start) or a custom label.

### SimDriver reference

The `SimDriver` (in `server/robot-agent.mjs`) uses:
- Position: meters in the odometry frame (starts at origin).
- Orientation: radians, CCW-positive from the robot's initial heading.
- Velocity: m/s linear, rad/s angular (unicycle model).

### Your driver's responsibility

1. When the controller sends a `cmd` with `v_ = 0.5`, interpret it as **0.5 m/s** (or your advertised unit) in your robot's forward direction.
2. When you `readTelemetry()`, **convert your SDK's frame & units to match your advertised caps**. Example:

```javascript
// SDK returns: { x_mm, y_mm, yaw_deg }  (UR arm in mm/degrees)
readTelemetry(id) {
  const sdk = this.getRobotState(id);
  if (!sdk) return null;
  return {
    x: sdk.x_mm / 1000,      // mm → m
    y: sdk.y_mm / 1000,      // mm → m
    theta: sdk.yaw_deg * Math.PI / 180,  // deg → rad
    vx: sdk.vel_mm_s / 1000,
    w: sdk.ang_vel_deg_s * Math.PI / 180,
    battery: sdk.battery_pct,
    joints: sdk.joint_angles_rad  // already in rad; no conversion
  };
}
```

3. The controller will `setRobotPose(id, x, y, theta)` with pose in your units/frame. Store it as the odometry origin for dead-reckoning.

### Coordinate frame details

- **Odometry frame** (`"odom"`) is inertial (doesn't move with the robot). The robot's pose is the transform from odom → base_link.
- When seeded with `pose(id, 0, 0, 0)`, the robot's initial pose is at odom origin, heading along the positive X axis.
- `theta = 0` means heading East (positive X). Rotation is counterclockwise-positive (right-hand rule about Z axis).

---

## Safety Guarantees & Failure Modes

The protocol is **defensive by design**. Here's what is and isn't guaranteed:

### Guaranteed

1. **E-stop latch** — once `estop` is received, the robot refuses motion until an explicit `release` arrives. The robot-agent enforces this at line 212 in `server/robot-agent.mjs`:
   ```javascript
   if (estopped) {
     send(msg.fault(m.id, "estopped", "cmd ignored while e-stopped", "warning"));
   }
   ```
   Even if a `cmd` slips past the network, the driver's `applyCmd()` sees the latch and zeros velocities (defense-in-depth).

2. **Watchdog safe-stop** — if the controller goes silent for `WATCHDOG_MS` (500 ms default), the robot automatically zeros all velocities. The agent measures silence as any gap in the `CMD`, `PING`, `POSE`, or `SKILL` ops:
   ```javascript
   if (Date.now() - lastControlAt > WATCHDOG_MS) {
     for (const id of ids) driver.applyCmd(id, 0, 0);  // all robots stop
   }
   ```
   This is a **hard real-time guarantee** (wall-clock timeout, not message-count based).

3. **Heartbeat liveness** — the controller sends `ping` every ~500 ms. If the robot doesn't pong back within `LINK_STALE_MS` (1500 ms), the controller sees the link as unhealthy and engages a safe-stop:
   ```javascript
   if (connected && caps && Date.now() - lastPongAt > LINK_STALE_MS) {
     safeStop("link stale");
   }
   ```

4. **Malformed-frame drop** — every frame is validated by `decode()`. If it's not valid JSON, the wrong version, an unknown op, or has wrong field types, it's silently dropped. The process never crashes on bad input.

5. **Protocol version mismatch** — if the robot sends `hello` with a different `protocol` version, the controller rejects it and closes the connection, logging the mismatch. Older agents that never send `hello` fall back to legacy capabilities after a 1.2 s grace period.

### NOT guaranteed

- **Reliable message delivery** — WebSocket is built on TCP, so no packet loss on a local network, but there's no application-layer retry. A single `cmd` dropped at the IP layer is just skipped; the watchdog safeguards against sustained loss.
- **Ordered delivery within a tick** — if two `cmd` messages arrive out of order, the second one (even if older) is applied. You must not rely on strict ordering; use the `seq` field if you need it.
- **Guaranteed skill completion** — a skill can fail (network glitch, driver bug, hardware fault). The robot always sends a terminal `skillResult` (DONE or FAILED).

### Failure modes & recovery

| Scenario | Behavior | Recovery |
|----------|----------|----------|
| Robot offline | Controller auto-reconnects every 1.5 s. Policy layer backs off from driving. | Bring robot online; link auto-establishes. |
| Link stale (no pong) | Controller zeros all cmd, sends `estop`. | Robot is safe-stopped. Controller retries the link. |
| Protocol v1 vs v0 | Controller rejects agent, closes WS. | Upgrade both to v1 or downgrade controller. |
| Malformed frame | Frame dropped (observability only). | Sender retransmits; watchdog protects if sender dies. |
| E-stopped | All cmds refused until `release`. | Operator runs `zelpi hal release`. |

---

## Writing a Real Driver

The `RobotDriver` interface is the SDK seam. To go from the simulator to real hardware, **implement one driver** against your robot's SDK and plug it in. Nothing else in the stack changes.

### The RobotDriver Interface

```typescript
interface RobotDriver {
  // Return the robot's static metadata & constraints
  capabilities(): Capabilities

  // Register a robot's initial pose (odometry origin)
  seedPose(id: string, x: number, y: number, theta: number): void

  // Apply a unicycle velocity to robot `id`
  // Called every control tick (~20 Hz from the server, or on-demand from the controller)
  applyCmd(id: string, v: number, w: number): void

  // Execute a high-level skill. Return terminal status + optional error message.
  async runSkill(id: string, skill: string, args: Object): Promise<{ status: string, error?: string }>

  // Read current odometry & state (or null if unknown)
  readTelemetry(id: string): { x, y, theta, vx?, w?, battery?, joints? } | null

  // Emergency stop: latch motion & refuse cmds until release()
  estop(): void

  // Clear the e-stop latch
  release(): void
}
```

### Unitree G1/H1 — implemented, unverified against real hardware

Unlike the hypothetical skeletons below, this one is real code that ships
in the package: `server/createUnitreeDriver.mjs` + a companion Python
process, `server/drivers/unitree_bridge.py`, wrapping the **official**
`unitree_sdk2py` G1 `LocoClient` (a DDS-based high-level locomotion API).

**Read this before you rely on it:** it was built against
`unitree_sdk2py`'s documented example API
(`example/g1/loco/g1_loco_client_example.py` in Unitree's own repo) —
**no G1/H1 unit was available to test it against.** Every SDK call in the
bridge is wrapped so a mismatched SDK version degrades to a clear fault
message instead of crashing, but the method names/FSM semantics themselves
are unverified until someone runs it live. H1 support (`--robot h1`)
assumes the same `LocoClient`/FSM surface as G1 (shared `unitree_hg` IDL on
newer H1 firmware) — older H1 units on the legacy `unitree_legged_sdk` are
not covered.

Why a Python subprocess bridge instead of a pure-JS driver (unlike
`createRosDriver.mjs`, a plain WebSocket client): `unitree_sdk2py` talks to
the robot over CycloneDDS, which Node can't speak natively and which isn't
exposed over WebSocket the way `rosbridge_suite` is for ROS. The bridge
owns the DDS participant on the robot's companion computer; Node and
Python talk newline-delimited JSON over stdio.

```bash
# once, on the robot's companion computer:
pip install unitree_sdk2py

# g1.json:
# { "robot": "g1", "iface": "eth0", "dof": 29, "limits": { "vMax": 1.0, "wMax": 1.0 } }

node server/robot-agent.mjs --driver unitree --unitree-config g1.json
# then, from the controller:
npx zelpi hal connect ws://<companion-computer>:9091
```

Capabilities advertise `drive: "legged"` (a dedicated drive kind — a
walking, strafing biped doesn't honestly fit `diff`/`omni`/`holonomic`/
`arm`). `dof` has no safe hardcoded default — it varies by hardware
variant (waist joint, hands or not) — set it explicitly in the config.
Skills are deliberately kept to the small, well-attested subset of the
example API: `balanceStand`, `zeroTorque`, `damp` — `estop()` calls
`Damp()` (Unitree's own documented immediate safe-stop); `release()`
clears the local latch only and does **not** re-stand the robot
automatically, since commanding a biped upright unattended is a real
hazard — send an explicit `balanceStand` skill afterward. Telemetry's
`theta` comes from the real IMU yaw when available; `x`/`y` are not real
odometry (the loco client's FSM carries no absolute pose) and are left at
`0` rather than fabricated — attach `zelpi slam attach` for real
localization.

The Node-side translation logic (capabilities, e-stop gating, skill
routing/timeout) has real test coverage against a fake stdio peer —
`test/unitree-driver.test.mjs` — with no Python or `unitree_sdk2py`
install required to run it.

### Hypothetical skeletons for other SDKs

The rest of this section sketches the shape a driver would take for
hardware nobody has built a real integration for yet — useful as a
starting template, not a claim that the code below runs.

**File:** `server/robot-agent.mjs`, add after `createSimDriver`:

Then add it to `selectDriver()`:

```javascript
function selectDriver(name) {
  switch (name) {
    case "sim":
      return createSimDriver();
    case "unitree":
      return createUnitreeDriver(driverConfig);  // real — see above
    // case "xarm":    return createXArmDriver();
    // case "agilex":  return createAgileXDriver();
    default:
      console.warn(`[robot-agent] unknown driver "${name}", falling back to sim`);
      return createSimDriver();
  }
}
```

### Skeleton for xArm (UFACTORY 6-DOF arm)

```javascript
function createXArmDriver() {
  const arm = new XArm({ ip: "192.168.1.100", port: 5001 });
  const robots = new Map();
  
  return {
    capabilities() {
      return {
        protocol: 1,
        robotKind: "xarm6",
        drive: "arm",  // not a base
        dof: 6,
        skills: ["grasp", "release", "move_to_pose"],
        units: { len: "mm", ang: "rad" },  // xArm uses mm
        frame: "tool",  // TCP frame
        limits: { vMax: 500, wMax: 3.14 },  // mm/s, rad/s
        rateHz: 50
      };
    },
    
    seedPose(id, x, y, theta) {
      robots.set(id, { pose: { x, y, theta } });
      // Set the tool's world frame origin
    },
    
    applyCmd(id, v, w) {
      // Arm doesn't take unicycle cmds; ignore or interpret as Cartesian velocity
    },
    
    async runSkill(id, skill, args) {
      if (skill === "grasp") {
        await arm.setGripper(true);
        return { status: "done" };
      }
      if (skill === "release") {
        await arm.setGripper(false);
        return { status: "done" };
      }
      return { status: "failed", error: "unsupported" };
    },
    
    readTelemetry(id) {
      const pose = arm.getToolPose();  // {x_mm, y_mm, z_mm, rx_rad, ry_rad, rz_rad}
      const battery = arm.getBatteryPercent();
      return {
        x: pose.x_mm,
        y: pose.y_mm,
        theta: pose.rz_rad,  // simplified; full pose is 6D
        battery
      };
    },
    
    estop() { arm.estop(); },
    release() { arm.release(); }
  };
}
```

---

## ROS 1 / ROS 2 via rosbridge

You don't need to hand-write a driver for ROS-based hardware — `createRosDriver`
(`server/createRosDriver.mjs`) ships built in. It speaks the standard
[rosbridge v2 JSON protocol](http://wiki.ros.org/rosbridge_suite) over a plain
WebSocket (`advertise`/`subscribe`/`publish`/`call_service`), so **this
process never needs ROS installed** — only a running `rosbridge_websocket`
node on the robot's ROS graph (works identically for ROS 1 and ROS 2).

### 1. Start rosbridge on the robot (if not already running)

```bash
# ROS 2
ros2 launch rosbridge_server rosbridge_websocket_launch.xml

# ROS 1
roslaunch rosbridge_server rosbridge_websocket.launch
```

Many robot stacks already run this for web/teleop UIs — check before starting
a second instance.

### 2. Write a config file describing your robot's topics/services

```json
{
  "url": "ws://192.168.1.50:9090",
  "robotId": "rx1",
  "robotKind": "custom-ros-arm",
  "drive": "arm",
  "dof": 7,
  "skills": ["grasp", "home"],
  "topics": {
    "cmdVel": "/rx1/cmd_vel",
    "odom": "/rx1/odom",
    "initialPose": "/rx1/initialpose"
  },
  "skillServices": { "grasp": "/rx1/grasp_action", "home": "/rx1/home_service" },
  "estopService": "/rx1/estop"
}
```

`topics`/`skillServices`/`estopService` all default sensibly from `robotId`
if omitted (`/<robotId>/cmd_vel`, etc.) — only set what differs from that
convention. `drive` must be one of `diff|omni|holonomic|arm`; `skills` should
list only skills that have a matching entry in `skillServices` (skills without
one fail with a clear "no ROS service mapped" error rather than silently
no-oping).

### 3. Run the agent with the ROS driver, then connect

```bash
node server/robot-agent.mjs --driver ros --ros-config rx1.json
# then, from the controller:
npx zelpi hal connect ws://<robot-agent-host>:9091
```

`applyCmd` publishes `geometry_msgs/Twist` to `topics.cmdVel`; `readTelemetry`
is fed by a `nav_msgs/Odometry` subscription on `topics.odom` (quaternion
decoded to yaw); `runSkill` calls the mapped ROS service via rosbridge's
`call_service` op and waits for `service_response`; `estop`/`release` zero
motion locally immediately (defense-in-depth, same as every other driver) and
additionally call `estopService` if configured. Reconnects automatically with
a 1.5s backoff if the rosbridge connection drops.

### `zelros` — general-purpose ROS CLI (separate from `hal`)

`zelpi hal`/`createRosDriver` only expose the narrow robot-shaped interface
(cmd_vel/odom/skills). For direct ROS work — inspecting arbitrary topics,
publishing a one-off message, calling a service — zelpi ships a `zelros`
command set, usable two ways: as its own global binary (`zelros ...`) or as a
`zelpi` subcommand (`zelpi zelros ...`) — same code path either way, whichever
is more convenient in a given shell/script:

```bash
zelros connect ws://127.0.0.1:9090   # persist the default rosbridge URL
zelros status                        # reachable? node/topic counts via rosapi
zelros topics                        # list topics + types
zelros echo /some/topic              # stream messages (Ctrl-C to stop)
zelros pub /some/topic std_msgs/msg/String '{"data":"hi"}'
zelros call /rosapi/topic_type '{"topic":"/some/topic"}'
zelros scan                          # ROS2 discovery probe, no ROS install needed
```

Same transport as the RobotDriver (rosbridge v2 JSON over WebSocket, no ROS
install needed on this process) — `zelros` is just unconstrained to any
topic/service instead of a pre-configured robot's fixed set.

**Known limitation**: `zelros scan`'s ROS2 discovery probe listens for real
SPDP multicast traffic (verified against a real ROS2 Humble install) — but if
your ROS2 graph runs inside **WSL2** while zelpi runs on Windows, the probe
will report nothing even though ROS2 is genuinely running. This is a WSL2
networking limitation (its NAT doesn't forward multicast across the Windows↔
WSL2 boundary), not a bug in the probe — `zelros connect ws://127.0.0.1:9090`
works fine in this setup since TCP *is* forwarded, just skip discovery and
connect directly.

### Genie Sim (AgibotTech) — Isaac Sim/Omniverse humanoid platform

`zelpi geniesim` bridges [AgibotTech/genie_sim](https://github.com/AgibotTech/genie_sim)
into the same `createRosDriver` path above — it's a config-driven ROS bridge,
not a new driver. What's different about genie_sim is everything upstream of
that bridge: it requires Linux + Docker + an NVIDIA RTX-class GPU + ROS 2
Jazzy + Isaac Sim 5.1/6.0, is not pip-installable, and its own launch file
doesn't start `rosbridge_server` for you. See
[`docs/EXTENDING.md`](EXTENDING.md#genie-sim--agibots-isaac-simomniverse-humanoid-platform)
for the full `install`/`deps`/`up`/`bridge` walkthrough, including the
dependency-installer prompts and a known open question about the
`jointCommand` message type. Not verified end-to-end on any machine used for
this project — see Known Gaps below.

### Floor-Level Material Transfer Robot (Jetson Nano AMR) — Zelantrix reference platform

This is a real, physical robot (per the "Robot Hardware & Software
Integration Brief" prepared for Zelantrix integration review), not a
simulator like Genie Sim. It's a differential-drive AMR built on an NVIDIA
Jetson Nano (128-core Maxwell, quad A57, 4 GB shared RAM, 20 W envelope),
running ROS 2 Humble + CycloneDDS on `ROS_DOMAIN_ID=42` inside a single
privileged, host-networked Docker container alongside navigation,
perception, and motor control. Its own brief states the integration point
plainly: "ROS 2 / CycloneDDS on domain 42 is the existing transport; a new
layer can either join this graph or run alongside it" — this is exactly
`createRosDriver.mjs`'s job, so no new driver code is needed here, only a
config (`sim/configs/material_transfer_robot_ros.json`) plus one thing the
brief doesn't already provide: **`rosbridge_websocket` is not running on
this robot today.**

**Step 1 — expose the existing ROS graph over WebSocket.** Add
`ros-humble-rosbridge-server` to the robot's container image and launch
`ros2 launch rosbridge_server rosbridge_websocket_launch.xml` as an
additional process **inside the same container** the rest of the stack
runs in — not a sidecar container — so it inherits `ROS_DOMAIN_ID=42` and
`RMW_IMPLEMENTATION=rmw_cyclonedds_cpp` from the existing environment and
joins the real graph rather than a second, isolated one. If you add this
to the container's entrypoint/Dockerfile, launch it *after* the existing
CUDA/Tegra/ZED `LD_LIBRARY_PATH` reordering the brief describes for this
image — rosbridge itself doesn't touch those libraries, but sourcing order
in a shared entrypoint script is easy to get backwards.

**Step 2 — configure and connect**, using
`sim/configs/material_transfer_robot_ros.json` as a starting point:

```bash
node server/robot-agent.mjs --driver ros --ros-config sim/configs/material_transfer_robot_ros.json
# then, from the controller:
npx zelpi hal connect ws://<robot-agent-host>:9091
```

`drive: "diff"`, `dof: 0` — this platform has no arm or gripper (payload
capacity is 1 kg carried on the deck, per the brief's physical spec table;
`setJoints` does not apply here). `topics.cmdVel`/`topics.odom` are set to
the un-namespaced `/cmd_vel`/`/odom` (Nav2's own convention) rather than
`createRosDriver`'s `/<robotId>/...` default, since the brief's QoS table
names bare `/cmd_vel` and `/tf`, not a namespaced topic — **confirm both
topic names against this robot's real graph** (e.g. `zelros topics` once
rosbridge is up) before relying on them; the brief explicitly scopes
navigation-stack internals as out of scope, so this is inferred from
convention, not confirmed from source the way genie_sim's topics were.

### `zelpi transporterdemo` — running the pipeline for real

The gaps above (no confirmed rosbridge, no derived limits, no estop/skill
services to wire up) used to just be documented as open questions. They're
not fully closeable without the physical robot, but the software side of
each one now has a working answer, exercised by `zelpi transporterdemo run`
(`cli/transporterdemo.mjs`):

```bash
zelpi transporterdemo run
zelpi transporterdemo run --wheel-diameter 0.12 --track-width 0.35  # your robot's real numbers
```

This runs the **real, unmodified** `createRosDriver.mjs` against
`sim/configs/material_transfer_robot_ros.json` — the same config a real
deployment starts from — with three things resolved:

1. **No rosbridge on the robot yet → babyros stands in.** `zelpi
   transporterdemo` starts a `babyros` broker (identical rosbridge v2 wire
   protocol; `createRosDriver` can't tell the difference) so the ROS-bridge
   half of this integration is exercised against something live instead of
   sitting unverified against a spec sheet.
2. **`vMax`/`wMax` placeholders → computed, not guessed.** `computeLimits()`
   derives them from the brief's own 739530 motor spec (12 V, 150 rpm
   no-load) plus wheel diameter and track width — which the brief genuinely
   doesn't specify, so these two remain **required flags** (defaults of
   0.15 m / 0.42 m are clearly labeled ASSUMED in the run output, not
   presented as real numbers). Pass your robot's actual measurements and
   the limits become real; the formula and the 0.6× load-derate / 0.5×
   teleop-safety margins are printed every run so nothing is hidden in code.
3. **No estop/skill services documented → a working reference pattern.**
   `babyros` gained real generic service routing for this
   (`advertise_service` + forwarded `call_service` in
   `lib/babyros/broker.mjs`, `BabyNode.advertiseService()` in
   `lib/babyros/node.mjs`) — not a demo-only shortcut, a genuine capability
   any babyros node can use now. `lib/babyros/fmtrWorld.mjs`, a simulation
   of this robot's documented differential-drive spec, advertises
   `/fmtr/estop` and `/fmtr/dock`; the demo config points
   `estopService`/`skillServices` at them exactly as a real deployment
   would point them at the real robot's equivalent services once they
   exist. `createRosDriver.estop()`/`runSkill("dock")` call them through
   the identical code path used for every other ROS-backed robot in this
   project — proven by driving the simulated robot, reading its telemetry
   back, calling the dock skill, and confirming e-stop actually halts
   motion, all in one run.

**What this still can't resolve, because it needs the physical robot** —
the run's own output ends with this list, not just this doc:
- The real robot's actual `/cmd_vel`/`/odom` topic names — confirm with
  `zelros topics` against the real rosbridge once it's running, not this
  demo's babyros stand-in.
- Real safe `vMax`/`wMax` — wheel diameter/track width default to labeled
  assumptions; get the commissioned robot's real measurements before
  driving it near people or payload.
- A real ROS-level estop service or motor-power cutoff. The brief documents
  neither — the MDDS30 driver runs directly off the 3S LiPo with no
  intermediate regulation and no described software cutoff path.
  `/fmtr/estop` above only proves the *wiring pattern* against a simulated
  stand-in; it is not a claim that the real robot has an equivalent service.

Two things from the original review remain genuinely unresolved and don't
have a software answer at all:
- **The 4 GB RAM / 20 W envelope is already fully subscribed** by
  navigation, perception inference, motor control, and the existing
  web-backend link, per the brief's own compute section — `rosbridge_websocket`
  itself is cheap (tens of MB, negligible steady-state CPU), but do **not**
  plan to run any zelpi-side inference (VLA forward passes, SLAM, etc.) on
  this same Jetson Nano; treat it as a bridge endpoint only and run compute-
  heavy work off-board, feeding actions in over HAL/ROS the same way
  `vla_bridge.py` does for the sim driver.
- **No isolated sandbox exists on this robot** (the brief states this
  explicitly) — the container rosbridge runs in has full privileged access
  to every device node and Unix group the rest of the stack uses. Exposing
  this ROS graph over a WebSocket means anything zelpi's `ros-config`
  doesn't intentionally touch is still reachable via `zelros` on the same
  URL; scope what you connect to deliberately.

---

## Driving a robot with a VLA

This section covers the real path: download an actual checkpoint, run real
inference in Python, and drive whatever robot is connected via HAL — the sim
driver, a native SDK driver, or the ROS bridge above — all through the exact
same mechanism, since HAL doesn't care where an action comes from.

### 1. Download real weights

```bash
zelpi hub install smolvla       # download ONLY — no fleet-sim registration
zelpi hub pull smolvla           # download, then register in the fleet sim
                                 # (--sim-only skips the download entirely)
                                 # both only work for registry entries with a
                                 # real public checkpoint (hf !== "—" in
                                 # hub.mjs's MODELS[]) — proprietary/
                                 # catalog-only entries (e.g. isaac-wbc) are
                                 # rejected with a clear message instead of
                                 # faking a download.
zelpi hub status                 # lists what's actually on disk
```

Installs into a **dedicated venv** (`~/.zelpi/models/.venv`, or the first
non-system drive with room on Windows — pip's TEMP/cache are redirected onto
the same drive as the install target, same fix already needed for the
HY-World/PyBullet installers) using `huggingface_hub.snapshot_download`
(`sim/scripts/hub_install.py`) — a real download, not a simulation.

**Security note**: `hub install`/`hub pull` also accept an arbitrary raw
HuggingFace repo id, not just registry keys (`resolveModel()` falls back to
treating any unrecognized string as a repo id). This is the same trust model
as `pip install <package>` or `npm install <package>` — you are choosing to
download and run code/weights from a specific repo you named. Reviewed for
this project specifically: `hub_install.py` calls
`huggingface_hub.snapshot_download` with no `trust_remote_code`, and none of
`vla_bridge.py`'s policy-loading path passes it either, so loading a
checkpoint does not execute arbitrary Python from the repo by default (unlike
some HF model classes that require `trust_remote_code=True` explicitly).
Every checkpoint downloaded and loaded during this project's testing used
`.safetensors` weights, not pickle-based `.bin` files, avoiding the
well-known PyTorch-pickle deserialization risk. All spawn calls use argv
arrays (never `shell: true`) so there is no command-injection surface from a
crafted model key or repo id.

### 2. The `setJoints` skill — continuous joint-space control over HAL/ROS

The native protocol's `cmd` message is unicycle-only (`{v, w}`) — it has no
way to carry a VLA's per-joint continuous action. Rather than a protocol
version bump, this is a **skill convention** (the existing `skill`/`args`
mechanism already supports arbitrary shapes):

```
runSkill(id, "setJoints", { joints: number[], gripper?: number })
```

Support is built into:
- **`SimDriver`** (`server/robot-agent.mjs`) — stores the joints, echoes them
  back via `telemetry.joints`, so a bridge script can verify round-trip
  before pointing at real hardware.
- **The ROS driver** (`server/createRosDriver.mjs`) — publishes
  `std_msgs/Float64MultiArray` (joint values, gripper appended last if
  present) to `topics.jointCommand` (default `/<robotId>/joint_command`),
  and feeds `telemetry.joints` from a `sensor_msgs/JointState` subscription
  on `topics.jointStates` (default `/<robotId>/joint_states`). Float64MultiArray
  was chosen over `sensor_msgs/JointState` or a trajectory message for the
  *outbound* side specifically because this generic driver has no way to know
  a given robot's joint names or desired trajectory timing — pair it with a
  small node on the ROS side that maps the array onto your robot's actual
  joint command topic/message type.
- **Any custom SDK driver** — implement `runSkill(id, "setJoints", args)`
  the same way `applyCmd` is implemented today; nothing else in the stack
  changes.

**Safety boundary — read before wiring this to a real arm**: `setJoints`
validates *shape* (every value is a finite number, and — as of this session —
the array length matches the robot's advertised `dof`) but does **not**
clamp *magnitude* or *rate of change* against any physical joint limits.
`applyCmd` has this for velocity (`cfg.limits.vMax`/`wMax`); there is no
equivalent per-joint position/velocity limit here, because HAL is
deliberately robot-agnostic and has no generic way to know a specific arm's
physical range. A VLA outputting a wild or untrained value will be forwarded
as-is. In practice this means: **your robot's own joint controller must be
the actual safety boundary** (firmware/driver-level position and velocity
limits, torque limiting, or a trajectory-smoothing layer on the ROS side) —
do not treat `setJoints`' input validation as a substitute for that.

### 3. Connect Python inference to HAL — `hal_client.py`

`sim/scripts/hal_client.py` is a minimal Python client speaking the
controller side of protocol v1 — the same wire format `cli/hal.mjs` uses —
so a VLA inference loop can connect **directly to a robot-agent** (bypassing
zelpi's own JS engine/fleet entirely) and drive it with real model output.
Needs only the `websocket-client` PyPI package.

```python
from hal_client import HalClient

client = HalClient("ws://127.0.0.1:9091", robot_id="vla-0")
client.connect(timeout=5)
print(client.capabilities)              # {"drive": "arm", "dof": 7, "skills": [...]}

while True:
    obs = client.telemetry                       # {"joints": [...], ...}
    action = my_vla.predict(obs)                  # your real inference call
    client.send_joints(action.joints, gripper=action.gripper)
```

Point `--url`/`HalClient(url)` at **any** robot-agent — the bundled sim driver
for development, or a real robot's native/ROS-bridged agent for the real
thing — the inference loop doesn't change.

### 4. Real inference end-to-end — `vla_bridge.py`

`sim/scripts/vla_bridge.py` is the concrete, runnable proof of the full
chain: it loads a real downloaded checkpoint with `lerobot`, builds an
observation in exactly the shape that checkpoint's own `config.json`
declares, runs a genuine forward pass through the real model weights, and
sends the resulting action into a running robot-agent via `hal_client.py`'s
`setJoints`.

It is **not hardcoded to SmolVLA** — the policy family is auto-detected from
the checkpoint's own saved config (`PreTrainedConfig.from_pretrained` →
`cfg.type` → `get_policy_class`, the same generic mechanism lerobot's own
training/eval scripts use), so the same script works for any lerobot-native
checkpoint pulled via `zelpi hub install`: `smolvla`, `act`,
`diffusion-policy`, `pi0`, …

```bash
# one-time: pull lerobot + its inference deps into the same venv `hub install` made
"~/.zelpi/models/.venv/Scripts/python" -m pip install lerobot transformers tokenizers

node server/robot-agent.mjs --driver sim &
python sim/scripts/vla_bridge.py --checkpoint ~/.zelpi/models/smolvla --hal-url ws://127.0.0.1:9091
```

Some older checkpoints (e.g. `lerobot/act_aloha_sim_insertion_human`,
`lerobot/diffusion_pusht`) were published before lerobot's newer
pre/post-processor-pipeline format and need a one-time migration — lerobot
ships this as a first-party tool, not something this project reimplements:

```bash
python -m lerobot.processor.migrate_policy_normalization --pretrained-path ~/.zelpi/models/act
# writes the migrated checkpoint to ~/.zelpi/models/act_migrated
```

### Verified

Both the protocol/installer plumbing and real model forward passes have been
run against live systems, not mocks — across three distinct policy
architectures, proving `vla_bridge.py`'s auto-detection genuinely generalizes
rather than being tuned to one model's shape:

- `hal_client.py` → sim driver round-trip (`setJoints` values echoed back via
  telemetry) and `hal_client.py` → `createRosDriver.mjs` → real
  rosbridge_websocket → a real independent `rclpy` subscriber node (received
  the exact published `Float64MultiArray`, joints + gripper appended).
- **SmolVLA** (`lerobot/smolvla_base`, flow-matching VLA): real forward pass
  (~56s on CPU for the first denoising call, ~0.02s for subsequent actions
  popped from the same predicted chunk) → 6-DoF action → sim robot-agent
  telemetry via `setJoints`.
- **ACT** (`lerobot/act_aloha_sim_insertion_human`, action-chunking
  transformer): real forward pass (~0.4s on CPU) → 14-DoF action (ALOHA
  bimanual) → sim robot-agent telemetry via `setJoints`.
- **Diffusion Policy** (`lerobot/diffusion_pusht`): real forward pass (~39s
  on CPU for the denoising loop) → 2-DoF action → sim robot-agent telemetry
  via `setJoints`.

**Registry fix**: `cli/hub.mjs`'s `MODELS[]` previously pointed `act` and
`diffusion-policy` at `lerobot/act` / `lerobot/diffusion_policy` and
`groot-n1` at `nvidia/GR00T-N1` — none of these repos actually exist on
HuggingFace (verified via the Hub API, not assumed). They're now pointed at
real, downloadable checkpoints: `lerobot/act_aloha_sim_insertion_human`,
`lerobot/diffusion_pusht`, and `nvidia/GR00T-N1.5-3B`. Note the ACT/Diffusion
Policy checkpoints are **task-specific benchmark checkpoints** (ALOHA
insertion, PushT), not generalists — same caveat as the honest-limitation
note below, just sharper since these were trained on a single simulated
task rather than a broad multi-task mixture. `groot-n1`'s real weights
(5.45 GB) were not run through a live forward pass here (size/compute cost
on this machine) but `hub install groot-n1` uses the identical, already-proven
`snapshot_download` path.

**Flaky-download note**: during this verification, `zelpi hub install`
against the default `huggingface_hub` HTTP client repeatedly hit mid-stream
connection resets from HF's CDN (`us.aws.cdn.hf.co`) on this network,
independent of file size — small text-generation timeouts, then
`IncompleteRead` errors even after raising `HF_HUB_DOWNLOAD_TIMEOUT` /
`HF_HUB_ETAG_TIMEOUT`. Installing the `hf_transfer` PyPI package and setting
`HF_HUB_ENABLE_HF_TRANSFER=1` (a Rust-based chunked downloader
`huggingface_hub` uses automatically when present) resolved it immediately.
If `hub install` fails with connection errors, this is the first thing to
try before assuming the checkpoint or the installer is broken.

**Honest limitation**: a pretrained public VLA checkpoint (SmolVLA, OpenVLA,
π0, …) was trained on a *specific* embodiment's camera views and action
space (e.g. ALOHA, SO-100, LIBERO). Downloading one and pointing it at an
arbitrary robot via `setJoints` will not produce *useful* behavior without
fine-tuning on that robot's own data — this is a property of pretrained VLAs
in general, not a gap in this pipeline. `vla_bridge.py` uses synthetic camera
inputs (no real camera rig here) precisely to isolate and prove the
mechanical path (weights on disk → real forward pass → action → HAL/ROS →
robot) independent of that; getting a specific checkpoint to do something
meaningful on your specific robot is a fine-tuning project on top of it.

### GPU setup and requirements — `zelpi gpu`

A default `pip install torch` gives the **CPU-only** build — on a machine
with a perfectly good NVIDIA GPU, everything silently runs 10-100x slower
than it should. zelpi closes that gap from inside the OS:

```bash
zelpi gpu           # detect the GPU + report whether the models venv's torch can use it
zelpi gpu setup     # swap the venv's torch/torchvision for matching CUDA builds (~2.6 GB)
                    #   --cuda cu128 to pick a different channel · --force to skip GPU detection
```

`hub install`/`hub pull` also offer this automatically after a model
download when they detect a GPU that torch can't use (interactive prompt
defaulting to yes; `--gpu` or `PIOS_AUTO_GPU=1` for unattended installs,
`--no-gpu` to suppress). The interactive shell prints a one-line hint at
boot in the same situation — detected via a fast file probe of torch's own
`version.py`, never by importing torch (which alone takes seconds).

**Measured forward-pass latency** (same checkpoints, same `vla_bridge.py`,
same machine — RTX 3060 Laptop 6 GB vs. its CPU), first call per action
chunk; subsequent actions pop from the predicted chunk in ~0.02s either way:

| Policy | Architecture | CPU | GPU (RTX 3060, 6 GB) |
|---|---|---|---|
| ACT | action-chunking transformer | ~0.4s | ~1.7s first call (CUDA warmup), ~0.02s after |
| SmolVLA | flow-matching VLA | ~56s | **~3.9s** (~14x) |
| Diffusion Policy | diffusion denoising | ~39s | **~4.1s** (~10x) |

All three were verified end-to-end on real GPU hardware (checkpoint → CUDA
forward pass → `setJoints` → robot-agent telemetry). Doing so surfaced and
fixed a real bug the CPU-only runs couldn't catch: a checkpoint's saved
config pins the device it was trained/migrated on, so the preprocessor moved
*inputs* to `cuda` while the *weights* stayed on `cpu` — `vla_bridge.py` now
explicitly moves the policy to the selected device.

For actually closing a control loop against a moving robot, a GPU is not
optional — flow-matching/diffusion policies need well under 100ms per action.
With `chunk_size: 50`, SmolVLA's ~3.9s denoise amortizes to ~80ms/action on
this entry-level 6 GB laptop GPU — borderline usable for slow control loops,
where the CPU's ~1.1s/action amortized is not. ACT is fast enough either way.

---

## Deploying on the Robot

The robot side is **tiny**. You need:

1. **`server/robot-agent.mjs`** — the main agent loop.
2. **`lib/hal/protocol.mjs`** — the shared protocol codec.
3. **`ws` npm package** — WebSocket server (and client, for the ROS driver).

That's it for a native/SDK driver. Using the `ros` driver adds one more file
— **`server/createRosDriver.mjs`** — and needs `rosbridge_websocket` running
on the ROS side (see [ROS 1 / ROS 2 via rosbridge](#ros-1--ros-2-via-rosbridge)
above); no other ROS/DDS footprint either way.

### Minimal setup

```bash
# On the robot (or dev machine for testing):

# 1. Copy the three files
mkdir ~/pi-os-robot
cp server/robot-agent.mjs ~/pi-os-robot/
cp lib/hal/protocol.mjs ~/pi-os-robot/
echo '{"name":"pi-os-robot","version":"1.0.0","type":"module","dependencies":{"ws":"^8.0.0"}}' > ~/pi-os-robot/package.json

# 2. Install the one dependency
cd ~/pi-os-robot
npm install

# 3. Start the agent
node robot-agent.mjs
# Output:
# [robot-agent] native ZelPi robot on ws://127.0.0.1:9091
# (driver: sim, protocol v1, no ROS — watchdog 500ms · e-stop · skills + telemetry)
```

### On real hardware (e.g., Unitree Go2)

1. **SSH into the robot**:
   ```bash
   ssh unitree@192.168.1.220
   ```

2. **Copy the files** (scp or git):
   ```bash
   scp -r pi-os-robot unitree@192.168.1.220:~/
   ```

3. **Edit `robot-agent.mjs`** to select your driver:
   ```javascript
   const driverName = parseDriverName(process.argv.slice(2));
   // e.g., driverName = "unitree" (hardcode for convenience)
   ```

4. **Start the agent**:
   ```bash
   cd ~/pi-os-robot
   npm install  # first time only
   node robot-agent.mjs --driver unitree
   # or: PIOS_ROBOT_DRIVER=unitree node robot-agent.mjs
   ```

5. **From the controller machine**, connect:
   ```bash
   zelpi hal connect ws://192.168.1.220:9091
   ```

### Environment variables

| Env | Default | Purpose |
|-----|---------|---------|
| `PORT` | 9091 | WebSocket server port on the robot. |
| `PIOS_ROBOT_DRIVER` | `"sim"` | Driver name (`sim`, `unitree`, `xarm`, etc.). |

---

## Controller Integration

The embedded backend automatically connects to a robot when you configure it.

### Environment

| Env | Purpose |
|-----|---------|
| `PIOS_ROBOT_URL` | e.g. `ws://192.168.1.220:9091`. Loaded by `createEmbeddedServer()` in `cli/embedded.mjs`. |

### Command-line (ZelPi CLI)

```bash
# Connect
zelpi hal connect ws://192.168.1.220:9091

# Disconnect (back to sim)
zelpi hal disconnect

# E-stop (latch motion)
zelpi hal estop "manual" 

# Release (resume)
zelpi hal release

# View status
zelpi hal

# List skills & capabilities
zelpi hal skills
```

### Programmatic

```javascript
import { createRobotLink } from "./cli/hal.mjs";

const link = createRobotLink(engine, { url: "ws://192.168.1.220:9091" });

// Check status
const status = link.status();
console.log(status);
// {
//   enabled: true,
//   connected: true,
//   driver: "native-ws",
//   url: "ws://192.168.1.220:9091",
//   caps: { robotKind: "go2", drive: "diff", ... },
//   lastTelemetry: { id: "go2-01", x: 1.5, y: 2.0, theta: 0.1, battery: 87, ... },
//   linkHealthy: true,
//   estopped: false,
//   skills: ["move", "stop", "navigate", ...],
// }

// E-stop
link.estop("safety halt");

// Release
link.release();

// Graceful shutdown
link.stop();
```

---

## `zelpi hal` Command Reference

### `zelpi hal`

Show link status, driver, telemetry, e-stop state.

```bash
$ zelpi hal
ZelPi · Hardware Abstraction Layer  (native · no ROS)
  Driver: native-ws
  Link: ● healthy · ws://192.168.1.220:9091
  Robot: go2 (drive diff · 12 DoF · 50Hz)
  Telemetry: go2-01 @ (1.45, 2.30) θ0.05 · 🔋87% · 45ms ago
  E-stop: clear
  Skills: move, stop, navigate, stand, sit
  Config: ws://192.168.1.220:9091
```

### `zelpi hal connect <url>`

Link the controller to a robot agent.

```bash
$ zelpi hal connect ws://192.168.1.220:9091
→ linking HAL → ws://192.168.1.220:9091 (restarting backend, no ROS)
✓ robot linked · · 5 skills
```

If the link doesn't establish, the status will show `○ link pending`:

```bash
$ zelpi hal connect ws://192.168.1.220:9091
✓ robot linked · · 5 skills

$ zelpi hal
Link: ○ link pending — is the robot agent running at ws://192.168.1.220:9091?
      (node server/robot-agent.mjs)
```

### `zelpi hal disconnect`

Revert to the sim driver (software twin).

```bash
$ zelpi hal disconnect
→ disconnecting HAL → sim driver (restarting backend)
✓ HAL on sim driver (software twin).
```

### `zelpi hal estop [reason]`

Engage the emergency stop. Motion is refused immediately; the robot safe-stops.

```bash
$ zelpi hal estop "manual operator intervention"
✗ E-STOP ENGAGED · manual operator intervention
  Link: ● motion zeroed, commands refused
```

### `zelpi hal release`

Clear the e-stop latch and resume driving.

```bash
$ zelpi hal release
✓ e-stop released — commands resumed
  Link: ● driving
```

### `zelpi hal skills`

Show advertised skills and hardware capabilities.

```bash
$ zelpi hal skills
ZelPi · HAL — supported skills & capabilities
  Skills: move, stop, navigate, rotate, stand, sit
  Robot: go2 · drive diff · 12 DoF
  Units: m / rad · frame odom
  Limits: vMax 1.5 · wMax 2.0 · 50Hz
```

---

## Bring-Up Checklist & Troubleshooting

### Checklist

- [ ] Robot agent running: `node server/robot-agent.mjs --driver <name>` on the robot (or test machine).
- [ ] Network connectivity: `ping <robot-ip>` from controller machine.
- [ ] WebSocket port open: `telnet <robot-ip> 9091` (or use browser DevTools to test WS handshake).
- [ ] Protocol version match: agent sends `hello` with `protocol: 1`; controller accepts or rejects.
- [ ] Capabilities sensible: check `zelpi hal skills` shows robot kind, drive, limits.
- [ ] E-stop clear: start with `zelpi hal release` (persisted intent clears on startup).
- [ ] Telemetry flowing: `zelpi hal` shows recent timestamp and valid pose.
- [ ] Motion safe: stand-alone test: `zelpi intent "move forward"` with operator standing by.

### Troubleshooting

| Problem | Cause | Solution |
|---------|-------|----------|
| `Link: ○ link pending` | Agent not running or unreachable. | SSH into robot; confirm `node robot-agent.mjs` is running. Check firewall (port 9091). |
| `Link stale (no pong)` | Agent not responding to heartbeat. | Check network latency; agent may be blocked by CPU-intensive work. Reduce heartbeat interval if needed. |
| `E-STOP ENGAGED` | E-stop was latched (CLI or watchdog). | Operator runs `zelpi hal release`. Clear the reason: check logs for watchdog timeout or manual trigger. |
| `Protocol mismatch: agent v0 ≠ controller v1` | Agent is old version. | Upgrade agent: `npm install` on robot to get latest `robot-agent.mjs` + `hal/protocol.mjs`. |
| `Telemetry stale (>1s)` | Agent telemetry loop slow or blocked. | Check agent logs; reduce driver overhead (e.g., SDK calls blocking). Increase `TELEMETRY_HZ` if robot can sustain it. |
| Robot won't move | Could be: e-stop latched, watchdog timeout, or `applyCmd` not reaching SDK. | (1) `zelpi hal` — is e-stop clear? (2) Check robot logs. (3) Test SDK directly (`python unitree_go2_test.py` or similar). |
| `Malformed frames dropped` | Bad JSON from agent or controller. | Rare; check agent logs for encoding errors. Confirm encoding is UTF-8. |
| Pose updates erratic | Odometry frame mismatch or unit conversion bug. | (1) Verify units: `zelpi hal skills` shows `units`. (2) In driver, check `readTelemetry()` — is frame origin correct? (3) Run sim first to baseline: `zelpi hal disconnect`. |

---

## Alternative: Rosbridge (deprecated but supported)

The legacy ROS 2 rosbridge link (`server/rosbridge.ts`, `PIOS_ROS=on`) still works. Use it only if:

- You already have ROS 2 + Gazebo installed.
- You need DDS compatibility with other middleware.
- You want to leverage existing ROS 2 packages (navigation, manipulation, etc.).

**Recommended:** use the native HAL for new deployments. It's faster, simpler, and runs everywhere Node does.

To switch to rosbridge:

```bash
npm run sim:robot       # mock rosbridge server
npm run server:ros      # PIOS_ROS=on ROSBRIDGE_URL=ws://127.0.0.1:9090
```

See `docs/BACKEND_PHASES.md` (Phase 4) for details.

---

## Reference: Key Tuning Parameters

These live in `server/robot-agent.mjs` and `cli/hal.mjs`:

| Parameter | File | Default | Meaning |
|-----------|------|---------|---------|
| `PORT` | `robot-agent.mjs` | 9091 | WebSocket server port on robot. |
| `CONTROL_DT` | `robot-agent.mjs` | 0.05 s | Control loop period (20 Hz). |
| `TELEMETRY_HZ` | `robot-agent.mjs` | 20 | Telemetry publish rate. |
| `WATCHDOG_MS` | `robot-agent.mjs` | 500 | Link silence before safe-stop. |
| `DRIVE_HZ` | `cli/hal.mjs` | 15 | Velocity command rate from controller. |
| `HEARTBEAT_MS` | `cli/hal.mjs` | 500 | Ping cadence (~2 Hz). |
| `LINK_STALE_MS` | `cli/hal.mjs` | 1500 | No pong → link unhealthy. |

Increase `WATCHDOG_MS` if the link has latency bursts; decrease if you want faster safe-stop. Decrease `HEARTBEAT_MS` for tighter link health (more traffic). Tune `TELEMETRY_HZ` to robot odometry frequency (higher = more responsive, more bandwidth).

---

## Known Gaps — read this before a production decision

Everything above is accurate about what's been built and verified. This
section is the other half: what hasn't, stated plainly rather than left
implicit.

- **No physical robot has ever been connected.** Every verification in this
  document is sim-driver-to-sim-driver, or the ROS bridge against a real but
  independent `rclpy` node standing in for a robot — never an actual motor,
  actual camera, or actual arm. The protocol and the ROS wire format are
  real; a real robot closing the loop is not something this project has
  done. If you're evaluating this for a production robot, budget time for
  a first real hardware integration — treat it as unstarted, not "mostly
  done."
- **The Zelantrix floor-level material transfer robot integration has a
  verified software pipeline, not a verified real robot.** `zelpi
  transporterdemo run` genuinely exercises `createRosDriver.mjs` against a
  simulation of this robot's documented motor spec, over a real babyros
  broker standing in for `rosbridge_websocket`, including a working
  estop/dock service round-trip — that part isn't paper-only anymore. What
  remains unverifiable without the physical unit: the real robot's actual
  `/cmd_vel`/`/odom` topic names (inferred from Nav2 convention here, not
  confirmed against its graph — navigation internals were out of scope of
  the brief), real safe `vMax`/`wMax` (wheel diameter/track width are
  labeled assumptions, overridable via `--wheel-diameter`/`--track-width`),
  and any real ROS-level estop or motor-power cutoff (the brief documents
  none). See the "Floor-Level Material Transfer Robot" section above.
- **`setJoints` has no physical safety clamp.** See the note earlier in this
  doc — shape validation only, no magnitude/velocity/torque limiting. Your
  robot's own controller has to be the real safety boundary.
- **The Unitree G1/H1 driver is untested against real hardware, full stop.**
  It's real code against the official `unitree_sdk2py` example API, not a
  hypothetical skeleton like the ones below it — but no G1/H1 unit was ever
  connected. Method names, FSM semantics, and H1's assumed compatibility
  with the G1 client are all unverified until someone runs it live. Only
  the Node-side translation logic (e-stop gating, skill routing) has real
  test coverage; the SDK boundary itself does not.
- **GPU inference is now verified, but only on one card.** All three policy
  families ran real CUDA forward passes on an RTX 3060 Laptop (6 GB) — see
  the `zelpi gpu` section above for measured numbers. Other GPUs, multi-GPU,
  and non-NVIDIA accelerators remain untested; `zelpi gpu setup` only
  handles the NVIDIA/CUDA path.
- **No CI existed before this review** (now added: `.github/workflows/test.yml`),
  and the model registry had three entries pointing at HuggingFace repos that
  don't exist, undetected until manually checked. Treat any registry
  entry or documented capability with appropriate skepticism until you've
  personally run it — this project's own history is evidence that
  unverified claims do creep in.
- **The zero-config hosted LLM path (`pios` proxy) has a real, currently
  ~1-in-9 residual failure rate** even after adding a retry — Google's
  Gemma upstream is intermittently flaky at a rate this project doesn't
  control. Configure your own API key (`GOOGLE_API_KEY`/`OPENAI_API_KEY`)
  for anything you depend on working reliably.
- **This has been tested on Windows + WSL2 by one person, on one machine.**
  No multi-user, multi-fleet, or sustained-load testing has occurred. Linux
  and macOS get CI coverage now but no manual verification of the
  hardware-facing pieces (HAL, ROS bridge, hub install) has happened there.
- **`zelpi geniesim`'s `install`/`deps` path is now verified end-to-end on
  real Linux (WSL2 Ubuntu 22.04, real Docker + NVIDIA Container Toolkit + GPU
  passthrough all confirmed working) — but `geniesim up`'s actual Docker
  image build has still never completed on any machine used for this
  project.** The real verification run surfaced and fixed three genuine
  issues, not silent failures: (1) `geniesim_cli`'s Windows checkout breaks
  `pip install -e` because a symlinked `VERSION` file checks out as a broken
  text placeholder — `geniesim_install.py` now repairs this automatically;
  (2) stock Debian/Ubuntu has no bare `python` binary (only `python3`) —
  `cli/geniesim.mjs` now prefers `python3` on non-Windows (the same
  hardcoded-`"python"` bug still exists in `hyworld`/`lingbot`/`gpu setup`,
  unfixed, since it wasn't a blocker for those); (3) `geniesim doctor`'s own
  interactive "apply fixes" prompt could cascade into a full, uncontrolled
  `geniesim bootstrap` stack install on EOF/closed stdin — `zelpi geniesim
  doctor` now always declines it, staying strictly read-only. Also required
  but zelpi-side-uninstalled: `apt install python3.<minor>-venv` (a small,
  standard OS package) for `python -m venv` to bootstrap pip at all. The
  `/joint_command` message shape was also corrected from a speculative guess
  (`trajectory_msgs/JointTrajectory`) to the real, source-verified one
  (`sensor_msgs/JointState`, keyed by joint name — see `docs/EXTENDING.md`
  § Genie Sim), with `createRosDriver.mjs` now supporting it explicitly.
  Still genuinely unverified: `geniesim up`'s `docker build` (large, untested
  on this project — Isaac Sim's base image may also require an NVIDIA NGC
  login, unconfirmed), `docker up`/`ros2 launch genie_sim_bringup`, the
  in-container rosbridge install, and an actual `setJoints` round-trip
  against a live genie_sim instance. The one GPU available for this testing
  (RTX 3060, 6 GB) is also below Isaac Sim's typical recommendation. Treat
  the Docker image build and everything downstream of it as unstarted
  integration work, not "mostly done," until someone with a beefier GPU
  actually runs it.

None of this means the underlying design is wrong — the protocol, the driver
interface, and the ROS bridge's architecture all held up under real testing
this session. It means the distance between "this architecture is sound" and
"this is production-ready" is still real, and mostly in hours of hardware
time and reliability engineering, not further design work.

---

## Quick Start Recap

1. **Verify network**: `ping <robot-ip>`
2. **Start agent**: `node server/robot-agent.mjs --driver <your-driver>` on robot.
3. **Connect controller**: `zelpi hal connect ws://<robot-ip>:9091`
4. **Check status**: `zelpi hal` (should show "healthy" and recent telemetry).
5. **Test motion**: `zelpi intent "move forward 1 meter"` with operator ready to `zelpi hal estop`.
6. **Deploy**: wrap agent startup in a systemd service or robot startup script so it auto-starts.

---

**Questions?** The protocol is versioned and stable. The driver interface is clean. The watchdog and e-stop are rock-solid. You can deploy real robots with confidence.