# EngramX + OpenClaw Integration

Hybrid memory plugin for [OpenClaw](https://github.com/openclaw/openclaw): local SQLite for fast search, [EngramX](https://engramx.ai) canister as the persistent source of truth, and periodic database backup to the canister for disaster recovery. Auto-detects active databases (SQLite, QMD, LanceDB) from `openclaw.json`.

## Architecture

The canister exposes two storage systems: **Files** (memory markdown + session JSONL) and **Blobs** (SQLite backup chunks). Sessions are stored as JSONL files under `sessions/{agentId}/{key}.jsonl` — no separate session API.

```
┌──────────────────────────────────────────────────────────────────────┐
│                          OpenClaw Gateway                           │
│                                                                     │
│  memory_search / memory_get ─────► Local SQLite (sqlite-vec + FTS5) │
│       (sub-100ms hybrid search)         ▲                           │
│                                         │ file watch + re-index     │
│  engram_append_memory ──► EngramX ──────┤                           │
│       (source of truth)   canister      │ sync to local disk        │
│                              │          ▼                           │
│                              │     memory/*.md (local files)        │
│                              │     sessions/{agentId}/*.jsonl       │
│                              │                                      │
│  periodic backup ────────────┤                                      │
│       (every 6h)             │     detected DBs ───────► EngramX    │
│                              │     (SQLite/QMD/LanceDB, 1.9 MiB chunks)│
│                              │                                      │
│  engram_wallet_balance ──────┘     Direct canister access (wallet)  │
└──────────────────────────────────────────────────────────────────────┘
```

### Why hybrid?

| Concern | Local SQLite | EngramX Canister |
|---|---|---|
| **Search latency** | 50-200ms (vector + FTS5) | Not available (search removed) |
| **Write latency** | <1ms | 2-5s (ICP consensus) |
| **Persistence** | Lost if machine dies | Survives anything (on-chain) |
| **Vector search** | sqlite-vec cosine distance | Not available |
| **Disaster recovery** | None | Backup API (500 MiB max) |

The SQLite database is a **derived cache** — it's rebuilt from markdown files + an embedding provider. EngramX stores the source markdown. If you lose your machine, pull the files from EngramX and the SQLite index rebuilds automatically.

## What it provides

### Search tools (local SQLite — fast)

| Tool | Description |
|---|---|
| `memory_search` | Hybrid vector + FTS5 search across memory (sub-100ms) |
| `memory_get` | Read a local memory file by path |

These are identical to OpenClaw's built-in `memory-core` tools. Search never hits the canister.

### Engram tools (canister — persistent)

| Tool | Description |
|---|---|
| `engram_append_memory` | Append to a memory file (writes to canister + local disk). Owner-protected paths are read-only for operators. |
| `engram_read_memory` | Read a file directly from the canister |
| `engram_list_memory` | List all files in the canister (includes session JSONL files) |
| `engram_wallet_balance` | Check engram wallet balance |

> No `engram_transfer` / write / delete / rollback tools — operators are append-only
> and have no owner-level fund control by design.

Sessions are stored as regular files (`sessions/{agentId}/{key}.jsonl`) and are visible via `engram_list_memory`. There is no separate session tool.

### Background services

- **Memory sync** — pulls markdown files from canister to local disk on startup and periodically (default: every 60 min). Local writes are pushed back to the canister.
- **Database backup** — auto-detects active databases from `openclaw.json` and uploads snapshots periodically (default: every 6 hours). Uses `sqlite3 .backup` for consistent SQLite snapshots, `tar.gz` for directory-based backends (QMD, LanceDB). SHA-256 deduplication skips unchanged databases.
- **Session sync — REMOVED (2026-08-09).** The engram is a curated record, not a transcript archive: raw conversation logs generate unbounded cost against the capped rent and are the largest prompt-injection surface, so this plugin no longer ships the affordance. If you deliberately want session archives on-chain, write them yourself via the SDK (e.g. `SessionManager`) — tagged `sessionTranscript`, reviewable and repudiable by the owner — and budget for what that costs.

### CLI commands

```
openclaw memory search <query>    # Search local SQLite (fast)
openclaw engram status            # Canister status
openclaw engram sync              # Pull/push memory files now
openclaw engram backup            # Backup all detected databases now
openclaw engram restore           # List available backups (grouped by type)
openclaw engram files             # List canister memory files (includes sessions)
```

---

## Option 1: Hosted (engramx.ai)

Use this when your engram is running on the IC mainnet via [engramx.ai](https://engramx.ai).

### 1. Create an engram

Sign up at [engramx.ai](https://engramx.ai) and create an engram. Note your **canister ID**.

### 2. Create an operator invite

In the engramx.ai dashboard, go to Access > Operators and click "Create Invite".

### 3. Pair and install

```bash
# First host — push local memory to engram
npx @engramx/client pair <INVITE_CODE> --engram <CANISTER_ID> --openclaw

# Replacement host — restore from engram
npx @engramx/client pair <INVITE_CODE> --engram <CANISTER_ID> --openclaw --recover
```

This generates a session key, registers the operator, installs the plugin, configures `openclaw.json`, and either migrates local files or restores database backups. A local checkpoint is saved first — run `engramx rollback` to undo.

### 4. Verify

```bash
openclaw engram status
```

### Manual configuration

If you need to customize the plugin config, edit `~/.openclaw/openclaw.json`:

```json5
{
  "plugins": {
    "slots": {
      "memory": "memory-engramx"
    },
    "entries": {
      "memory-engramx": {
        "canisterId": "xxxxx-xxxxx-xxxxx-xxxxx-xxx",
        // Memory file sync: canister <-> local disk
        "sync": {
          "onStartup": true,        // pull files when OpenClaw starts
          "intervalMinutes": 60,     // sync every hour
          "pushOnWrite": true        // write to both canister and local
        },
        // Database backup: auto-detects SQLite/QMD/LanceDB
        "backup": {
          "enabled": true,
          "intervalMinutes": 360     // backup every 6 hours
        }
      }
    }
  }
}
```

---

## Option 2: Local Development

Use this to develop and test against a local ICP replica before `@engramx/client` is on npm.

### Prerequisites

- [Node.js](https://nodejs.org/) >= 22.12
- [pnpm](https://pnpm.io/) 10.x
- [ICP CLI](https://github.com/nicoll-douglas/icp-cli) (`icp`)
- The [engramx](https://github.com/vhew/engramx) repo cloned locally
- The [openclaw](https://github.com/openclaw/openclaw) (OpenClaw) repo cloned locally

### 1. Start EngramX locally

```bash
cd /path/to/engramx
pnpm local:start
pnpm local:create-engram    # note the ENGRAM_ID
```

### 2. Configure protected paths (optional)

Designate which file paths are owner-protected (read-only for operators):

```bash
icp canister call <ENGRAM_ID> setProtectedPaths '(vec { "SOUL.md"; "USER.md"; "AGENTS.md" })' -e local
```

### 3. Build and link `@engramx/client`

```bash
cd /path/to/engramx/packages/client
pnpm install && pnpm build
pnpm link --global
```

### 4. Set up the plugin in OpenClaw

```bash
cp -r /path/to/engramx/integrations/openclaw /path/to/openclaw/extensions/memory-engramx
cd /path/to/openclaw/extensions/memory-engramx
pnpm install
pnpm link --global @engramx/client
```

### 5. Create an operator and pair

```bash
# Create an invite (as the engram owner)
icp canister call <ENGRAM_ID> createOperatorInvite \
  '("openclaw-dev", record { canRead = true; canWrite = true; canTransfer = false; dailySpendingLimitE8s = opt 0 })' \
  -e local

# Pair and configure OpenClaw in one step
npx @engramx/client pair <INVITE_CODE> --engram <ENGRAM_ID> --openclaw --host http://localhost:4943
```

The `--openclaw` flag installs the plugin config in `~/.openclaw/openclaw.json` automatically.

### 6. Run OpenClaw

```bash
cd /path/to/openclaw
pnpm gateway:watch
```

### 7. Verify

```bash
openclaw engram status
openclaw engram sync
openclaw engram backup    # triggers backup of all detected databases
```

### Resetting local state

```bash
cd /path/to/engramx
pnpm local:reset --nuke && pnpm local:start
# Repeat steps 1 and 5 to recreate engram + operator
```

---

## Configuration Reference

| Key | Type | Default | Description |
|---|---|---|---|
| `canisterId` | string | *required* | Engram canister ID (also reads `ENGRAM_CANISTER_ID` env) |
| `sessionKeyPath` | string | `~/.engramx/session.key` | Path to Ed25519 operator session key |
| `host` | string | `https://ic0.app` | ICP replica host |
| `memoryDir` | string | `memory` | Local directory for memory markdown files |
| `sync.onStartup` | boolean | `true` | Pull memory files from canister on startup |
| `sync.intervalMinutes` | number | `60` | Periodic sync interval (0 = disabled) |
| `sync.pushOnWrite` | boolean | `true` | Write to both canister and local disk |
| `backup.enabled` | boolean | `true` | Enable periodic database backup (auto-detects SQLite/QMD/LanceDB) |
| `backup.intervalMinutes` | number | `360` | Backup interval in minutes (minimum 30) |
| `backup.dbLabel` | string | auto | Label for backup snapshots |

## Environment Variables

| Variable | Description |
|---|---|
| `ENGRAM_CANISTER_ID` | Fallback canister ID if not set in config |
| `ENGRAM_SESSION_KEY_PATH` | Override session key path |
| `OPENCLAW_STATE_DIR` | Override `.openclaw` directory location |

## Disaster Recovery

If you lose your machine:

```bash
# On a clean machine — create a new operator invite from the dashboard, then:
npx @engramx/client pair <INVITE_CODE> --engram <CANISTER_ID> --openclaw --recover
```

This registers a new operator, installs the plugin, restores the latest database backups (SQLite, QMD, LanceDB — whichever were detected), and saves a checkpoint. Memory files sync automatically on next OpenClaw startup.

A local checkpoint is saved before any changes. Run `engramx rollback` to undo.

The only thing that needs regeneration is the embedding vectors, which are computed locally by the embedding provider (OpenAI, Gemini, etc.). The embedding cache in the database backup avoids re-computing embeddings for unchanged content.

### Manual recovery

If you prefer fine-grained control:

1. Set up OpenClaw + this plugin on a new machine
2. Run `openclaw engram sync` — pulls all memory files from the canister
3. Run `openclaw engram restore` — lists available backups grouped by database type
4. Download and restore via the client SDK's backup API