# OpenClaw Scheduler

[![CI](https://github.com/amittell/openclaw-scheduler/actions/workflows/ci.yml/badge.svg)](https://github.com/amittell/openclaw-scheduler/actions/workflows/ci.yml)
[![License](https://img.shields.io/badge/license-MIT-blue)](LICENSE)
[![Node](https://img.shields.io/badge/node-22%20%7C%2024%20%7C%2025%20%7C%2026-green)](https://nodejs.org)

A durable continuity and governed-workflow sidecar for [OpenClaw](https://openclaw.ai) agents and shell workflows. Current OpenClaw already provides [SQLite cron state, command jobs, retries, and run history](https://docs.openclaw.ai/cli/cron), plus [background tasks](https://docs.openclaw.ai/automation/tasks), [Task Flow](https://docs.openclaw.ai/automation/taskflow), and [Lobster approval checkpoints](https://docs.openclaw.ai/tools/lobster). Use this scheduler when shell jobs must survive Gateway downtime, the scheduler needs a separate failure domain, workflows need direct conditional job graphs, or `@amittell/agentcli` needs its durable runtime target.

This is an independent community project and is not affiliated with the OpenClaw project.

It can coexist with native OpenClaw cron. Do not run the same job in both systems.

**Release repo:** `github.com/amittell/openclaw-scheduler`
**Development mirror:** `writhub.io/alexm/openclaw-scheduler`
**Default location:** `~/.openclaw/scheduler/`
**Service:** `ai.openclaw.scheduler` (macOS launchd: LaunchAgent or LaunchDaemon)
**Runtime:** Node.js 22, 24, 25, or 26 (ESM), SQLite via `better-sqlite3`, cron parsing via `croner`
**Schema:** version 28
**Tests:** run with `npm test` (legacy suite, isolated focused tests, docs, and agentcli integration when a checkout is present; hosted CI requires a pinned public checkout)
**Platform:** macOS · Linux · Windows (WSL2)

In practice, this gives you:
- shell jobs that still work when the Gateway is unhealthy
- a scheduler failure domain separate from the Gateway
- AI jobs that stay isolated from your personal chats
- conditional chains, fenced cancellation, atomic approvals, and transactional external delivery

---

## Table of Contents

1. [Why This Exists](#why-this-exists)
2. [Concrete Use Cases](#concrete-use-cases)
3. [When To Use It](#when-to-use-it)
4. [Choosing a Runtime](#choosing-a-runtime)
5. [Quick Start](#quick-start)
6. [Five-Minute Setup](#five-minute-setup)
7. [Starter Recipes](#starter-recipes)
8. [Common Migrations](#common-migrations)
9. [Platform Support](#platform-support)
10. [Architecture](#architecture)
11. [How Jobs Execute](#how-jobs-execute)
12. [Delivery Modes](#delivery-modes)
13. [Delivery Aliases](#delivery-aliases)
14. [Shell Jobs](#shell-jobs)
15. [HITL Approval Gates](#hitl-approval-gates)
16. [Idempotency](#idempotency)
17. [Context Retrieval](#context-retrieval)
18. [Task Tracker](#task-tracker)
19. [Resource Pools](#resource-pools)
20. [Workflow Chains](#workflow-chains)
21. [Retry Logic](#retry-logic)
22. [Chain Safety](#chain-safety)
23. [Inter-Agent Messaging](#inter-agent-messaging)
24. [Backup & Recovery](#backup--recovery)
25. [Agent Registry](#agent-registry)
26. [Database Schema](#database-schema)
27. [CLI Reference](#cli-reference)
28. [Configuration](#configuration)
29. [Service Management](#service-management)
30. [Error Handling & Backoff](#error-handling--backoff)
31. [Migration & History](#migration--history)
32. [Upgrading](#upgrading)
33. [Removing the Scheduler](#removing-the-scheduler)
34. [Best Practices](#best-practices)
35. [File Reference](#file-reference)
36. [Testing](#testing)
37. [Sub-agent Dispatch](#sub-agent-dispatch)
38. [Working with agentcli](#working-with-agentcli)
39. [Trust Architecture](#trust-architecture)
40. [Troubleshooting](#troubleshooting)
41. [Companion Scripts](#companion-scripts)

---

## Why This Exists

OpenClaw's built-in cron and Task Flow are the right default for many automations. This project addresses a narrower operator need: continuity outside the Gateway process and one local runtime for conditional shell and agent graphs.

The pain usually looks like this:

- A shell script is operationally important, but it should keep running even if the gateway is unhealthy.
- One step should trigger another, but only on success, only on failure, or only if the output contains a specific signal.
- A risky action needs a human in the loop instead of firing immediately.
- An agent needs to hand work to another agent or process, and you want that handoff tracked and auditable.

`openclaw-scheduler` exists to solve those problems without replacing the native runtime for ordinary cron work. It provides jobs, runs, chains, shell execution, approvals, a delivery outbox, and message routing backed by a separate SQLite database.

## Concrete Use Cases

These are the kinds of workflows this scheduler is meant for:

- `metrics capture -> analysis -> approval -> report publish`
  You want each step tracked, retried if needed, and gated before the final action.
- `shell ingest fails -> agent diagnoses failure -> operator approves remediation`
  The ingest should still run without the gateway, but the failure follow-up can use an agent.
- `workspace audit -> diagnosis -> memory compression`
  The audit is a shell step, the diagnosis is an agent step, and the remediation should only run if the diagnosis actually recommends it.
- `bot health check -> alert -> repair action`
  A shell check runs on schedule, an agent summarizes the issue, and a repair step waits for approval.

The differentiator is not just "better cron". It is mixed shell + agent workflows with durable state and control over what happens after success, failure, timeout, or explicit signals in output.

## When To Use It

Use it when you want:

- shell jobs that do not depend on OpenClaw gateway availability
- an independently operated scheduler and database
- parent/child workflow chains
- approval gates before risky steps
- auditable inter-agent or agent-to-shell handoffs

Do not use it merely to obtain run history, command jobs, or basic retries. Native OpenClaw provides those capabilities. If all you need is one scheduled command or agent turn, this is probably more system than you need.

## Choosing a Runtime

> If you have never used OpenClaw's built-in cron, skip migration and go directly to [Five-Minute Setup](#five-minute-setup).

| Requirement | Recommended owner |
|-------------|-------------------|
| One command or agent turn on a schedule | Native OpenClaw cron |
| Native restart-safe flow or resumable approval checkpoint | OpenClaw Task Flow or Lobster |
| Shell execution while the Gateway is unavailable | OpenClaw Scheduler |
| Direct parent/child triggers on result or output content | OpenClaw Scheduler |
| Declarative governed runbook manifest | `@amittell/agentcli` targeting OpenClaw Scheduler |

---

## Version 0.5.0 Runtime Safety

- A singleton dispatcher lease and fencing token prevent a second dispatcher from taking ownership of live work.
- Active runs carry dispatcher ownership. Only the owning fence can commit the terminal transition or trigger downstream work.
- Cancellation is a durable request. Shell jobs terminate their tracked process group; agent jobs record and send the Gateway cancellation request. Check `runs get` or `status` for authoritative state.
- Expired dispatch claims return to the pending queue only when no active run owns them.
- Run completion, job state, and child dispatch creation commit atomically.
- External delivery uses a transactional outbox that is separate from agent prompt messages. Completion ownership is scoped by run, multipart deliveries have independently retryable per-part checkpoints, and completion debt closes only after every part is actually delivered. Attachments retain size and SHA-256 metadata.
- Approval decisions use atomic versioned transitions and cannot dispatch disabled, rejected, expired, or cancelled work.
- Governance declarations are enforced at execution time. Unsupported sandbox, path, network, credential, trust, proof, or cost requirements fail closed.
- Database schema and consolidation failures stop startup. `openclaw-scheduler doctor --json` reports schema, lease, queue, outbox, approval, and cancellation diagnostics.
- AgentCLI handoff v4 binds the complete canonical execution artifact to every
  dispatch, approval, run, provider session, credential presentation, runtime
  event, and signed evidence record. Versions 1 through 3 remain supported.

These controls do not make arbitrary side effects idempotent. Destructive commands must still detect prior completion and be safe to retry.

---

## Quick Start

**New to the scheduler?** Start with [QUICK-START.md](QUICK-START.md) -- a focused guide covering installation, converting existing OpenClaw crons, and building your first workflow chain.

For the full reference, use the npm-first path below and then jump straight to [Five-Minute Setup](#five-minute-setup).

### Option A: npm-first (publish/install flow)

```bash
mkdir -p ~/.openclaw/scheduler
npm install --ignore-scripts=false --prefix ~/.openclaw/scheduler openclaw-scheduler@latest
npm exec --prefix ~/.openclaw/scheduler openclaw-scheduler -- setup
```

This installs the package without cloning the repo. The launcher command maps to:
- `openclaw-scheduler setup` → `setup.mjs`
- `openclaw-scheduler start` → `dispatcher.js`
- `openclaw-scheduler webhook-check` → `scripts/telegram-webhook-check.mjs`
- `openclaw-scheduler <anything-else>` → `cli.js`

The explicit `--ignore-scripts=false` is required because `better-sqlite3`
builds or installs its trusted native binding during the npm lifecycle. If a
prior install used `ignore-scripts=true`, rerun the install command above or
run `npm rebuild --ignore-scripts=false better-sqlite3` before starting.

For npm installs, scheduler state defaults to `~/.openclaw/scheduler/` rather than `node_modules/openclaw-scheduler/`, so upgrades do not trample the database path.

If your Node runtime changes later, rebuild the native SQLite binding before restarting the scheduler:

```bash
cd ~/.openclaw/scheduler
npm rebuild better-sqlite3 --ignore-scripts=false
```

This is commonly needed after a Homebrew Node upgrade on macOS or any major Node ABI change.

### macOS shell setup for ad hoc commands

If you use `zsh` on macOS, put your minimal Homebrew PATH bootstrap in `~/.zshenv`, not only in `~/.zprofile` or `~/.zshrc`.

Why:
- `launchd` services do not depend on your interactive shell startup files
- ad hoc commands like `ssh host 'node cli.js status'` run a non-interactive shell
- non-interactive `zsh` reads `~/.zshenv`, but does not read `~/.zprofile`

Recommended `~/.zshenv`:

```zsh
# ~/.zshenv — sourced by all zsh instances, including non-interactive SSH commands
if [ -x /opt/homebrew/bin/brew ]; then
  eval "$(/opt/homebrew/bin/brew shellenv)"
fi

export PATH="$HOME/.local/bin:$HOME/bin:/opt/homebrew/bin:/opt/homebrew/sbin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin"
```

If you load OpenClaw completions in `~/.zshrc`, initialize completions first so `compdef` is available:

```zsh
autoload -Uz compinit
compinit

if [ -f "$HOME/.openclaw/completions/openclaw.zsh" ]; then
  source "$HOME/.openclaw/completions/openclaw.zsh"
fi
```

Avoid pinning a versioned Node path like `/opt/homebrew/opt/node@22/bin` in shell startup files. Prefer the stable Homebrew symlink `/opt/homebrew/bin/node`, which survives normal `brew upgrade node`.

### Option B: source clone (dev/contributor flow)

```bash
git clone https://github.com/amittell/openclaw-scheduler ~/.openclaw/scheduler
cd ~/.openclaw/scheduler
npm install
npm test                             # should end with: 0 failed
npm run lint                         # static checks
npm run typecheck                    # exported API declarations
npm run coverage                     # coverage summary + lcov report
npm run verify:local                 # full local maintainer gate
npm run verify:smoke                 # lightweight smoke gate used by GitHub Actions
```

GitHub Actions runs `npm run verify:smoke` on Linux and macOS across the supported Node.js lines. That gate includes lint, type checking, every repository test, documentation validation, and a package dry run. `npm run verify:local` adds coverage and is enforced again by `prepublishOnly`. Focused test files run sequentially with isolated databases. A dedicated required job separately checks the exact public handoff-v3 producer and the previous handoff-v2 producer.

### Option C: local npm pack (simulate the published package from source)

Use this when you want to test the exact package layout end users get from npm without publishing to npmjs.org yet.

```bash
git clone https://github.com/amittell/openclaw-scheduler /tmp/openclaw-scheduler
cd /tmp/openclaw-scheduler
npm ci
npm run verify:local
npm pack

mkdir -p ~/.openclaw/packages/openclaw-scheduler
npm install --ignore-scripts=false --prefix ~/.openclaw/packages/openclaw-scheduler --omit=dev --no-package-lock ./openclaw-scheduler-*.tgz
```

Point your service at `~/.openclaw/packages/openclaw-scheduler/node_modules/openclaw-scheduler/dispatcher.js`, and keep mutable state in `~/.openclaw/scheduler` via `SCHEDULER_HOME` and `SCHEDULER_DB`.

The package also exports a small safe programmatic API surface for tooling:

```js
import { db, jobs, runs, shellResults } from 'openclaw-scheduler';
```

Then run the interactive setup wizard:

```bash
npm exec openclaw-scheduler -- setup
# or: node setup.mjs
```

The wizard will:
- Run DB migrations
- Append scheduler queue/inbox-consumer entries to your agent's `MEMORY.md` and `workspace-index.md`
- Create **Inbox Consumer** + **Stuck Run Detector** scheduler jobs
- Configure dispatcher auto-start service:
  - macOS: LaunchAgent (personal auto-login Mac) or LaunchDaemon (headless/pre-login startup)
  - Linux/WSL2: systemd user service (or PM2 fallback)

After setup:

```bash
npm exec openclaw-scheduler -- status # verify scheduler is running
node scripts/stuck-run-detector.mjs   # should print: No stale runs older than 15 minute(s).
tail -5 /tmp/openclaw-scheduler.log   # live logs
```

Dispatcher setup is covered in:
- [INSTALL.md](INSTALL.md) (macOS launchd: LaunchAgent or LaunchDaemon)
- [INSTALL-LINUX.md](INSTALL-LINUX.md) (Linux/WSL2 systemd + PM2 fallback)
- [INSTALL-WINDOWS.md](INSTALL-WINDOWS.md) (WSL2 setup path)
For additional hosts, see [INSTALL-ADDITIONAL-HOST.md](INSTALL-ADDITIONAL-HOST.md).

---

## Five-Minute Setup

This is the shortest path from "I installed it" to "I have a real job running."

### 1. Install and initialize

```bash
mkdir -p ~/.openclaw/scheduler
npm install --ignore-scripts=false --prefix ~/.openclaw/scheduler openclaw-scheduler@latest
alias ocs='npm exec --prefix ~/.openclaw/scheduler openclaw-scheduler --'
ocs setup
ocs status
```

What this does:
- installs the package into `~/.openclaw/scheduler`
- creates or migrates `scheduler.db`
- installs the scheduler service using the launchd mode you choose (`agent` or `daemon`)
- creates the built-in helper jobs like `Inbox Consumer` and `Stuck Run Detector`

If you plan to use the scheduler often, add the `ocs` alias to your shell profile.

### 2. Add your first real job

This example runs a simple shell health check every 15 minutes.

```bash
ocs jobs add '{
  "name": "Disk Space Check",
  "schedule_cron": "*/15 * * * *",
  "session_target": "shell",
  "payload_message": "df -h /",
  "delivery_mode": "none",
  "run_timeout_ms": 120000,
  "origin": "system"
}'
```

What it means in plain English:
- `schedule_cron`: run every 15 minutes
- `session_target: "shell"`: run a shell command directly, no AI needed
- `payload_message`: the command to run
- `delivery_mode: "none"`: do not send the output anywhere automatically
- `origin: "system"`: this job was created by the system, not from a user chat

### 3. Run it now and inspect the result

```bash
ocs jobs list
# copy the job ID for "Disk Space Check"

ocs jobs run <job-id>
ocs runs list <job-id> 5
```

If you want the full run record:

```bash
ocs runs get <run-id>
```

At that point you have a working scheduler install, a real job, and visible run history. Everything after this is layering on more power: AI jobs, delivery, retries, workflow chains, and approvals.

---

## Starter Recipes

These are copy-paste examples for the most common first workflows.

### 1. Shell health check with failure alerts

Use this when you want a script to run reliably even if the OpenClaw gateway is down.

```bash
ocs jobs add '{
  "name": "API Health Check",
  "schedule_cron": "*/15 * * * *",
  "session_target": "shell",
  "payload_message": "curl -fsS http://127.0.0.1:8080/health || exit 1",
  "delivery_mode": "announce",
  "delivery_channel": "telegram",
  "delivery_to": "YOUR_CHAT_ID",
  "run_timeout_ms": 120000,
  "origin": "system"
}'
```

Why this is useful:
- it runs even if the gateway is unhealthy
- it only announces on failure
- every run is stored in history

### 2. Daily AI summary

Use this when you want a scheduled agent report instead of a shell script.

```bash
ocs jobs add '{
  "name": "Daily Ops Summary",
  "schedule_cron": "0 9 * * *",
  "schedule_tz": "America/New_York",
  "session_target": "isolated",
  "payload_message": "Summarize the last 24 hours of important errors, deploys, and follow-ups in 5 bullet points.",
  "delivery_mode": "announce-always",
  "delivery_channel": "telegram",
  "delivery_to": "YOUR_CHAT_ID",
  "run_timeout_ms": 300000,
  "origin": "system"
}'
```

Why this is useful:
- the agent runs in its own isolated session
- delivery is retried durably and exhausted failures remain visible to operators
- the run history stays separate from your personal chat threads

### 3. Approval-gated follow-up step

Use this when a risky step should wait for a human before it runs.

```bash
ocs jobs add '{
  "name": "Delete Old Backups",
  "parent_id": "<parent-job-id>",
  "trigger_on": "success",
  "approval_required": 1,
  "approval_timeout_s": 3600,
  "approval_auto": "reject",
  "session_target": "shell",
  "payload_message": "find /backups -type f -mtime +14 -delete",
  "delivery_mode": "announce-always",
  "delivery_channel": "telegram",
  "delivery_to": "YOUR_CHAT_ID",
  "run_timeout_ms": 120000
}'
```

Why this is useful:
- the parent job can run automatically
- the risky cleanup step pauses until someone approves it
- the scheduler records who approved or rejected it

Approve or reject later with:

```bash
ocs approvals list --json
ocs approvals approve <approval-id> --reason "Change window open"
ocs approvals reject <approval-id> --reason "Not today"
```

---

## Common Migrations

If you already have cron jobs, OpenClaw cron entries, or shell scripts, this is the simplest way to think about the conversion.

### 1. OpenClaw built-in cron -> inspect, import, test, then disable duplicates

The default importer reads the supported OpenClaw CLI rather than an internal storage file. It calls `openclaw cron list --json` and `openclaw cron get <id> --json`:

```bash
openclaw-scheduler migrate --dry-run --json > migration-report.json
openclaw-scheduler migrate --json
openclaw-scheduler jobs list --json
```

The dry run never opens, creates, or migrates the target scheduler database.
Existing-ID duplicate checks therefore occur during the real import.

Cron expressions and one-shot timestamps are preserved exactly. Intervals that five-field cron cannot represent exactly are rejected unless the operator explicitly passes `--allow-inexact-every`. A pre-SQLite export is opt-in:

```bash
openclaw-scheduler migrate --legacy-json ~/.openclaw/cron/jobs.json --dry-run --json
```

After representative imported jobs complete successfully, disable only the corresponding native jobs:

```bash
openclaw cron edit <job-id> --disable
```

Rollback by stopping the scheduler, disabling its imported copies, and re-enabling the native jobs. Retain the scheduler database until rollback verification is complete.

### 2. Plain shell cron line -> `session_target: "shell"`

If you have a normal cron line like:

```cron
*/5 * * * * /usr/local/bin/check-api.sh
```

Convert it to:

```bash
ocs jobs add '{
  "name": "API Check",
  "schedule_cron": "*/5 * * * *",
  "session_target": "shell",
  "payload_message": "/usr/local/bin/check-api.sh",
  "delivery_mode": "announce",
  "delivery_channel": "telegram",
  "delivery_to": "YOUR_CHAT_ID",
  "run_timeout_ms": 120000,
  "origin": "system"
}'
```

Choose `shell` when the task is deterministic and you do not need AI reasoning.

### 3. AI-ish cron job -> `session_target: "isolated"`

If the old job was really “run a prompt every morning”, use an isolated agent job instead of a shell script:

```bash
ocs jobs add '{
  "name": "Daily Status Summary",
  "schedule_cron": "0 8 * * *",
  "schedule_tz": "America/New_York",
  "session_target": "isolated",
  "payload_message": "Summarize the most important errors, deploys, and follow-ups from the last 24 hours in 5 bullet points.",
  "delivery_mode": "announce-always",
  "delivery_channel": "telegram",
  "delivery_to": "YOUR_CHAT_ID",
  "run_timeout_ms": 300000,
  "origin": "system"
}'
```

Choose `isolated` when the job needs reasoning, writing, summarization, or tools.

### 4. Two cron jobs with manual ordering -> parent/child chain

If your current workflow is:
- run a backup
- wait
- run verification

Model that as a chain instead of two unrelated cron entries:

```bash
ocs jobs add '{
  "name": "Nightly Backup",
  "schedule_cron": "0 2 * * *",
  "session_target": "shell",
  "payload_message": "/usr/local/bin/nightly-backup.sh",
  "delivery_mode": "announce",
  "delivery_channel": "telegram",
  "delivery_to": "YOUR_CHAT_ID",
  "run_timeout_ms": 600000,
  "origin": "system"
}'
```

Then create the follow-up:

```bash
ocs jobs add '{
  "name": "Verify Nightly Backup",
  "parent_id": "<backup-job-id>",
  "trigger_on": "success",
  "trigger_delay_s": 60,
  "session_target": "shell",
  "payload_message": "/usr/local/bin/verify-backup.sh",
  "delivery_mode": "announce",
  "delivery_channel": "telegram",
  "delivery_to": "YOUR_CHAT_ID",
  "run_timeout_ms": 120000
}'
```

This is one of the biggest upgrades over plain cron: the second step now runs because the first step succeeded, not because the clock happened to reach another minute.

### 5. Risky follow-up -> add `approval_required`

If the current process is “job runs, then a human decides whether to continue,” model that decision directly:

```bash
ocs jobs add '{
  "name": "Delete Temp Files",
  "parent_id": "<analysis-job-id>",
  "trigger_on": "success",
  "approval_required": 1,
  "approval_timeout_s": 3600,
  "approval_auto": "reject",
  "session_target": "shell",
  "payload_message": "find /tmp/myapp -type f -mtime +7 -delete",
  "delivery_mode": "announce-always",
  "delivery_channel": "telegram",
  "delivery_to": "YOUR_CHAT_ID",
  "run_timeout_ms": 120000
}'
```

That keeps the job automated, but only up to the point where human judgment is actually needed.

### Rule of thumb

When converting existing work:
- start with `shell` unless you clearly need AI reasoning
- add delivery only if someone really needs to see the output
- use chains when one step depends on another
- use approvals when the next step would be annoying, expensive, or risky if it ran by mistake

---

## Platform Support

| Platform | Service Manager | Shell Jobs | Status |
|----------|----------------|------------|--------|
| macOS | launchd (`agent` or `daemon`) | `/bin/zsh` | Tested |
| Linux | systemd user service or PM2 fallback | `/bin/bash` | Supported |
| Windows (WSL2) | systemd user service or PM2 fallback inside WSL2 | `/bin/bash` | Supported |
| Windows (native or WSL1) | Not applicable | Not applicable | Not supported; use WSL2 |

- **macOS:** Full guide in [INSTALL.md](INSTALL.md)
- **Linux:** Full guide in [INSTALL-LINUX.md](INSTALL-LINUX.md)
- **Windows:** Install WSL2, then follow [INSTALL-LINUX.md](INSTALL-LINUX.md). See [INSTALL-WINDOWS.md](INSTALL-WINDOWS.md) for WSL2 setup.

Override the shell for shell jobs with the `SCHEDULER_SHELL=/path/to/shell` environment variable.

---

## Architecture

The scheduler sits alongside the OpenClaw gateway as an independent process.
Isolated targets use stable per-job sessions that remain separate from the
user's main conversation. Main targets use the existing main session, and shell
targets do not create an agent session.

```
┌─────────────────────────────────────────────┐
│  Host Machine (e.g., scheduler-host.local)  │
│                                              │
│  OpenClaw Gateway (:18789)                   │
│    ├─ Telegram / Discord / etc.              │
│    ├─ Chat completions endpoint (/v1/...)    │
│    ├─ Tool execution (exec, browser, k8s...) │
│    └─ Memory search                          │
│                                              │
│  Scheduler (launchd service)                 │
│    ├─ SQLite DB (scheduler.db)               │
│    ├─ Job dispatch via chat completions      │
│    ├─ Workflow chain engine                  │
│    ├─ Retry logic                            │
│    ├─ Shell job execution                    │
│    ├─ HITL approval gates                    │
│    ├─ Idempotency ledger                     │
│    ├─ Inter-agent message queue              │
│    ├─ Task tracker                           │
│    └─ MinIO backup                           │
└─────────────────────────────────────────────┘
```

### Tick Loop

```
┌──────────────────────────────────────────────────────────────┐
│                   Dispatcher Loop (10s tick)                  │
│                                                              │
│  1. Gateway health check                                     │
│  2. Find due jobs → dispatch                                 │
│  3. Check running runs (stale/timeout detection)             │
│  4. HITL approval gate check                                 │
│  5. Message delivery + spawn handling                        │
│  6. Task tracker dead-man's-switch                           │
│  7. Expire old messages                                      │
│  8. Prune old runs + WAL checkpoint (hourly)                 │
│  9. Backup to MinIO (every 5 min)                            │
└──────────────────────────────────────────────────────────────┘
         │                              │
         ▼                              ▼
  ┌───────────────────┐        ┌──────────────────────┐
  │     SQLite DB      │        │   OpenClaw Gateway    │
  │                    │        │                       │
  │  • jobs            │        │  • /v1/chat/completions│
  │  • runs            │        │  • /tools/invoke      │
  │  • messages        │        │  • /health            │
  │  • agents          │        │  • system event CLI   │
  │  • approvals       │        └──────────────────────┘
  │  • task_tracker    │
  │  • idempotency_ledger│
  │  • delivery_aliases│
  │  • schema_migrations│
  └───────────────────┘
```

### Session Types

| Session | Created By | Lifetime | Used For |
|---------|-----------|----------|----------|
| User DM | Telegram message | Persistent per-peer | Your conversations |
| Group chat | Group message | Persistent per-group | Team discussions |
| Isolated job | Dispatcher via API | Persistent per job, reused across runs | Cron jobs, chain steps |
| Main session | Dispatcher API, or `openclaw system event` for fire-and-forget | Existing main session | Jobs needing main context |
| Shell | Dispatcher (direct) | Per-job (no session) | Cron scripts, backups, maintenance |
| Sub-agent | `sessions_spawn` | Task-scoped | Delegated work |

Isolated scheduler jobs cannot see user-chat history, but later runs of the same
job reuse its warm per-job session and may retain that job's earlier context.
Main-session jobs intentionally use the existing main conversation.

---

## How Jobs Execute

### Isolated Jobs (default)

```
Scheduler tick (every 10s)
  │
  ├─ getDueJobs() → "Hourly Workspace Backup is due"
  ├─ hasRunningRun()? → skip if overlap_policy='skip'
  ├─ createRun() → status='running'
  ├─ setAgentStatus('main', 'busy')
  │
  ├─ POST /v1/chat/completions
  │   session: agent:<agent_id>:scheduler:<job_id>  (stable per job, isolated from main)
  │   model: openclaw:main
  │   message: [job prompt + any pending inbox messages]
  │
  │   ← "Committed 3 files, pushed to origin"
  │
  ├─ finishRun('ok', summary)
  ├─ setAgentStatus('main', 'idle')
  ├─ Deliver to Telegram? → delivery_mode + channel + target
  ├─ Queue result message for traceability
  ├─ Advance next_run_at to next cron fire
  └─ Trigger child jobs if any (workflow chain)
```

### Main Session Jobs

For jobs that need the main session context (rare), use
`payload_kind: "systemEvent"`. Default, `execute`, or `plan` execution waits
for the agent response and captures it for normal completion handling:

```
Dispatcher → POST /v1/chat/completions → wait for main-session response
```

Set `execution_intent: "fire-and-forget"` only when the scheduler should inject
the event and return immediately without capturing the eventual response:

```
Dispatcher → exec: openclaw system event --text "..." --mode now
```

Handoff v4 main-session jobs use only the synchronous path because it carries
the artifact, runtime-instance, and fresh capability-nonce binding through the
Gateway request. The `openclaw system event` transport cannot carry that
binding, so v4 fire-and-forget jobs fail validation. Earlier handoff versions
keep the existing fire-and-forget behavior.

### Shell Jobs

```
Shell Job (session_target='shell')
  │
  ├─ getDueJobs() → "Hourly Backup is due"
  ├─ createRun() → status='running'
  ├─ run "<payload_message>" via shell (platform default or SCHEDULER_SHELL)
  │   (no gateway required)
  │   ← exit 0: "Backup complete, 3 files"
  │
  ├─ finishRun(exit===0 ? 'ok' : 'error')
  ├─ announce: post output if exit ≠ 0
  ├─ announce-always: post output regardless
  └─ Trigger child jobs if any
```

### Prompt Building

Each isolated job prompt includes:
1. Header: `[scheduler:<job_id> <job_name>]`
2. Pending inbox messages for the agent (up to 5)
3. Context from prior runs (if `context_retrieval` is set)
4. The job's `payload_message`

---

## Delivery Modes

The scheduler delivers job output through the OpenClaw gateway's messaging
system. All channels supported by the gateway work with the scheduler:
**Telegram**, **Discord**, **WhatsApp**, **Signal**, **iMessage**, and **Slack**.
Set `delivery_channel` to the channel name and `delivery_to` to the
channel-specific target (chat ID, channel ID, phone number, handle, etc.).

| Mode | When output is delivered |
|------|-------------------------|
| `none` | Never (background jobs) |
| `announce` | Agent jobs: delivers when run status is not `ok`. Shell jobs: non-zero exit only. Silently skipped for `main` session jobs (use `announce-always` instead) |
| `announce-always` | Always delivers output (LLM or shell), including `main` session jobs |

> **Note:** non-exempt jobs using `announce` or `announce-always` are rejected at validation time when `delivery_to` is absent. Runtime delivery is written to a transactional outbox, separate from agent prompt messages. Outbox claims expire and recover safely. Multipart output is stored as independently retryable rows with deterministic `:part:i/N` idempotency keys. Durable enqueue is not a delivery receipt; run-scoped completion debt closes only after every part is delivered. Attachment rows retain size and SHA-256 metadata.
>
> Examples in this document use Telegram for delivery_channel since it is the
> most common configuration. Replace with your channel of choice.

---

## Delivery Aliases

Delivery aliases let you define named delivery targets (e.g., `@my_team`) instead of hard-coding channel/target pairs in every job.

```bash
# Create a named alias
openclaw-scheduler alias add my_team telegram -100200000000

# Use @alias in job (resolves at dispatch time)
openclaw-scheduler jobs add '{
  "name": "Alert",
  "delivery_mode": "announce",
  "delivery_to": "@my_team",
  ...
}'

# List aliases
openclaw-scheduler alias list

# Remove an alias
openclaw-scheduler alias remove my_team
```

Aliases are resolved at dispatch time. If an alias is deleted, jobs fall back to suppressed delivery.

---

## Shell Jobs

Shell jobs run a command directly on the host — no gateway or LLM required. Ideal for backups, scripts, maintenance tasks, and anything that doesn't need AI.

```bash
openclaw-scheduler jobs add '{
  "name": "Hourly Backup",
  "schedule_cron": "0 * * * *",
  "schedule_tz": "America/New_York",
  "session_target": "shell",
  "payload_message": "/path/to/backup.sh",
  "delivery_mode": "announce",
  "delivery_channel": "telegram",
  "delivery_to": "YOUR_CHAT_ID",
  "run_timeout_ms": 600000,
  "origin": "system"
}'
```

**Key properties:**
- **No gateway dependency** — runs even when gateway is down
- `payload_message` is the command to execute (shell string passed to the configured shell)
- Output captured up to 1MB, with preview/offload budgets to keep large output out of the main run row
- Shell runs persist structured failure context on `runs`: `shell_exit_code`, `shell_signal`, `shell_timed_out`, `shell_stdout`, `shell_stderr`, plus optional `shell_stdout_path` / `shell_stderr_path` when large output is offloaded
- Failure-triggered agent children receive shell context with separate exit code, stdout, and stderr blocks
- `run_timeout_ms` controls max execution time (required, no default)
- Workflow chains work the same way — shell jobs can trigger children on success/failure
- Shell jobs now honor `max_retries` before failure children fire, the same as isolated agent jobs
- `openclaw-scheduler runs output <run-id> stdout|stderr` retrieves stored or offloaded shell output on demand

**With environment variables:**
```bash
openclaw-scheduler jobs add '{
  "name": "DB Dump",
  "schedule_cron": "0 3 * * *",
  "session_target": "shell",
  "payload_message": "PGPASSWORD=secret pg_dump mydb > /backups/mydb.sql && echo OK",
  "delivery_mode": "announce-always",
  "delivery_channel": "telegram",
  "delivery_to": "YOUR_CHAT_ID",
  "run_timeout_ms": 600000,
  "origin": "system"
}'
```

---

## HITL Approval Gates

Jobs with `approval_required: 1` pause every dispatch before execution. The gate
applies uniformly to root, manual, scheduled, one-shot, retry, and
chain-triggered work. Every gated attempt is represented by a durable dispatch
row and an `awaiting_approval` run.

```bash
# Scheduled root job that requires a scoped operator approval
openclaw-scheduler jobs add '{
  "name": "Deploy to Prod",
  "schedule_cron": "0 10 * * 1-5",
  "approval_required": 1,
  "approval_risk_level": "high",
  "approval_approver_scope": "user:alex",
  "approval_timeout_s": 3600,
  "approval_auto": "reject",
  "session_target": "shell",
  "payload_kind": "shellCommand",
  "payload_message": "Deploy the application to production",
  "delivery_mode": "none",
  "origin": "system",
  "run_timeout_ms": 300000
}'
```

When triggered, the job creator receives the pending approval identifier, risk,
scope, and CLI commands.

```bash
openclaw-scheduler approvals list --json
openclaw-scheduler approvals approve APPROVAL_ID --reason "Change window open"
openclaw-scheduler approvals reject APPROVAL_ID --reason "Postponing until the next change window"
```

**Key notes:**
- `approval_risk_level` is optional and accepts `low`, `medium`, or `high`.
- `approval_approver_scope` accepts an unprefixed exact identity or
  `exact:`, `user:`, `uid:`, or `principal:` matching for the local OS account.
  Domain scopes are not supported.
- AgentCLI handoff v4 exposes approval-scope enforcement as one coarse capability
  while its manifests may contain domain scopes. The scheduler therefore
  advertises `approval_scope_enforcement: false` so scoped agentcli manifests
  fail capability negotiation instead of being partially enforced. Direct
  scheduler jobs may still use the supported local scopes above.
- The scheduler derives the approver from the invoking operating-system user
  and UID. CLI flags and environment variables cannot select another identity.
- Prefer `approvals approve/reject APPROVAL_ID`; `jobs approve/reject JOB_ID`
  remains a legacy convenience for that job's current pending approval and
  cannot override approver identity.
- Use `approvals list --json` to copy the complete approval UUID before making
  a decision.
- `approval_auto: "approve"` is invalid with an approver scope. Scoped timeout
  resolution fails closed.
- The approval snapshots risk, scope, and a canonical SHA-256 binding of the
  execution contract. Changing, disabling, or deleting the job cancels the
  pending approval instead of applying it to different work.
- `approval_timeout_s` controls the auto-resolution deadline and
  `approval_auto` selects `"approve"` or `"reject"` for unscoped jobs.

### Declared output formats

Set `output_format` to `json`, `ndjson`, or `text` when downstream work requires
a validated result shape. Successful JSON output must parse as one JSON value;
each nonblank NDJSON line must parse independently; text is stored on the
normalized text path. The scheduler persists validity, byte count, SHA-256
digest, and either the parsed value or an artifact reference on the run.
Malformed JSON or NDJSON remains a successful execution with
`structured_output_valid: 0`, a `structured_output_warning`, and a null parsed
value; success children may still run because transport success is distinct
from output-shape validation.

### Post-success verification

Any synchronous job can declare a separate local shell verification command
that runs after its primary execution and structured-output parsing, but before
terminal evidence, delivery, and child dispatch:

```json
{
  "verify_shell": "test -f /srv/app/healthy",
  "verify_timeout_s": 30,
  "verify_on_failure": "error"
}
```

`verify_on_failure` accepts `error` or `warn`. An error policy converts a failed
verification to a terminal error and blocks success children; a warning policy
preserves primary success and records the warning. Verification stores only
status, timing, exit metadata, byte counts, and SHA-256 digests, never raw
verification output. Fire-and-forget jobs cannot declare verification.

---

## Idempotency

Control what happens when the dispatcher crashes mid-run. Despite the legacy
field name `delivery_guarantee`, this setting governs execution replay, not
external message-delivery confirmation.

```bash
# Enable at-least-once: crashed runs replay on next startup
openclaw-scheduler jobs update <id> '{"delivery_guarantee":"at-least-once"}'

# Default (at-most-once): no replay
openclaw-scheduler jobs update <id> '{"delivery_guarantee":"at-most-once"}'
```

**How it works:**
- **`at-most-once`** (default): if dispatcher crashes mid-run, run is marked `crashed` and the schedule advances normally. The run is not replayed.
- **`at-least-once`**: on startup, any `running` run from a crashed dispatcher is replayed with a new run. `replay_of` field tracks the original run ID for lineage.

**Idempotent agents** can return `IDEMPOTENT_SKIP` in their response to acknowledge they've already processed this execution (detected via the idempotency ledger).

The ledger also prevents double-dispatch in concurrent tick scenarios — each run acquires a lock before dispatch.

---

## Context Retrieval

Inject prior run summaries into a job's prompt so the agent has awareness of recent outcomes.

```bash
# Inject last 3 run summaries into job prompt
openclaw-scheduler jobs update <id> '{"context_retrieval":"recent","context_retrieval_limit":3}'

# Hybrid: recent runs + TF-IDF search for semantically relevant summaries
openclaw-scheduler jobs update <id> '{"context_retrieval":"hybrid","context_retrieval_limit":5}'
```

**Modes:**
| Mode | Description |
|------|-------------|
| `none` | No context injected (default) |
| `recent` | Last N run summaries, newest first |
| `hybrid` | Recent runs + TF-IDF similarity search against all prior summaries |

Useful for health check jobs that should know about yesterday's failures, or audit jobs that build incrementally on prior work.

---

## Task Tracker

The task tracker provides a dead-man's-switch for coordinating multi-agent sub-agent teams. Create a tracker, assign expected agents, and receive a summary when all agents complete (or time out).

```bash
# Create a task group to monitor N sub-agents
openclaw-scheduler tasks create '{
  "name": "v2-release-team",
  "expected_agents": ["schema-agent","frontend-agent","docs-agent"],
  "timeout_s": 1800,
  "delivery_channel": "telegram",
  "delivery_to": "YOUR_CHAT_ID"
}'

# Monitor
openclaw-scheduler tasks list
openclaw-scheduler tasks status <tracker-id>
```

Each agent in the team must send heartbeat updates. If an agent goes silent past its timeout, it's declared dead. When all agents complete or time out, a summary is delivered to the configured channel.

---

## Resource Pools

Prevent concurrent execution across different jobs that share a resource.

```bash
# Two jobs that must not run concurrently
openclaw-scheduler jobs add '{"name":"DB Migration","resource_pool":"database",...}'
openclaw-scheduler jobs add '{"name":"DB Backup","resource_pool":"database",...}'
```

If one job in a pool is currently running, all other pool members skip their tick (same behavior as `overlap_policy: 'skip'`, but cross-job rather than per-job). Pool membership is set via the `resource_pool` string field.

---

## Workflow Chains

Jobs can be linked into parent → child chains. When a parent completes, its children fire automatically.

### Pattern 1: Chained Jobs

```bash
# Parent: runs on cron
openclaw-scheduler jobs add '{
  "name": "Build App",
  "schedule_cron": "0 10 * * *",
  "payload_message": "Build the application",
  "run_timeout_ms": 300000,
  "origin": "system"
}'
# → id: "abc123..."

# Child: fires when parent succeeds
openclaw-scheduler jobs add '{
  "name": "Deploy App",
  "payload_message": "Deploy to production",
  "parent_id": "abc123...",
  "trigger_on": "success",
  "run_timeout_ms": 300000
}'

# Child: fires when parent fails
openclaw-scheduler jobs add '{
  "name": "Build Alert",
  "payload_message": "Build failed -- check logs",
  "parent_id": "abc123...",
  "trigger_on": "failure",
  "delivery_mode": "announce",
  "delivery_to": "YOUR_CHAT_ID",
  "run_timeout_ms": 300000
}'
```

**Trigger types:**
- `success` — parent run status = `ok`
- `failure` — parent run status = `error` or `timeout`
- `complete` — any completion (success, failure, or timeout)
- Child jobs are chain-triggered only. Use `trigger_delay_s` to delay a child run; one-shot `schedule_kind: "at"` is for root jobs only.

### Pattern 2: Output-Based Trigger Conditions

```bash
# Only fire child if parent output contains "ALERT"
openclaw-scheduler jobs add '{
  "name": "Alert Handler",
  "parent_id": "<monitor-job-id>",
  "trigger_on": "success",
  "trigger_condition": "contains:ALERT",
  "payload_message": "Handle the alert",
  "run_timeout_ms": 300000
}'

# Regex condition
openclaw-scheduler jobs add '{
  "name": "Critical Error Handler",
  "parent_id": "<monitor-job-id>",
  "trigger_on": "success",
  "trigger_condition": "regex:ERROR.*critical",
  "payload_message": "Handle critical error",
  "run_timeout_ms": 300000
}'
```

`regex:` conditions use RE2 syntax and linear-time matching. Backreferences,
lookahead, and lookbehind are rejected. The full condition is limited to 1,024
characters, and regex evaluation fails closed when the parent output exceeds
65,536 UTF-8 bytes. Use `contains:` when a literal substring is sufficient.

### Pattern 3: Multi-Agent Workflows

Chain jobs targeting different agents:

```
Build (agent: main, cron: 10am)
  └─ Deploy (agent: ops, trigger: success)
      └─ Health Check (agent: main, trigger: success, delay: 60s)
```

```bash
openclaw-scheduler jobs add '{
  "name": "Deploy",
  "payload_message": "deploy",
  "agent_id": "ops",
  "parent_id": "<build-id>",
  "trigger_on": "success",
  "run_timeout_ms": 300000
}'
```

### Pattern 4: Delayed Triggers

```bash
openclaw-scheduler jobs add '{
  "name": "Post-Deploy Check",
  "payload_message": "Verify services healthy",
  "parent_id": "<deploy-id>",
  "trigger_on": "success",
  "trigger_delay_s": 60,
  "run_timeout_ms": 300000
}'
```

### Pattern 5: Runtime Spawning

A running agent can create new jobs on the fly by sending a `spawn` message:

```json
{
  "from_agent": "main",
  "to_agent": "scheduler",
  "kind": "spawn",
  "body": "{\"name\":\"Dynamic Task\",\"payload_message\":\"analyze results\",\"delete_after_run\":true,\"run_now\":true}"
}
```

### Visualizing Chains

```bash
openclaw-scheduler jobs tree

# Output (all root jobs and their chains):
# Build App
#   └─ Deploy App [→success] (agent:ops)
#   └─ Build Alert [→failure]
#   └─ Health Check [→complete +60s]
```

---

## Retry Logic

Jobs can auto-retry before declaring failure and triggering failure children.

```bash
openclaw-scheduler jobs add '{
  "name": "Flaky Deploy",
  "schedule_cron": "0 10 * * *",
  "payload_message": "deploy to prod",
  "max_retries": 3,
  "run_timeout_ms": 300000,
  "origin": "system"
}'
```

**How it works:**

1. Job fails → check `max_retries`
2. Retries remaining → schedule retry with exponential backoff (30s, 60s, 120s, ...)
3. Retry run tracks lineage: `retry_of` → failed run ID, `retry_count` incremented
4. All retries exhausted → trigger failure children + apply error backoff
5. Any retry succeeds → trigger success children, reset `consecutive_errors`

**Key:** failure children don't fire until all retries are exhausted. This prevents false alerts on transient failures.

This retry ladder now applies uniformly to shell jobs, isolated agent jobs, and main-session jobs that surface dispatch failures.

| Field | Default | Description |
|-------|---------|-------------|
| `max_retries` | 0 | Max retry attempts (0 = no retry) |
| `runs.retry_of` | null | ID of the failed run being retried |
| `runs.retry_count` | 0 | Which attempt this is (0 = first try) |

---

## Chain Safety

### Max Chain Depth

`MAX_CHAIN_DEPTH = 10` — enforced on:
- `createJob` — can't add a child deeper than 10 levels
- `updateJob` — can't move a job to create a chain deeper than 10
- `triggerChildren` — runtime safeguard stops dispatch at depth 10

### Cycle Detection

`detectCycle()` walks up the parent chain on both create and update. Catches:
- Self-referential: A → A
- Deep cycles: A → B → C → A
- Throws with descriptive error message

### Chain Cancellation

```bash
openclaw-scheduler jobs cancel <job-id>
# Requests cancellation for active runs on this job and its descendants
```

Cancellation is fenced against dispatcher ownership. Pending work is cancelled atomically. A running shell job receives process-group termination and cannot finalize or trigger children after cancellation wins. Agent work records and sends a Gateway cancellation request. Finished runs are unchanged. Poll `runs get`, `runs running`, or `status`; chat notifications are asynchronous and can be stale.

---

## Inter-Agent Messaging

Agents exchange messages through the scheduler's queue.

### Features
- **Priority:** 0 (normal), 1 (high), 2 (urgent) — inbox sorted by priority then time
- **Threading:** `reply_to` links messages into conversations
- **Read receipts:** pending → delivered → read (with timestamps)
- **Broadcast:** `to_agent = 'broadcast'` reaches all agents
- **TTL/Expiry:** `expires_at` auto-expires unread messages
- **Metadata:** JSON blob for structured data
- **Kinds:** `text`, `task`, `result`, `status`, `system`, `spawn`, `decision`, `constraint`, `fact`, `preference`
- **Owner field:** `owner` tracks message originator for audit
- **Job linking:** messages can reference `job_id` and `run_id`

### Delivery

Messages are delivered inline with job prompts. When the dispatcher builds a prompt, it includes up to 5 pending messages for the target agent, marked as `delivered`.

### Usage

```bash
# Send a message
openclaw-scheduler msg send <from-agent> <to-agent> "message body"

# Read inbox
openclaw-scheduler msg inbox <agent-id>

# Mark all read
openclaw-scheduler msg readall <agent-id>
```

### Signal Queue Consumer Example

Use this when you want scripts to enqueue only actionable signals, then a single consumer job pushes those signals to Telegram.

```bash
# 1) Enqueue a signal
openclaw-scheduler msg send monitor-agent main "Found 3 critical errors in prod logs"

# 2) Run the inbox consumer daemon (event-driven, watches SQLite WAL)
#    Delivers within ~250ms of a message landing in the DB.
#    No polling cron needed — the daemon watches for WAL changes.
node scripts/inbox-consumer.mjs --watch --to YOUR_CHAT_ID --channel telegram

# Or as a one-shot drain (useful for testing):
node scripts/inbox-consumer.mjs --to YOUR_CHAT_ID --channel telegram
```

---

## Backup & Recovery

MinIO backups are disabled by default. Set `SCHEDULER_BACKUP=1` to enable. Requires `mc` (MinIO client) installed and configured with a `backupstore` alias.

The scheduler can back up its SQLite database to MinIO automatically.

```bash
# Manual snapshot
node backup.js snapshot

# Manual rollup (hourly aggregate)
node backup.js rollup

# Check backup status
node backup.js status

# Restore from snapshot
node backup.js restore

# Prune old backups
node backup.js prune
```

**Configuration via environment:**

| Variable | Default | Description |
|----------|---------|-------------|
| `SCHEDULER_BACKUP_MC_ALIAS` | `backupstore` | MinIO client alias |
| `SCHEDULER_BACKUP_BUCKET` | `scheduler-backups` | MinIO bucket name |
| `SCHEDULER_BACKUP_PREFIX` | `scheduler` | Path prefix within bucket |

Requires `mc` (MinIO client) in PATH and a configured `backupstore` alias.

**Built-in (when running as a background service):**
- Snapshot every 5 minutes (`SCHEDULER_BACKUP_MS`)
- Rollup on the first tick of each hour

---

## Agent Registry

| Operation | Function | Description |
|-----------|----------|-------------|
| Register | `upsertAgent(id, opts)` | Create or update |
| Get | `getAgent(id)` | Fetch by ID |
| List | `listAgents()` | All agents |
| Set status | `setAgentStatus(id, status, sessionKey)` | idle/busy/offline |
| Touch | `touchAgent(id)` | Update last_seen_at |

The dispatcher automatically manages agent status during dispatch (idle → busy → idle).

```bash
openclaw-scheduler agents list
openclaw-scheduler agents get <id>
openclaw-scheduler agents register <id> [name]
```

---

## Database Schema

**Schema version:** 28 | **Mode:** WAL | **Foreign keys:** ON

### Tables

| Table | Description |
|-------|-------------|
| `jobs` | Job definitions (schedule, payload, chain config, delivery) |
| `runs` | Execution history (status, timing, summaries, retry lineage) |
| `messages` | Inter-agent prompt queue (`pending`, `prompt_claimed`, delivered/read terminal states) |
| `agents` | Agent registry (status, capabilities, last seen) |
| `approvals` | Atomic, versioned HITL decisions and dispatch state |
| `task_tracker` | Multi-agent task group definitions |
| `task_tracker_agents` | Per-agent status within a task group |
| `idempotency_ledger` | Dispatch deduplication and at-least-once tracking |
| `delivery_aliases` | Named delivery targets (channel + target pairs) |
| `job_dispatch_queue` | Leased durable dispatch claims and replay state |
| `dispatcher_leases` | Singleton dispatcher ownership and fencing tokens |
| `delivery_outbox` | Transactional external delivery queue with multipart group/part coordinates |
| `delivery_attachments` | Durable attachment content/path and integrity metadata |
| `completion_debts` | Run-scoped completion delivery ownership and recovery state |
| `evidence_records` | Immutable, content-addressed SHA-256 evidence, one row per run |
| `message_receipts` | Delivery receipt tracking for messages |
| `team_tasks` | Team-scoped task definitions and status |
| `team_mailbox_events` | Projected events from team mailbox activity |
| `schema_migrations` | Baseline schema version log |

### Jobs (key columns)

```
id, name, enabled, schedule_kind, schedule_cron, schedule_at, schedule_tz,
session_target, agent_id, payload_kind, payload_message,
payload_model, payload_model_fallback, payload_thinking, execution_intent, execution_read_only,
overlap_policy, run_timeout_ms, max_queued_dispatches, max_pending_approvals,
max_trigger_fanout, output_store_limit_bytes, output_excerpt_limit_bytes,
output_summary_limit_bytes, output_offload_threshold_bytes,
max_retries, delivery_mode, delivery_channel,
delivery_to, delivery_guarantee, delete_after_run, ttl_hours,
parent_id, trigger_on, trigger_delay_s, trigger_condition,
resource_pool, auth_profile, auth_profile_fallback,
origin, source_channel, source_target, source_message_id, source_thread_id,
approval_required, approval_timeout_s, approval_auto,
approval_risk_level, approval_approver_scope, output_format,
verify_shell, verify_timeout_s, verify_on_failure,
context_retrieval, context_retrieval_limit,
preferred_session_key, job_type, watchdog_target_label,
watchdog_check_cmd, watchdog_timeout_min, watchdog_alert_channel,
watchdog_alert_target, watchdog_self_destruct, watchdog_started_at,
next_run_at, last_run_at, last_status, consecutive_errors,
created_at, updated_at
```

### Runs (key columns)

```
id, job_id, status, started_at, finished_at, duration_ms,
last_heartbeat, session_key, session_id, summary,
error_message, shell_exit_code, shell_signal, shell_timed_out,
shell_stdout, shell_stderr, shell_stdout_path, shell_stderr_path,
shell_stdout_bytes, shell_stderr_bytes, dispatched_at, run_timeout_ms,
triggered_by_run, retry_of, retry_count, replay_of,
delegation_validation, output_format, structured_output, structured_output_valid,
structured_output_warning, structured_output_bytes, structured_output_sha256,
structured_output_path, verification_result, approval_used
```

**Run statuses:** `pending`, `running`, `ok`, `error`, `timeout`, `skipped`, `cancelled`, `crashed`, `recovery_blocked`, `awaiting_approval`, `approved`

### Messages (key columns)

```
id, from_agent, to_agent, reply_to, kind, subject, body,
metadata, priority, channel, owner, status, delivered_at,
read_at, expires_at, created_at, job_id, run_id
```

### Agents (10 columns)

```
id, name, status, last_seen_at, session_key, capabilities,
delivery_channel, delivery_to, brand_name, created_at
```

---

## CLI Reference

```bash
# ── Jobs ──────────────────────────────────────────
openclaw-scheduler jobs list                     # List all (supports --type <type>)
openclaw-scheduler jobs list --include-handoff-artifacts --json # Validate and hydrate persisted v4 artifacts
openclaw-scheduler jobs get <id>                 # Full details as JSON
openclaw-scheduler jobs add --file job.json      # Inline JSON and --stdin are also supported
openclaw-scheduler jobs update <id> --stdin      # Partial update from standard input
openclaw-scheduler jobs validate --file job.json # Validate without DB initialization
openclaw-scheduler jobs enable <id>
openclaw-scheduler jobs disable <id>              # NOTE: one-shot at-jobs with delete_after_run: true are auto-pruned after 24h (ordinary disabled cron jobs are kept indefinitely)
openclaw-scheduler jobs delete <id>              # Cascades to runs
openclaw-scheduler jobs tree                     # Visual chain hierarchy
openclaw-scheduler jobs cancel <id>              # Cancel running chain

# ── Runs ──────────────────────────────────────────
openclaw-scheduler runs list <job-id> [limit]    # Run history
openclaw-scheduler runs get <run-id>             # Full run details
openclaw-scheduler runs output <run-id> stdout   # Stored/offloaded stdout or stderr
openclaw-scheduler runs evidence <run-id>        # Verify and return immutable execution evidence
openclaw-scheduler runs running                  # Active runs
openclaw-scheduler runs stale [threshold-s]      # Stale runs (default 90s)

# ── Messages ──────────────────────────────────────
openclaw-scheduler msg send <from> <to> <body>
openclaw-scheduler msg inbox <agent-id> [limit]
openclaw-scheduler msg outbox <agent-id> [limit]
openclaw-scheduler msg thread <message-id>
openclaw-scheduler msg ack <message-id> [actor] [note]
openclaw-scheduler msg receipts <message-id> [limit]
openclaw-scheduler msg team-inbox <team-id> [limit] [member-id] [task-id]
openclaw-scheduler msg read <message-id>
openclaw-scheduler msg readall <agent-id>
openclaw-scheduler msg unread <agent-id>

# ── Agents ────────────────────────────────────────
openclaw-scheduler agents list
openclaw-scheduler agents get <id>
openclaw-scheduler agents register <id> [name]

# ── Approvals ─────────────────────────────────────
openclaw-scheduler approvals list --json                 # Full pending approval IDs
openclaw-scheduler approvals approve <id> [--reason ...] # Approve by approval ID
openclaw-scheduler approvals reject <id> [--reason ...]  # Reject by approval ID
openclaw-scheduler jobs approve <job-id>                  # Legacy current-gate lookup
openclaw-scheduler jobs reject <job-id> [reason]          # Legacy current-gate lookup

# ── Task Tracker ──────────────────────────────────
openclaw-scheduler tasks create '<json>'        # Create task group
openclaw-scheduler tasks list                   # Active task groups
openclaw-scheduler tasks status <id>            # Detailed status
openclaw-scheduler tasks history [limit]        # Recently completed groups
openclaw-scheduler tasks heartbeat <id> <label> running|completed|failed [msg]
openclaw-scheduler tasks register-session <id> <label> <session-key>  # Enable auto-heartbeat

# ── Queue ─────────────────────────────────────────
openclaw-scheduler queue list [agent] [limit]    # Pending + delivered messages
openclaw-scheduler queue clear [agent]           # Mark all messages read
openclaw-scheduler queue prune                   # Prune old messages

# ── Team Adapter ─────────────────────────────────
openclaw-scheduler team map [limit]                          # Project team messages into events
openclaw-scheduler team tasks <team-id> [limit]              # List team tasks
openclaw-scheduler team events <team-id> [limit] [task-id]   # List team events
openclaw-scheduler team gate <team-id> <task-id> <members-json> [timeout-s]
openclaw-scheduler team check-gates [limit]                  # Evaluate task gates
openclaw-scheduler team ack <message-id> [actor] [note]      # Team-aware ACK

# ── Idempotency ──────────────────────────────────
openclaw-scheduler idem status <job-id>          # Recent idempotency keys
openclaw-scheduler idem check <key>              # Check if key is claimed
openclaw-scheduler idem release <key>            # Manually release a key
openclaw-scheduler idem prune                    # Force prune expired entries

# ── Delivery Aliases ──────────────────────────────
openclaw-scheduler alias list                 # List all aliases
openclaw-scheduler alias add <name> <channel> <target> [description]
openclaw-scheduler alias remove <name>

# ── Runtime Diagnostics ──────────────────────────
openclaw-scheduler status --json              # Live DB, lease, queue, outbox, approval state
openclaw-scheduler doctor --json              # Fail-closed schema and operational checks

# ── Schema Introspection ─────────────────────────
openclaw-scheduler schema jobs              # JSON schema for job fields (types, defaults, enums)
openclaw-scheduler schema runs              # Run statuses and key fields
openclaw-scheduler schema messages          # Message kinds and statuses
openclaw-scheduler schema approvals         # Approval statuses
openclaw-scheduler schema dispatches        # Dispatch kinds and statuses
openclaw-scheduler schema all               # Everything

# ── Status ────────────────────────────────────────
openclaw-scheduler status
openclaw-scheduler version                       # Print version (also: --version)
```

All CLI commands support `--json` for machine-readable output (useful for piping into `jq` or agent toolchains).

---

## Configuration

| Variable | Default | Description |
|----------|---------|-------------|
| `OPENCLAW_GATEWAY_URL` | `http://127.0.0.1:18789` | Gateway endpoint |
| `OPENCLAW_GATEWAY_TOKEN` | *(required)* | Gateway auth token |
| `OPENCLAW_GATEWAY_TOKEN_PATH` | `~/.openclaw/credentials/.gateway-token` | Path to a gateway token file under `~/.openclaw/credentials`, `/run/secrets`, or `/var/run/secrets` (used when `OPENCLAW_GATEWAY_TOKEN` is not set) |
| `SCHEDULER_HOME` | `~/.openclaw/scheduler` | Base dir for scheduler data when installed from npm or when the package dir is not a writable source checkout |
| `SCHEDULER_DB` | auto (existing `~/.openclaw/scheduler/scheduler.db` first; otherwise `./scheduler.db` in a writable source checkout; otherwise scheduler home) | SQLite database path |
| `SCHEDULER_BACKUP_STAGING_DIR` | `~/.openclaw/scheduler/.backup-staging` | Temp folder used by `backup.js` snapshot/restore |
| `SCHEDULER_TICK_MS` | `10000` | Tick interval in ms, from `1000` through `3600000` |
| `SCHEDULER_STALE_THRESHOLD_S` | `90` | Stale run threshold in seconds, from `10` through `604800` |
| `SCHEDULER_HEARTBEAT_CHECK_MS` | `30000` | Health check interval in ms, from `5000` through `3600000` |
| `SCHEDULER_MESSAGE_DELIVERY_MS` | `15000` | Message and spawn interval in ms, from `5000` through `3600000` |
| `SCHEDULER_DELIVERY_BATCH_SIZE` | `10` | Delivery batch size, from `1` through `1000` |
| `SCHEDULER_PRUNE_MS` | `3600000` | Prune interval in ms, from `60000` through `604800000` |
| `SCHEDULER_BACKUP_MS` | `300000` | MinIO backup interval in ms, from `60000` through `604800000` |
| `SCHEDULER_LEASE_TTL_MS` | `30000` | Dispatcher lease TTL in ms, from `15000` through `3600000` |
| `SCHEDULER_MAX_CONCURRENCY` | `4` | Worker concurrency, from `1` through `64` |
| `SCHEDULER_MAX_PENDING_WORK` | `1000` | Pending-work bound, from concurrency through `10000` |
| `SCHEDULER_BACKUP` | *(unset)* | `1` or `true` enables MinIO backups; `0` or `false` disables them |
| `SCHEDULER_BACKUP_MC_ALIAS` | `backupstore` | MinIO alias used by `mc` for backup snapshots |
| `SCHEDULER_BACKUP_BUCKET` | `scheduler-backups` | MinIO bucket for snapshots |
| `SCHEDULER_BACKUP_PREFIX` | `scheduler` | Object prefix inside bucket |
| `SCHEDULER_ARTIFACTS_DIR` | `~/.openclaw/scheduler/artifacts` | Directory for offloaded shell stdout/stderr files |
| `SCHEDULER_DEBUG` | *(unset)* | `1` or `true` enables debug logging; `0` or `false` disables it |
| `SCHEDULER_ALERT_TARGET` | *(unset; alerts disabled)* | Explicit numeric Telegram operator chat ID for repeated isolated-job failures. Surrounding whitespace is trimmed; zero, usernames, prefixes, embedded whitespace and unsafe integers are rejected. No default recipient or fallback route is used. Alerts are attempted only after successful completion bookkeeping commits. |
| `SCHEDULER_SHELL` | `/bin/zsh` (macOS), `/bin/bash` (Linux/WSL2) | Shell used for shell jobs |
| `SCHEDULER_PROVIDER_PATH` | *(unset)* | Directory of provider plugin `*.js` files loaded at startup. High trust boundary -- only point at operator-controlled code. See [gateway contract](docs/gateway-contract.md#local-provider-plugins) |
| `DISPATCH_CONFIG_DIR` | `~/.openclaw/dispatch` | Override dispatch config directory for `config.json` |
| `DISPATCH_STATE_DIR` | `~/.openclaw/scheduler/dispatch` | Allowed state root for the dispatch labels ledger |
| `DISPATCH_LABELS_PATH` | `<DISPATCH_STATE_DIR>/labels.json` | Override the labels ledger path. Relative paths resolve beneath the state root; absolute paths must remain beneath it after traversal normalization, and existing parents must remain beneath it after symbolic-link resolution |
| `DISPATCH_INDEX_PATH` | *(auto)* | Override path to dispatch/index.mjs (used by watcher) |
| `DISPATCH_HOST` | `hostname()` | Host identifier sent with dispatch hook events |
| `DISPATCH_WEBHOOK_URL` | *(unset)* | Webhook URL for dispatch lifecycle events (hooks.mjs) |
| `LOKI_PUSH_URL` | *(unset)* | Loki push endpoint for dispatch event logging (hooks.mjs) |
| `TELEGRAM_BOT_TOKEN` | *(unset)* | Bot token for webhook health check utility |
| `TELEGRAM_WEBHOOK_URL` | *(unset)* | Expected webhook URL for Telegram webhook check |
| `INBOX_AGENT` | `main` | Target agent for inbox-consumer.mjs |
| `INBOX_DELIVERY_CHANNEL` | *(unset)* | Delivery channel for inbox-consumer.mjs forwarding |
| `INBOX_DELIVERY_TO` | *(unset)* | Delivery target for inbox-consumer.mjs forwarding |
| `INBOX_LIMIT` | `10` | Batch size for inbox-consumer.mjs |

---

Provider-backed identity / authorization / proof behavior, including
provider-resolved `authorization_ref` and its fail-closed error semantics, is documented in the
[gateway contract](docs/gateway-contract.md#dispatch-time-authorization-evaluation).

---

## Service Management

> **Platform note:** The commands below are for macOS (launchd). For Linux, see [INSTALL-LINUX.md](INSTALL-LINUX.md). For Windows, see [INSTALL-WINDOWS.md](INSTALL-WINDOWS.md).

Choose the launchd mode that matches your host:
- **LaunchAgent**: best for a personal Mac that auto-logs in and should run the scheduler in your user session
- **LaunchDaemon**: best for a headless Mac or for starting the scheduler before login

Install either mode with the setup wizard:

```bash
openclaw-scheduler setup --service-mode agent
# or
openclaw-scheduler setup --service-mode daemon
```

For agent jobs that use profile/session preparation, supply the installed public OpenClaw
CLI entry through installation configuration (the executable `openclaw` entry or its
absolute `openclaw.mjs` target, not the Node binary):

```bash
OPENCLAW_CLI_PATH=/absolute/path/to/openclaw openclaw-scheduler setup --service-mode agent
```

Setup persists this explicit path in new LaunchAgent or LaunchDaemon environments.
For an existing service, it offers to update only `OPENCLAW_CLI_PATH`, preserving
all other plist and environment values. It does not reload the service; follow the
printed stop/bootstrap instructions in a maintenance window after active work and
callers are held. A `kickstart` alone does not reload the on-disk environment.
Without the setting, setup keeps its existing behavior and retains any already
configured CLI path. Relative paths and control characters are rejected. The CLI
must already exist and be executable under the service's preserved `PATH`/`HOME`;
setup does not install, launch, or authenticate it. The wizard still performs its
normal database/job/memory setup steps, so it is not a service-only activation tool.

### macOS LaunchAgent

```bash
# Start / bootstrap
launchctl bootstrap gui/$UID ~/Library/LaunchAgents/ai.openclaw.scheduler.plist

# Stop
launchctl bootout gui/$UID/ai.openclaw.scheduler

# Restart
launchctl kickstart -k gui/$UID/ai.openclaw.scheduler

# Status
launchctl print gui/$UID/ai.openclaw.scheduler
ps aux | grep dispatcher | grep -v grep

# Logs
tail -f /tmp/openclaw-scheduler.log

# Quick health
openclaw-scheduler status
```

### macOS LaunchDaemon

```bash
# Start / bootstrap
sudo launchctl bootstrap system /Library/LaunchDaemons/ai.openclaw.scheduler.plist

# Stop
sudo launchctl bootout system/ai.openclaw.scheduler

# Restart
sudo launchctl kickstart -k system/ai.openclaw.scheduler

# Status
sudo launchctl print system/ai.openclaw.scheduler
ps aux | grep dispatcher | grep -v grep

# Logs
tail -f /tmp/openclaw-scheduler.log

# Quick health
openclaw-scheduler status
```

Both modes use `RunAtLoad: true` and `KeepAlive: true`. LaunchDaemon also sets `UserName: <your-user>` so the service runs under your OpenClaw account while still surviving headless reboots.

---

## Error Handling & Backoff

### On dispatch failure

1. Run marked `error`, `consecutive_errors` increments
2. If `max_retries > 0` and retries remain → schedule retry (failure children wait)
3. If retries exhausted → trigger failure children, apply backoff

### Backoff schedule

| Consecutive errors | Delay |
|-------------------|-------|
| 1 | 30s |
| 2 | 1 min |
| 3 | 5 min |
| 4 | 15 min |
| 5+ | 1 hour |

Backoff is applied on top of the cron schedule (whichever is later). Resets to 0 on success.

### Stale run detection

- Every 30s, dispatcher checks if running runs still have active sessions
- No activity for 90s → marked `timeout`
- Fallback: runs exceeding `run_timeout_ms` are force-timed-out

### Gateway health

`GET /health` is checked before dispatch. If it is unreachable, isolated jobs
are deferred and shell jobs continue. Main-session jobs also require the
Gateway: default, `execute`, or `plan` jobs use synchronous agent execution,
while `fire-and-forget` jobs use `openclaw system event`. A failed synchronous
health check defers the job for 60 seconds; later request failures and
fire-and-forget failures use the configured retry behavior.

---

## Migration & History

### Importing from current OpenClaw

```bash
openclaw-scheduler migrate --dry-run --json
openclaw-scheduler migrate --json
```

Use `--legacy-json ~/.openclaw/cron/jobs.json` only for an old export.

### Schema baseline

The current package uses schema version 30. Schema v30 adds identifier-only
authoritative inbound source fields to durable watcher and watchdog jobs.

- Net-new installs apply `schema.sql` transactionally.
- Existing databases run the idempotent consolidation migration, then reapply the current schema. Migration 28 preserves existing state while rebuilding legacy completion debt rows with a derived delivery scope, multipart outbox coordinates, and immutable checksum evidence. Migration 29 adds handoff v4 artifacts, proof replay claims, runtime events, provider sessions, credential presentations, exact source-run bindings, and signed evidence metadata without rewriting earlier jobs or runs.
- Any schema or consolidation error aborts startup. It is never logged and ignored.

### Native-job ownership and release history

Disable only native jobs that have a verified scheduler replacement. Native
OpenClaw cron and heartbeat may continue serving unrelated work. Rollback stops
or disables the scheduler copies and re-enables those native jobs. See
[CHANGELOG.md](CHANGELOG.md) for release and schema history instead of relying
on duplicated version tables in this reference.

---

## Upgrading

Already have the scheduler running and need to update? See [UPGRADING.md](UPGRADING.md) for the full guide. Short version by platform:

### macOS (launchd, git-clone install)
```bash
cd ~/.openclaw/scheduler
git pull && npm install
npm run verify:local
launchctl kickstart -k gui/$(id -u)/ai.openclaw.scheduler
```

### Linux / Windows WSL2 (systemd, git-clone install)
```bash
cd ~/.openclaw/scheduler
git pull && npm install
npm run verify:local
systemctl --user restart openclaw-scheduler
```

---

## Removing the Scheduler

To stop the scheduler and restore OpenClaw's built-in cron/heartbeat, see [UNINSTALL.md](UNINSTALL.md).

Quick summary:
1. Stop the service (launchctl / systemctl / pm2)
2. Re-enable OC cron globally: `openclaw config set cron.enabled true` and remove `OPENCLAW_SKIP_CRON=1` from gateway service env
3. Re-enable OC cron jobs: `openclaw cron edit <id> --enable` for each job
4. Re-enable heartbeat: `openclaw config set agents.defaults.heartbeat.every "5m"` and restore any per-agent `agents.list[].heartbeat` overrides you use
5. Optionally delete `~/.openclaw/scheduler/`

---

## Best Practices

See [BEST-PRACTICES.md](BEST-PRACTICES.md) for:
- Choosing between `shell`, `isolated`, and `main` session targets
- Writing effective payload prompts for LLM jobs
- When to use chains vs standalone jobs
- Delivery mode selection
- How to integrate the scheduler with your OpenClaw agent
- Example MEMORY.md entries for agent awareness

---

## File Reference

```
~/.openclaw/scheduler/
│
│  Core scheduler
├── dispatcher.js          # Main process — tick loop, dispatch, chains, retry, backups
├── dispatcher-strategies.js  # Dispatch strategy functions (prepare, execute, finalize)
├── dispatcher-maintenance.js # Stale run reaping, TTL pruning, WAL checkpoints
├── dispatcher-approvals.js   # Approval timeout resolution and auto-approve/reject
├── dispatcher-delivery.js    # Post-run delivery pipeline (announce, announce-always)
├── dispatcher-shell.js       # Shell job execution and result normalization
├── dispatcher-runtime.js     # Singleton lease and bounded worker ownership
├── dispatcher-utils.js       # Shared dispatcher helpers and dependency wiring
├── dispatch-queue.js         # Leased durable dispatch queue and stale-claim recovery
├── runtime-lease.js          # Dispatcher lease and fencing token persistence
├── run-state.js              # Fenced cancellation and terminal run transitions
├── run-completion.js         # Atomic completion, job update, and child enqueue
├── delivery-outbox.js        # Transactional external delivery queue
├── attachment-store.js       # Durable attachment staging and integrity metadata
├── governance.js             # Execution-time governance enforcement
├── db.js                  # SQLite connection, fail-closed schema init, WAL checkpoints
├── schema.sql             # Complete schema (v28)
├── migrate-consolidate.js # Single migration for existing DBs through v28
├── jobs.js                # Job CRUD, cron, chains, cycle detection, resource pools, queue
├── runs.js                # Run lifecycle, stale/timeout, cancellation, context summary
├── messages.js            # Inter-agent message queue (priority, TTL, typed messages)
├── agents.js              # Agent registry
├── gateway.js             # OpenClaw API client (chat completions, events, delivery, aliases)
├── approval.js            # HITL approval queries
├── approval-state.js      # Atomic approval decisions and dispatch claims
├── idempotency.js         # Idempotency ledger (execution replay dedup)
├── retrieval.js           # Context retrieval (recent/hybrid run summaries)
├── task-tracker.js        # Dead-man's-switch for multi-agent sub-agent teams
├── team-adapter.js        # Team mailbox/task projection and task completion gates
├── backup.js              # MinIO snapshot/rollup/restore (requires `mc` CLI)
├── cli.js                 # CLI management tool
├── migrate.js             # Import through OpenClaw cron CLI or explicit legacy JSON
├── scripts/
│   ├── dispatch-cli-utils.mjs       # Dispatch CLI path resolution helpers
│   ├── inbox-consumer.mjs           # Drains queue messages and delivers to Telegram
│   ├── stuck-run-detector.mjs       # Detects stale running runs (alert-only via non-zero exit)
│   └── telegram-webhook-check.mjs   # Telegram webhook health check / repair utility
│
│  Service & docs
├── ~/Library/LaunchAgents/ai.openclaw.scheduler.plist  # macOS LaunchAgent location after install
├── /Library/LaunchDaemons/ai.openclaw.scheduler.plist  # macOS LaunchDaemon location after install
├── INSTALL.md             # Full installation guide — macOS (first host)
├── INSTALL-ADDITIONAL-HOST.md  # Installation guide for additional hosts
├── INSTALL-LINUX.md       # Installation guide for Linux (systemd user service)
├── INSTALL-WINDOWS.md     # Installation guide for Windows (WSL2 only)
├── UPGRADING.md           # Upgrade guide (all platforms)
├── UNINSTALL.md           # Removal guide (all platforms)
├── BEST-PRACTICES.md      # Job type selection, prompt writing, agent integration
├── QUICK-START.md         # Focused guide: install, convert crons, first workflow
├── openclaw-scheduler.service  # Linux systemd user service template
├── CHANGELOG.md           # Version history
└── README.md              # This file
```

---

## Testing

```bash
# Run the legacy suite, isolated focused tests, docs, and agentcli integration when present
npm test

# Require scheduler-owned and agentcli-owned integration suites
npm run test:agentcli

# Add lint, types, coverage, and package verification
npm run verify:local
```

The agentcli checkout defaults to `../agentcli`; set `AGENTCLI_PATH` when it is
elsewhere. A missing checkout is reported as an explicit local skip.
`SKIP_AGENTCLI_INTEGRATION=1 npm test` records an intentional scheduler-only
iteration. Hosted CI runs the scheduler-owned contract and the compatible
upstream-owned tests against exact public handoff-v3 and handoff-v2 commits. At
the v3 pin, the upstream test that hard-codes `field_version: "2"` is reported
as an explicit skip while all other upstream integration tests run.

Focused `tests/*.test.mjs` files run sequentially in separate processes and
databases so shared module state cannot contaminate another file.

### Test categories

- Schema creation & integrity
- Job CRUD, cron parsing, due detection
- Run lifecycle (create, heartbeat, finish, stale, timeout)
- Agent registry (upsert, status, capabilities)
- Message queue (priority, broadcast, TTL, typed messages)
- Cascade deletes, pruning
- Workflow chains (parent/child, trigger matching, tree traversal, trigger conditions)
- Cycle detection (self, deep)
- Max chain depth enforcement
- Retry tracking and sequencing
- Chain cancellation
- Shell job execution
- Approval gate lifecycle
- Idempotency key claiming/releasing
- Context retrieval (recent/hybrid)
- Dispatcher integration (full dispatch pipeline with mock gateway)

---

## Sub-agent Dispatch

The dispatch module (`dispatch/index.mjs`) spawns and steers isolated agent sessions via the OpenClaw Gateway API and tracks them by a human-readable label. Unlike the scheduler's job/run model, dispatch calls the gateway directly -- no scheduler tick delay, no DB write required to start a session. Each session is assigned a unique session key and recorded in the originating host's local `labels.json` ledger. When the agent calls `done` from that same local dispatch shell, final delivery is claimed by label, session, and run, then enqueued durably in `delivery_outbox`. The routed watcher uses the same claim and outbox path, so the two completion paths cannot both enqueue the same run. Do not run the completion marker from inside `ssh`, Docker, tmux, or another nested shell, because that can update a different label store. The watcher normalizes completions into short human-readable summaries, and it only falls back to terminal assistant output when the transcript contains strict clean completion evidence. The module also supports symlink-based branding: a wrapper directory (such as `my-brand`) contains a `config.json` with a custom name and a symlink to `dispatch/index.mjs`, giving the same CLI a different identity in notifications and logs.

Routed watchers write no delivery body to stdout after a durable enqueue and emit `WATCHER_ALREADY_DELIVERED` on stderr for the scheduler wrapper. Despite its legacy name, that marker means the completion was already enqueued and the wrapper must not enqueue it again; it is not a channel delivery receipt. The run-scoped completion debt stays open until every outbox part is delivered. A watcher with no configured delivery route retains its historical stdout behavior, but only after it acquires the durable run-scoped completion claim. If completion claim storage is unavailable, delivery fails closed with `COMPLETION_CLAIM_UNAVAILABLE`. Multipart output is split into independent outbox rows and can resume from its recorded per-part checkpoint.

### Quick Example

```bash
# Dispatch a sub-agent task and deliver the result to Telegram
openclaw-scheduler enqueue \
  --label       "fix-deploy-script"                                          \
  --message     "Fix the deploy script in ~/app to handle missing .env files" \
  --mode        fresh                                                         \
  --thinking    high                                                          \
  --timeout     3600                                                          \
  --source-context '{"channel":"telegram","target":"YOUR_CHAT_ID","messageId":"INBOUND_MESSAGE_ID","threadId":null}' \
  --deliver-to  YOUR_CHAT_ID                                                     \
  --delivery-mode announce

# Fallback (if openclaw-scheduler is not in PATH):
node ~/.openclaw/scheduler/dispatch/index.mjs enqueue \
  --label "fix-deploy-script" --message "..." \
  --source-context '{"channel":"telegram","target":"YOUR_CHAT_ID","messageId":"INBOUND_MESSAGE_ID"}' \
  --deliver-to YOUR_CHAT_ID

# Literal-safe prompt input when the text contains shell metacharacters:
cat prompt.md | openclaw-scheduler enqueue \
  --label "fix-deploy-script" \
  --message-stdin \
  --source-context '{"channel":"telegram","target":"YOUR_CHAT_ID","messageId":"INBOUND_MESSAGE_ID"}' \
  --deliver-to YOUR_CHAT_ID
```

For every chat-triggered dispatch, pass `--source-context` from the actual
inbound event. Its `channel`, `target`, and `messageId` are required;
`threadId` is optional. This envelope is the authoritative request source.
`--deliver-to` and `--deliver-channel` describe the durable completion target
and must match the source when both are present. A mismatch exits 2 before any
Gateway spawn, label mutation, watcher job, or notification. Legacy `--origin`
and delivery-only behavior remain available for manual/local calls where inbound
metadata is genuinely unavailable; active-session lookup is only such a
fallback.

### Flag Reference

| Flag | Default | Description |
|------|---------|-------------|
| `--label` | required | Human-readable name for the session. Used for status lookups, reuse, and watchdog tracking. |
| `--message` | required* | Prompt sent to the agent. |
| `--message-file` | -- | Path to a file whose contents are used as the prompt. Use `-` to read from stdin. Safer than inline shell quoting for prompts with backticks, quotes, or markdown. |
| `--message-env` | -- | Environment variable name whose value is used as the prompt. Useful when the prompt contains shell-significant characters. |
| `--message-stdin` | -- | Read the prompt from stdin explicitly. If stdin is piped and no explicit prompt source is set, dispatch auto-reads stdin. |
| `--mode` | `fresh` | `fresh` creates a new session. `reuse` continues the last session recorded for this label. |
| `--thinking` | -- | Reasoning budget: `low`, `high`, or `xhigh`. |
| `--model` | configured OpenClaw default | Model override, e.g. `anthropic/claude-sonnet-4-6`. When omitted, dispatch uses wrapper `config.defaultModel`, wrapper `config.dispatch.model`, `DISPATCH_DEFAULT_MODEL`, `agents.defaults.dispatch.model`, or `agents.defaults.model`; otherwise the Gateway selects its configured default. Rejected explicit overrides fail enqueue. |
| `--source-context` | -- | Authoritative inbound JSON envelope containing `channel`, `target`, `messageId`, and optional `threadId`. Required for chat-triggered callers; unknown fields are rejected so only identifiers are persisted. |
| `--deliver-to` | -- | Durable completion target (e.g. Telegram chat ID). Registers the scheduler watcher job for final delivery. With source context it must match `target`; delivery channel must also match. |
| `--delivery-mode` | `announce` | `announce` delivers only when output is non-empty. `announce-always` delivers unconditionally. `none` suppresses delivery. |
| `--timeout` | `300` | Session timeout in seconds. |
| `--monitor` | on | Auto-register a watchdog job that alerts if the session goes silent past the configured threshold. |
| `--no-monitor` | -- | Disable watchdog registration for this dispatch. |

*One prompt source is required: `--message`, `--message-file`, `--message-env`, `--message-stdin`, or piped stdin.

### Subcommand Reference

| Subcommand | Description |
|------------|-------------|
| `enqueue` | Spawn a new agent session (or resume one with `--mode reuse`) and optionally register a scheduler watcher for delivery. |
| `status` | Show current status for a label: session key, spawn time, running/done/error, and liveness data from the sessions store. |
| `stuck` | Check all running sessions against the stuck threshold. Exits 1 if genuinely stuck sessions remain after auto-resolving completed ones. |
| `result` | Retrieve the last assistant reply from a session transcript via `chat.history`. |
| `route` | Return the stored authoritative source envelope and durable delivery route for safe follow-up. |
| `sync` | Reconcile `labels.json` with sessions store state. Auto-marks sessions as done or error based on idle time. Supports `--dry-run`. |
| `done` | Agent-side completion signal. The agent calls this as its final action from the originating local dispatch shell to mark itself done immediately (push-based; no idle timeout wait). Do not call it from inside a remote or nested shell, because the label lookup is local to the dispatch host. The stored completion payload is normalized for channel-safe delivery, with checklist/sha metadata used as a fallback when the raw summary is generic or noisy. |
| `send` | Inject a message into a running session for mid-run steering. The agent sees it as a new user turn. |
| `steer` | Alias for `send`. The name makes steering intent explicit. |
| `heartbeat` | Check whether a session has been active within the last 10 minutes. Accepts `--label` or `--session-key`. |
| `list` | List all tracked labels in `labels.json`, sorted by most recent. Accepts `--status running|done|error` and `--limit`. |

`status`, `result`, and `route` expose `sourceContext` in machine-readable JSON
after completion. For a later follow-up, resolve the route from the ledger
instead of a remembered chat map:

```bash
route_json=$(openclaw-scheduler route --label fix-deploy-script)
source_channel=$(printf '%s' "$route_json" | jq -r '.sourceContext.channel')
source_target=$(printf '%s' "$route_json" | jq -r '.sourceContext.target')
source_thread=$(printf '%s' "$route_json" | jq -r '.sourceContext.threadId // empty')
```

Pass those stored identifiers to the messaging API used by the caller. The
scheduler never stores inbound message text or credentials in this envelope.

### Multi-agent Orchestration

The main agent acts as the orchestrator and delegates parallel units of work to sub-agents via `enqueue`. Each sub-agent runs in an isolated session, completes its assigned task, and calls `done` as its last action. Results are enqueued for durable outbox delivery to the requesting chat (Telegram, Discord, WhatsApp, Signal, iMessage, or Slack) without the orchestrator polling.

**Spawn depth constraint:** The gateway enforces `maxSpawnDepth: 2`. The main agent (depth 0) spawns sub-agents (depth 1), which can spawn nested sub-agents (depth 2). Depth 3 is blocked. Current OpenClaw derives and enforces depth in its native session/runtime policy; dispatch does not send the obsolete `sessions.patch spawnDepth` field.

**Example: 3 parallel workers**

```bash
# Orchestrator dispatches three workers in parallel.
# All three run concurrently in isolated sessions.
SOURCE_CONTEXT='{"channel":"telegram","target":"YOUR_CHAT_ID","messageId":"INBOUND_MESSAGE_ID"}'

openclaw-scheduler enqueue \
  --label   "worker-schema"   \
  --message "Review the DB schema and write documentation for all tables" \
  --thinking high --timeout 600 --source-context "$SOURCE_CONTEXT" --deliver-to YOUR_CHAT_ID

openclaw-scheduler enqueue \
  --label   "worker-frontend" \
  --message "Audit the React components for accessibility issues" \
  --thinking high --timeout 600 --source-context "$SOURCE_CONTEXT" --deliver-to YOUR_CHAT_ID

openclaw-scheduler enqueue \
  --label   "worker-docs"     \
  --message "Update the API docs to reflect the new /v2 endpoints" \
  --thinking high --timeout 600 --source-context "$SOURCE_CONTEXT" --deliver-to YOUR_CHAT_ID

# Each worker gets an outbox-delivered, channel-safe completion summary when done.
# No polling needed. Watchdog jobs are auto-registered for each.
```

Check status at any time:

```bash
openclaw-scheduler list --status running
openclaw-scheduler dispatch status --label worker-schema
```

### Branding and Configuration

`dispatch/index.mjs` resolves `config.json` relative to the directory of the invoking script, not the module itself. This means a symlink at `~/.openclaw/my-brand/index.mjs -> ~/.openclaw/scheduler/dispatch/index.mjs` will load `~/.openclaw/my-brand/config.json`, giving the same CLI a different brand name and defaults. All config fields are optional.

**`config.json` fields:**

| Field | Default | Description |
|-------|---------|-------------|
| `name` | `"dispatch"` | Brand name shown in Telegram notifications and log output. |
| `startupGraceMs` | `90000` | Grace period (ms) after spawn before stuck detection and auto-resolve activate. |
| `stuckThresholdMs` | `600000` | Silence duration (ms) before a session is considered stuck. |
| `maxWatcherAgeMs` | `7200000` | Max watcher process age (ms) before it is treated as stale. |
| `watchdogIntervalCron` | `"*/15 * * * *"` | Cron schedule for the auto-registered watchdog job. |
| `watchdogTimeoutMin` | `60` | Sessions running longer than this (minutes) without completing trigger a watchdog alert. |
| `deliver_watcher_ttl_hours` | `48` | TTL (hours) for scheduler-registered deliver-watcher jobs. These jobs are transient; they auto-prune once delivery is confirmed. Lower values prune faster; higher values retain audit history longer. |

**Environment variables:**

| Variable | Description |
|----------|-------------|
| `DISPATCH_STATE_DIR` | Allowed state root for dispatch tracking. Default: `~/.openclaw/scheduler/dispatch`. |
| `DISPATCH_LABELS_PATH` | Override path for `labels.json`. Default: `<DISPATCH_STATE_DIR>/labels.json`; the resolved path must remain beneath the state root. |
| `OPENCLAW_GATEWAY_TOKEN` | Gateway auth token. Falls back to `~/.openclaw/openclaw.json` if unset. |

**Minimal `config.json`:**

```json
{
  "name": "my-brand",
  "watchdogIntervalCron": "*/15 * * * *",
  "watchdogTimeoutMin": 60
}
```

### Monitoring and the Watchdog

When `--deliver-to` is set and `--no-monitor` is not passed, `enqueue` automatically registers a watchdog job in the scheduler DB alongside the delivery watcher job. The watchdog runs on the configured cron schedule and calls `stuck --threshold-min <watchdogTimeoutMin>` for the dispatched label. If the session has been silent past the threshold, the watchdog posts an alert to the configured delivery target and then disables itself.

Check active dispatch sessions:

```bash
# List all running dispatch sessions
openclaw-scheduler list --status running

# Check whether any session is stuck (exits 1 if found)
node ~/.openclaw/scheduler/dispatch/index.mjs stuck --threshold-min 15

# Status for a specific label
openclaw-scheduler dispatch status --label fix-deploy-script
```

The watchdog disarms itself automatically when the agent calls `done`, when `status` or `sync` auto-resolves the session from gateway idle state, or when `result` is fetched after a successful completion.

---

## Working with agentcli

[agentcli](https://github.com/amittell/agentcli) is the control-plane companion
for the scheduler. It provides manifest authoring, validation, local execution,
identity binding, and capability negotiation. The scheduler provides the durable
runtime: scheduling, retries, approvals, delivery, and persistent state.

The scheduler works without agentcli -- most jobs are created by the OpenClaw
agent itself when a user requests a scheduled task via Telegram or another
messaging channel, and operators can also create jobs directly via the CLI.
Adding agentcli on top gives you declarative workflow manifests, stable job IDs,
v0.2 identity and authorization support, evidence capability negotiation,
handoff v4 capability negotiation, and repeatable applies for
workflows that outgrow ad-hoc job creation.

Default job listings omit immutable v4 artifact payloads. Control-plane clients
that must validate an existing v4 job before updating it use
`openclaw-scheduler jobs list --include-handoff-artifacts --json`. The opt-in
read validates each persisted payload against its digest and fails the complete
request if any v4 artifact is missing or invalid; legacy jobs are returned
unchanged.

### Installing agentcli

```bash
npm install -g @amittell/agentcli
```

### Starting fresh with both tools

Write a manifest, validate it, then apply it to the scheduler:

```bash
# 1. Write a manifest
cat > my-workflow.json <<'JSON'
{
  "version": "0.1",
  "workflows": [{
    "id": "daily-ops",
    "name": "Daily Operations",
    "tasks": [{
      "id": "health-check",
      "name": "Morning Health Check",
      "prompt": "Run the daily health check and report any issues.",
      "target": { "session_target": "isolated", "agent_id": "main" },
      "schedule": { "cron": "0 9 * * *", "tz": "America/New_York" },
      "runtime": { "timeout_ms": 300000 },
      "delivery": { "mode": "announce", "channel": "telegram", "to": "YOUR_CHAT_ID" }
    }]
  }]
}
JSON

# 2. Validate locally (no scheduler needed)
agentcli validate my-workflow.json

# 3. Preview what would be created (dry-run)
agentcli apply my-workflow.json \
  --db ~/.openclaw/scheduler/scheduler.db \
  --scheduler-prefix ~/.openclaw/scheduler \
  --dry-run

# 4. Apply (creates the jobs)
agentcli apply my-workflow.json \
  --db ~/.openclaw/scheduler/scheduler.db \
  --scheduler-prefix ~/.openclaw/scheduler

# 5. Verify
openclaw-scheduler jobs list
```

### Adopting existing scheduler jobs

If you already have jobs created directly via `openclaw-scheduler jobs add` and
want to bring them under agentcli management:

1. Write a manifest with task names that match your existing job names exactly.

2. Run a one-time adoption by name:

```bash
agentcli apply my-workflow.json \
  --db ~/.openclaw/scheduler/scheduler.db \
  --scheduler-prefix ~/.openclaw/scheduler \
  --adopt-by name \
  --dry-run          # preview first

agentcli apply my-workflow.json \
  --db ~/.openclaw/scheduler/scheduler.db \
  --scheduler-prefix ~/.openclaw/scheduler \
  --adopt-by name    # execute adoption
```

This replaces each matched job with a new one under agentcli's stable ID
scheme (SHA256 of workflow_id:task_id). The old job is deleted after the
new one is created.

3. On subsequent applies, use the default (no `--adopt-by` flag). Jobs are
   matched by their stable ID, so the manifest can be renamed or reorganized
   without losing job mapping.

### Workflow chains

agentcli manifests support parent/child task relationships that compile to
scheduler trigger chains:

```json
{
  "version": "0.1",
  "workflows": [{
    "id": "deploy-pipeline",
    "name": "Deploy Pipeline",
    "tasks": [
      {
        "id": "build",
        "name": "Build",
        "shell": { "program": "sh", "args": ["-c", "npm run build"] },
        "target": { "session_target": "shell" },
        "schedule": { "cron": "0 2 * * *" },
        "runtime": { "timeout_ms": 600000 }
      },
      {
        "id": "deploy",
        "name": "Deploy",
        "shell": { "program": "sh", "args": ["-c", "fly deploy"] },
        "target": { "session_target": "shell" },
        "trigger": { "parent": "build", "on": "success" },
        "runtime": { "timeout_ms": 300000 }
      },
      {
        "id": "verify",
        "name": "Post-Deploy Verify",
        "prompt": "Verify all services are healthy after deploy.",
        "target": { "session_target": "isolated", "agent_id": "main" },
        "trigger": { "parent": "deploy", "on": "success" },
        "runtime": { "timeout_ms": 300000 },
        "delivery": { "mode": "announce-always", "channel": "telegram", "to": "YOUR_CHAT_ID" }
      }
    ]
  }]
}
```

This compiles to three scheduler jobs: Build runs on cron, Deploy triggers
on Build success, Verify triggers on Deploy success.

### v0.2 identity and authorization

agentcli v0.2 manifests add identity profiles, authorization proofs, evidence
declarations, and credential handoff. Identity, authorization, and supported
handoff fields compile to the scheduler's v0.2 runtime fields and are enforced
at dispatch time:

```json
{
  "version": "0.2",
  "identity_profiles": [{
    "id": "stripe-readonly",
    "provider": "stripe",
    "subject": { "kind": "service", "principal": "agent://payments/reader" },
    "auth": { "mode": "service", "scopes": ["read"] },
    "trust": { "level": "supervised" }
  }],
  "workflows": [{
    "id": "payment-ops",
    "tasks": [{
      "id": "check-balance",
      "name": "Check Balance",
      "shell": { "program": "sh", "args": ["-c", "stripe balance retrieve"] },
      "target": { "session_target": "shell" },
      "identity": { "ref": "stripe-readonly" },
      "contract": {
        "required_trust_level": "supervised",
        "trust_enforcement": "strict"
      },
      "schedule": { "cron": "0 9 * * *" },
      "runtime": { "timeout_ms": 60000 }
    }]
  }]
}
```

Credential materialization is target-specific. Shell jobs receive the exact
declared environment, temporary-file, or stdin bindings locally. Provider
output must match every artifact binding by name, medium, environment key,
file name, cardinality, and required state before any secret is exposed.
Isolated agent jobs first require the connected Gateway
to advertise `chat-completions-env-inject-v1`, then send the validated scoped
map through `x-openclaw-env-inject`. If the capability is absent, dispatch fails
closed with `GATEWAY_ENV_INJECT_UNSUPPORTED` before a credential-bearing request
is sent. Main-session jobs reject `identity.presentation` and
`credential_handoff`. Auth-profile-only isolated turns remain compatible with
Gateways that do not provide env injection.

Handoff v4 validates delegated identity chains against the exact source run,
resolves `authorization_ref` only through a configured provider, presents
credentials through one negotiated medium, and records append-only runtime
events. Its evidence envelope binds the canonical artifact, runtime instance,
identity, proof, authorization, complete output digests, postcondition, and
lineage. Offloaded stdout and stderr are hashed in full, not from their stored
excerpts. Historical verification reloads the exact artifact bound to the run.
The SSH
provider signs and verifies with `ssh-keygen -Y`; other declared providers must
implement equivalent verification. `openclaw-scheduler runs evidence RUN_ID
--json` re-verifies the persisted envelope against the stored execution.

Earlier handoff consumers and direct scheduler job specifications retain the
immutable canonical SHA-256 checksum backend. That legacy path remains a
separate capability and is never used to downgrade a v4 signed-evidence
declaration.

`openclaw-scheduler capabilities --json` reports `handoff_version: "4"`, the
exact top-level `handoff_contract` canonicalization and binding versions, and
the enforcement features `root_approval_gate`,
`approval_scope_enforcement: false`,
`structured_output_format`, `delegation_validation`,
`authorization_ref_resolution`, `evidence_generation: true`,
`checksum_evidence_generation: true`,
`evidence_integrity: "artifact-bound-signed-or-provider-verified-v4"`,
`evidence_contract: "agentcli-handoff-v4"`, `handoff_v4_artifact`,
`artifact_bound_proofs`, `signed_or_provider_verified_evidence`,
`provider_session_cache`, `credential_presentation`,
`source_run_bound_delegation`, `immutable_runtime_events`,
`gateway_capability_discovery`, `gateway_env_injection_negotiation`,
`multipart_delivery_checkpoints: true`, and
`completion_delivery_scope: "run"`.

See the [agentcli examples directory](https://github.com/amittell/agentcli/tree/main/examples)
for fully annotated manifests covering Stripe, Fly.io, Terraform, GitHub CLI,
and more. The executable public v4 flow is
[`tests/handoff-v4-e2e.test.mjs`](tests/handoff-v4-e2e.test.mjs); shared positive
and negative protocol vectors are under `fixtures/handoff-v4/`.

### Environment variables

| Variable | Default | Description |
|----------|---------|-------------|
| `AGENTCLI_SCHEDULER_DB` | *(none)* | Path to scheduler SQLite database |
| `AGENTCLI_SCHEDULER_PREFIX` | *(none)* | npm prefix where scheduler is installed |
| `AGENTCLI_SCHEDULER_BIN` | *(none)* | Direct path to scheduler CLI binary |
| `AGENTCLI_TARGET` | `standalone` | Default compilation target (`standalone` or `openclaw-scheduler`) |
| `AGENTCLI_OUTPUT` | *(none)* | Output format (`json` or `ndjson`) |

### Key commands

```bash
agentcli validate manifest.json           # Check manifest validity
agentcli compile manifest.json \
  --target openclaw-scheduler --explain   # Preview compiled job specs
agentcli apply manifest.json \
  --db path/to/scheduler.db --dry-run     # Preview changes
agentcli apply manifest.json \
  --db path/to/scheduler.db               # Create/update jobs
agentcli inspect jobs                     # List managed jobs
agentcli exec manifest.json task-id \
  --dry-run --signer none                 # Local execution (no scheduler)
```

---

## Trust Architecture

The scheduler acts as a control-plane broker for child execution principals.
Child tasks are bounded actors that receive only the credentials the scheduler
gives them and cannot escalate their own authority. The credential model
supports both precreated scoped keys and dynamic per-task key minting via
identity providers.

For the full trust model -- including when the scheduler/child boundary is a
real security boundary vs. an operational one, the credential flow from
operator to child, and what the model does and does not guarantee -- see
[docs/trust-architecture.md](docs/trust-architecture.md).

---

## Troubleshooting

### Dispatcher isn't dispatching

```bash
ps aux | grep dispatcher              # Is it running?
tail -20 /tmp/openclaw-scheduler.log   # Any errors?
curl http://127.0.0.1:18789/health     # Gateway reachable?
openclaw-scheduler jobs list                  # Is nextRun in the past?
openclaw-scheduler runs running               # Overlap blocking?
```

### Wrong next_run_at

All dates must be SQLite format (`YYYY-MM-DD HH:MM:SS`, UTC). `nextRunFromCron()` handles this. If manually setting dates, don't use ISO format with `T`/`Z`.

### Force a job to run now

```bash
openclaw-scheduler jobs run <job-id>
```

### Check schema version

```bash
openclaw-scheduler doctor --json
```

### Service won't start

```bash
# LaunchAgent
plutil -lint ~/Library/LaunchAgents/ai.openclaw.scheduler.plist
launchctl bootstrap gui/$UID ~/Library/LaunchAgents/ai.openclaw.scheduler.plist
launchctl print gui/$UID/ai.openclaw.scheduler

# LaunchDaemon
sudo plutil -lint /Library/LaunchDaemons/ai.openclaw.scheduler.plist
sudo launchctl bootstrap system /Library/LaunchDaemons/ai.openclaw.scheduler.plist
sudo launchctl print system/ai.openclaw.scheduler
```

### Logs not updating

Dispatcher logs to stderr (unbuffered). If logs look stale, the process may have crashed. Check the service that matches your launchd mode:
- LaunchAgent: `launchctl print gui/$UID/ai.openclaw.scheduler`
- LaunchDaemon: `sudo launchctl print system/ai.openclaw.scheduler`

### Job shows 'awaiting_approval'

```bash
openclaw-scheduler approvals list --json
openclaw-scheduler approvals approve <approval-id> --reason "Reviewed"
# Or: openclaw-scheduler approvals reject <approval-id> --reason "Not ready"
```

### Backup failing

```bash
mc alias list   # verify backupstore alias configured
# Check: SCHEDULER_BACKUP_MC_ALIAS, SCHEDULER_BACKUP_BUCKET, SCHEDULER_BACKUP_PREFIX env vars or defaults in backup.js
# Verify MinIO is reachable: mc ls backupstore/
```

---

## Companion Scripts

The `scripts/` directory contains optional operational helpers built on top of core scheduler primitives.

These scripts are not required for scheduling itself, but they are useful for production operations:
- `scripts/inbox-consumer.mjs` drains queued messages and delivers them to Telegram.
- `scripts/stuck-run-detector.mjs` detects stale `running` runs and exits non-zero for alerting.

### Signal Queue Pattern

The message queue (`messages` table) plus `cli.js msg send` implements a **signal-only** delivery path that complements `delivery_mode: announce`:

```
Failure path:  dispatcher → announce → Telegram (immediate, unconditional)
Signal path:   script → cli.js msg send → queue → Inbox Consumer → Telegram
```

Scripts write to the queue **only when they have found something** — not unconditionally. A companion `scripts/inbox-consumer.mjs` shell job (run every 5 min) drains the queue and delivers to Telegram. It exits 0 when the queue is empty, so there is no noise.

> **Important:** The dispatcher does **not** write to the message queue automatically.
> Every message in the queue was put there by a script with a specific receiver in mind.
> Traceability for completed jobs comes from the `runs` table, `delivery_mode: announce`,
> and run history/CLI views — not from queued messages.
