# The Grid - Daemon Mode Architecture

Technical design document for long-running autonomous execution in The Grid.

---

## Executive Summary

Daemon Mode enables The Grid to execute complex, multi-hour tasks without requiring an active user session. Users can "fire and forget" large projects, check progress asynchronously, and receive notifications when work completes or requires attention.

**Key Capabilities:**
- Long-running autonomous execution (hours to days)
- Process survival across terminal closures
- Progress monitoring and status checks
- Graceful pause, resume, and cancellation
- Crash recovery and checkpoint-based resumption

---

## Current State Analysis

### What The Grid Has Today

| Feature | Current State |
|---------|---------------|
| **State Persistence** | `.grid/STATE.md` - survives terminal close |
| **Checkpoint Protocol** | Programs return structured checkpoints |
| **Warmth Transfer** | `lessons_learned` in SUMMARY.md |
| **Scratchpad** | Live discovery sharing via `.grid/SCRATCHPAD.md` |
| **Session Resume** | Manual via `/grid` checking STATE.md |

### Current Limitations

1. **Session-Bound Execution**: Work stops when user closes terminal
2. **No Background Mode**: Can't run while user does other work
3. **Manual Resume Required**: User must explicitly restart after pause
4. **No Notifications**: No way to alert user when work completes
5. **Context Window Bound**: Single-session context limits (200k tokens)

---

## Architecture Overview

### Three-Layer Design

```
┌─────────────────────────────────────────────────────────────────────┐
│                        DAEMON CONTROLLER                             │
│  Lightweight process manager that survives terminal disconnection    │
│  - Spawns/monitors Claude Code processes                            │
│  - Manages checkpoint persistence                                    │
│  - Handles notifications                                             │
└─────────────────────────────────────────────────────────────────────┘
                                ↓
┌─────────────────────────────────────────────────────────────────────┐
│                        SESSION ORCHESTRATOR                          │
│  Master Control instance managing execution waves                    │
│  - Coordinates parallel Programs                                     │
│  - Handles inter-session state transfer                             │
│  - Implements durable execution patterns                            │
└─────────────────────────────────────────────────────────────────────┘
                                ↓
┌─────────────────────────────────────────────────────────────────────┐
│                        WORKER PROGRAMS                               │
│  Fresh Claude Code instances for actual work                         │
│  - Planners, Executors, Recognizers, etc.                           │
│  - Each gets fresh 200k context window                              │
│  - Reports progress to Orchestrator                                 │
└─────────────────────────────────────────────────────────────────────┘
```

### Daemon Controller (Process Layer)

The Daemon Controller is a lightweight Node.js process that:

1. **Spawns Claude Code sessions** via CLI
2. **Monitors process health** with heartbeats
3. **Persists state** to disk on every checkpoint
4. **Survives disconnection** via `nohup` or similar
5. **Sends notifications** via system notifications, webhooks, or email

```javascript
// Conceptual: daemon-controller.js
class DaemonController {
  constructor(config) {
    this.stateFile = '.grid/daemon/state.json';
    this.logFile = '.grid/daemon/daemon.log';
    this.notificationHandler = new NotificationHandler(config);
  }

  async spawn(taskDescription) {
    // 1. Create daemon state
    const daemonId = generateId();
    await this.persistState({ id: daemonId, status: 'starting' });

    // 2. Spawn Claude Code in headless mode
    const proc = spawn('claude', [
      '--agent', 'grid-daemon-orchestrator',
      '--input', taskDescription,
      '--output', `.grid/daemon/${daemonId}/output.md`
    ], { detached: true });

    // 3. Detach from terminal
    proc.unref();

    return daemonId;
  }

  async checkStatus(daemonId) {
    const state = await this.loadState(daemonId);
    return {
      status: state.status,
      progress: state.progress,
      lastCheckpoint: state.lastCheckpoint,
      logs: await this.tailLogs(daemonId, 20)
    };
  }
}
```

### Session Orchestrator (Coordination Layer)

An enhanced Master Control that implements **durable execution**:

```
SESSION LIFECYCLE
─────────────────

1. INITIALIZE
   ├── Load daemon state from disk
   ├── Verify last checkpoint integrity
   └── Determine resume point

2. EXECUTE WAVE
   ├── Spawn Programs (parallel within wave)
   ├── Monitor via scratchpad polling
   ├── Checkpoint after each Program completes
   └── Persist full state to disk

3. CHECKPOINT (after every wave)
   ├── Serialize: conversation history, plan state, warmth
   ├── Write to `.grid/daemon/{id}/checkpoint.json`
   ├── Update progress in daemon state
   └── Notify controller of progress

4. HANDLE INTERRUPTION
   ├── If graceful: complete current Program, checkpoint, exit
   ├── If crash: controller detects via heartbeat timeout
   └── Resume from last checkpoint on restart

5. COMPLETE
   ├── Final checkpoint with COMPLETE status
   ├── Generate summary report
   ├── Notify user
   └── Clean up or archive daemon state
```

### State Persistence Model

```yaml
# .grid/daemon/{daemon-id}/checkpoint.json
{
  "version": "1.0",
  "daemon_id": "20260123-143000-build-auth-api",
  "created": "2026-01-23T14:30:00Z",
  "updated": "2026-01-23T16:45:00Z",

  "status": "executing",  # starting | executing | paused | checkpoint | complete | failed

  "task": {
    "description": "Build REST API with user authentication",
    "mode": "autopilot",
    "original_prompt": "..."
  },

  "progress": {
    "current_wave": 2,
    "total_waves": 4,
    "completed_blocks": ["01", "02", "03"],
    "current_block": "04",
    "percent": 65
  },

  "execution_state": {
    "plan_data": { /* Full Planner output */ },
    "completed_summaries": { /* Block SUMMARY.md contents */ },
    "warmth": { /* Accumulated lessons_learned */ },
    "scratchpad_archive": [ /* All scratchpad entries */ ]
  },

  "checkpoint_stack": [
    {
      "type": "human-verify",
      "block": "04",
      "details": { /* Checkpoint data */ },
      "created": "2026-01-23T16:45:00Z"
    }
  ],

  "metrics": {
    "start_time": "2026-01-23T14:30:00Z",
    "elapsed_seconds": 8100,
    "programs_spawned": 12,
    "commits_made": 8,
    "estimated_remaining_seconds": 4500
  }
}
```

---

## Durable Execution Implementation

### Checkpoint Protocol

Based on research into durable execution patterns, checkpointing occurs at these boundaries:

| Event | Checkpoint Contents | Recovery Action |
|-------|---------------------|-----------------|
| **Wave Complete** | Full state, all summaries | Resume next wave |
| **Program Complete** | Program output, warmth | Resume wave |
| **User Checkpoint** | Checkpoint data, pause reason | Wait for user |
| **Crash** | Last known state | Verify + resume |
| **Graceful Stop** | Full state + stop reason | Resume on restart |

### Heartbeat & Health Monitoring

```
HEARTBEAT PROTOCOL
──────────────────

Orchestrator writes heartbeat every 30 seconds:
  .grid/daemon/{id}/heartbeat.json
  {
    "timestamp": "2026-01-23T16:45:30Z",
    "status": "executing",
    "current_action": "Spawning executor-03"
  }

Controller considers Orchestrator dead if:
  - No heartbeat for 2 minutes
  - Process not found in system

Recovery:
  1. Controller reads last checkpoint
  2. Spawns new Orchestrator with checkpoint
  3. Orchestrator verifies git state
  4. Resumes from checkpoint
```

### Crash Recovery

```python
def recover_from_crash(daemon_id):
    """Recovery protocol after unexpected termination."""

    # 1. Load last checkpoint
    checkpoint = load_checkpoint(daemon_id)

    # 2. Verify git state matches checkpoint
    actual_commits = get_git_commits_since(checkpoint['task']['start_time'])
    expected_commits = checkpoint['execution_state']['completed_summaries']

    if commits_match(actual_commits, expected_commits):
        # Clean recovery - resume from checkpoint
        return spawn_orchestrator(checkpoint, mode='resume')
    else:
        # Dirty state - need reconciliation
        return spawn_orchestrator(checkpoint, mode='reconcile')

def reconcile_state(checkpoint, actual_git_state):
    """Reconcile checkpoint with actual git state."""

    # Find divergence point
    last_matching_commit = find_last_matching(checkpoint, actual_git_state)

    # Option A: Trust git, update checkpoint
    # Option B: Trust checkpoint, revert git (dangerous)
    # Default: Trust git, log discrepancy

    updated_checkpoint = rebuild_from_git(last_matching_commit)
    return updated_checkpoint
```

---

## Multi-Session Context Management

### The Context Window Problem

A single Claude Code session is limited to ~200k tokens. Complex projects exceed this. Solution: **session chaining with warmth transfer**.

```
SESSION 1 (Waves 1-2)          SESSION 2 (Waves 3-4)
┌─────────────────────┐        ┌─────────────────────┐
│ Fresh 200k context  │        │ Fresh 200k context  │
│                     │        │                     │
│ - Execute Wave 1    │        │ - Load checkpoint   │
│ - Execute Wave 2    │   →    │ - Apply warmth      │
│ - Checkpoint        │        │ - Execute Wave 3    │
│ - Extract warmth    │        │ - Execute Wave 4    │
│ - Terminate         │        │ - Complete          │
└─────────────────────┘        └─────────────────────┘
         ↓                              ↑
    checkpoint.json ────────────────────┘
```

### Session Handoff Protocol

```python
def handoff_to_new_session(current_checkpoint):
    """Hand off to fresh session when context exhausted."""

    # 1. Save current state
    save_checkpoint(current_checkpoint)

    # 2. Extract warmth (compressed learnings)
    warmth = extract_warmth(current_checkpoint)

    # 3. Terminate current session gracefully
    terminate_session()

    # 4. Spawn fresh session with minimal context
    new_session = spawn_orchestrator({
        'checkpoint_path': current_checkpoint.path,
        'warmth': warmth,
        'mode': 'continue'
    })

    return new_session
```

### Warmth Compression

To fit learnings into new context windows, warmth is compressed:

```yaml
# Full warmth (too large for handoff)
lessons_learned:
  codebase_patterns:
    - "Uses barrel exports in src/index.ts"
    - "API routes in src/app/api/*/route.ts"
    - "Uses Zod for validation everywhere"
    - "Prisma client in src/lib/db.ts"
    - ... (50 more patterns)

# Compressed warmth (fits in context)
warmth_compressed:
  patterns: "barrel exports, Zod validation, Prisma in lib/db"
  gotchas: "auth middleware runs first, timestamps UTC"
  decisions: "chose JWT over sessions for statelessness"
  critical_files: ["src/lib/auth.ts", "prisma/schema.prisma"]
```

---

## User Interaction Patterns

### Fire-and-Forget Launch

```bash
# User launches daemon
/grid:daemon "Build complete e-commerce platform with Stripe integration"

# Grid responds
DAEMON SPAWNED
══════════════

ID: 20260123-143000-ecommerce
Task: Build complete e-commerce platform with Stripe integration
Mode: Autopilot

Status: Planning phase
Monitor: /grid:daemon status
Stop: /grid:daemon stop

You can close this terminal. Work continues in background.

End of Line.
```

### Status Checking

```bash
# From any terminal
/grid:daemon status

# Output
DAEMON STATUS
═════════════

ID: 20260123-143000-ecommerce
Runtime: 2h 15m
Status: Executing (Wave 3 of 5)

Progress: [████████████░░░░░░░░] 60%

Current: Block 07 - Payment Integration
  ├─ Thread 7.1: Stripe SDK setup ✓
  ├─ Thread 7.2: Checkout flow ⚡ In Progress
  └─ Thread 7.3: Webhook handlers ○ Pending

Recent Activity:
  16:42 - executor-07: Implementing checkout session creation
  16:38 - executor-06: Completed product catalog API
  16:30 - recognizer: Verified Wave 2 artifacts ✓

Commits: 14 made
Est. Remaining: ~1h 30m

End of Line.
```

### Checkpoints (User Attention Required)

```bash
# User gets notification (system notification, webhook, etc.)
# "Grid Daemon needs your attention"

/grid:daemon status

# Output
DAEMON CHECKPOINT
═════════════════

ID: 20260123-143000-ecommerce
Status: AWAITING USER

Checkpoint Type: human-verify
Block: 08 - Stripe Webhooks

What was built:
- Stripe webhook endpoint at /api/webhooks/stripe
- Event handlers for payment_intent.succeeded, .failed
- Signature verification middleware

How to verify:
1. Run: stripe listen --forward-to localhost:3000/api/webhooks/stripe
2. In another terminal: stripe trigger payment_intent.succeeded
3. Check logs show "Payment succeeded" event processed

Resume: /grid:daemon resume "approved"
   Or: /grid:daemon resume "Issue: webhook not receiving events"

End of Line.
```

### Resume After Checkpoint

```bash
/grid:daemon resume "approved"

# Output
DAEMON RESUMED
══════════════

Checkpoint cleared. Continuing execution...

Current: Block 09 - Order Management
Status: Executing

End of Line.
```

---

## Notification System

### Notification Triggers

| Event | Default Notification | Configurable |
|-------|---------------------|--------------|
| **Daemon Started** | Log only | Yes |
| **Wave Complete** | None | Yes |
| **Checkpoint Reached** | System notification | Yes |
| **Error/Failure** | System notification + sound | Yes |
| **Daemon Complete** | System notification | Yes |
| **Stall Detected** | After 30min inactivity | Yes |

### Notification Channels

```yaml
# .grid/config.json
{
  "daemon": {
    "notifications": {
      "system": true,           # macOS/Windows native notifications
      "sound": true,            # Audio alert on checkpoint/complete
      "webhook": null,          # POST to URL on events
      "email": null,            # Email notifications (requires setup)
      "slack": null             # Slack webhook URL
    },
    "notify_on": {
      "checkpoint": true,
      "complete": true,
      "error": true,
      "wave_complete": false,
      "stall": true
    },
    "stall_threshold_minutes": 30
  }
}
```

### System Notification Implementation

```javascript
// Using node-notifier for cross-platform notifications
const notifier = require('node-notifier');

function notifyUser(event) {
  notifier.notify({
    title: 'The Grid',
    message: formatEventMessage(event),
    icon: path.join(__dirname, 'grid-icon.png'),
    sound: event.type === 'checkpoint' || event.type === 'complete',
    wait: event.type === 'checkpoint'  // Keep notification until dismissed
  });
}
```

---

## Implementation Phases

### Phase 1: Foundation (Current Claude Code Capabilities)

**What's possible today:**

1. **Manual daemon pattern** using `nohup claude ... &`
2. **State persistence** via existing `.grid/STATE.md`
3. **Checkpoint-based resume** via `/grid` reading STATE.md
4. **Background agents** via Claude Code v2.0.60+ `Ctrl+B`

**Implementation:**
- Enhance STATE.md with daemon-specific fields
- Create `/grid:daemon` command that sets up state and runs in background
- Use Claude Code's native background agent support where available

### Phase 2: Process Management (Requires External Tooling)

**Needs:**
- Daemon controller process (Node.js or shell script)
- Process monitoring and heartbeat
- Crash recovery automation

**Implementation:**
```bash
# grid-daemon-launcher.sh
#!/bin/bash
DAEMON_ID=$(date +%Y%m%d-%H%M%S)-$(echo "$1" | tr ' ' '-' | head -c 20)
DAEMON_DIR=".grid/daemon/$DAEMON_ID"
mkdir -p "$DAEMON_DIR"

# Save task
echo "$1" > "$DAEMON_DIR/task.txt"

# Launch Claude Code in background
nohup claude --print --dangerouslySkipPermissions \
  -p "$(cat ~/.claude/commands/grid/daemon-executor.md)" \
  --input "$1" \
  > "$DAEMON_DIR/output.log" 2>&1 &

echo $! > "$DAEMON_DIR/pid"
echo "Daemon $DAEMON_ID started"
```

### Phase 3: Full Daemon Mode (Requires Claude Code Changes)

**Would need from Claude Code:**
- Native daemon/service mode
- IPC for status queries
- Built-in notification system
- Multi-session orchestration

**Proposal for Claude Code team:**
```
Feature Request: Daemon Mode for Claude Code

Use Case: Long-running autonomous development tasks

Requested Capabilities:
1. `claude daemon start "task"` - Launch headless session
2. `claude daemon status <id>` - Query running daemon
3. `claude daemon stop <id>` - Graceful termination
4. `claude daemon list` - Show all running daemons
5. Automatic checkpoint/resume on crash
6. Native system notifications
```

---

## Security Considerations

### Sandboxing

Daemon mode inherits Claude Code's permission model:
- `--dangerouslySkipPermissions` should NOT be used in daemon mode
- Each operation still requires appropriate permissions
- File system access limited to project directory

### Resource Limits

```yaml
# Daemon resource configuration
daemon:
  max_runtime_hours: 24          # Hard limit on execution time
  max_programs_parallel: 5       # Limit concurrent Programs
  max_commits_per_hour: 20       # Rate limit commits
  max_file_modifications: 100    # Safety limit on file changes
  require_approval_after: 50     # Force checkpoint after N commits
```

### Audit Trail

All daemon activity logged to `.grid/daemon/{id}/audit.log`:
```
2026-01-23T14:30:00Z | START | Task: "Build e-commerce platform"
2026-01-23T14:32:15Z | SPAWN | Planner (model: opus)
2026-01-23T14:35:42Z | PLAN | 12 blocks, 5 waves
2026-01-23T14:36:00Z | SPAWN | Executor-01 (block: 01)
2026-01-23T14:42:18Z | COMMIT | abc123 "feat(01): Initialize project"
...
```

---

## Failure Modes & Mitigations

| Failure Mode | Detection | Mitigation |
|--------------|-----------|------------|
| **Claude Code crash** | Heartbeat timeout | Auto-restart from checkpoint |
| **System reboot** | Daemon controller starts on boot | Resume from checkpoint |
| **Context exhaustion** | Token count monitoring | Session handoff |
| **API rate limit** | 429 response | Exponential backoff |
| **Git conflict** | Merge failure | Checkpoint, alert user |
| **Infinite loop** | Stall detection | Alert user, pause |
| **Permission denied** | Operation failure | Checkpoint, alert user |

---

## Metrics & Observability

### Daemon Metrics

```yaml
# Exposed via /grid:daemon metrics
metrics:
  runtime_seconds: 8100
  programs_spawned: 12
  programs_failed: 0
  commits_made: 8
  files_created: 24
  files_modified: 15
  lines_written: 2847
  checkpoints_hit: 3
  checkpoint_wait_seconds: 120
  context_resets: 1
  warmth_transfers: 1
```

### Health Dashboard (Future)

```
DAEMON HEALTH
═════════════

Active Daemons: 2

┌─────────────────────────────────────────────────────────────────────┐
│ ID: ecommerce-build     Status: EXECUTING     Health: ●●●●○        │
│ Runtime: 2h 15m         Progress: 60%         Est: 1h 30m          │
├─────────────────────────────────────────────────────────────────────┤
│ ID: api-refactor        Status: CHECKPOINT    Health: ●●●●●        │
│ Runtime: 45m            Progress: 80%         Waiting: human-verify│
└─────────────────────────────────────────────────────────────────────┘

System Resources:
  CPU: 12% (Claude Code processes)
  Memory: 2.1 GB
  Disk: 142 MB (.grid/ state)
```

---

## Future Enhancements

### Distributed Execution

Multiple machines contributing to single daemon:
- Shared state via cloud storage
- Work distribution via queue
- Merge reconciliation

### Learning Across Daemons

Global warmth database:
- Patterns learned across all daemons
- Shared gotchas and best practices
- Per-project and global layers

### Scheduled Daemons

Cron-like scheduling:
```bash
/grid:daemon schedule "daily at 3am" "Run test suite and fix failures"
```

### Daemon Chaining

Sequential daemon execution:
```bash
/grid:daemon chain \
  "Build feature X" \
  "Write tests for feature X" \
  "Update documentation"
```

---

## Appendix A: State File Schemas

### checkpoint.json Schema

```json
{
  "$schema": "http://json-schema.org/draft-07/schema#",
  "type": "object",
  "required": ["version", "daemon_id", "status", "task", "progress"],
  "properties": {
    "version": { "type": "string" },
    "daemon_id": { "type": "string" },
    "created": { "type": "string", "format": "date-time" },
    "updated": { "type": "string", "format": "date-time" },
    "status": {
      "type": "string",
      "enum": ["starting", "executing", "paused", "checkpoint", "complete", "failed"]
    },
    "task": {
      "type": "object",
      "properties": {
        "description": { "type": "string" },
        "mode": { "type": "string" },
        "original_prompt": { "type": "string" }
      }
    },
    "progress": {
      "type": "object",
      "properties": {
        "current_wave": { "type": "integer" },
        "total_waves": { "type": "integer" },
        "completed_blocks": { "type": "array", "items": { "type": "string" } },
        "current_block": { "type": "string" },
        "percent": { "type": "integer" }
      }
    }
  }
}
```

---

## Appendix B: Claude Code Feature Requests

For full daemon mode capability, The Grid would benefit from these Claude Code enhancements:

1. **Native daemon mode**: `claude daemon` subcommand
2. **IPC channel**: Query running sessions without terminal
3. **Notification API**: Hook into system notifications
4. **Session serialization**: Export/import session state
5. **Headless operation**: Run without TTY requirement
6. **Multi-session coordination**: Built-in session chaining

---

*Document Version: 1.0*
*Last Updated: 2026-01-23*
*Author: Grid Program 1 (Daemon Architecture Specialist)*

End of Line.