# Grid Persistence - Implementation Guide

**Version:** 1.0
**Date:** 2026-01-23

This guide explains how to implement and use Grid's persistence system.

## Quick Start

### Initialize a Project

```bash
/grid:init
```

This creates the complete `.grid/` directory structure with all state files.

### Resume an Interrupted Mission

```bash
/grid:resume
```

This reconstructs context from `.grid/` files and continues from the last checkpoint.

## Implementation Status

### Phase 1: Core Persistence (REQUIRED)

- [x] Template files created
- [x] `/grid:resume` added to help
- [ ] STATE.md write on every wave complete
- [ ] CHECKPOINT.md write on checkpoint/interrupt
- [ ] WARMTH.md aggregation after block complete
- [ ] `/grid:resume` command full implementation
- [ ] Basic context reconstruction

### Phase 2: Enhanced Recovery

- [ ] Session death detection via scratchpad staleness
- [ ] Git-based state reconstruction
- [ ] Corrupted state recovery
- [ ] Rollback support

### Phase 3: Advanced Features

- [ ] Multi-cluster support
- [ ] State diff visualization
- [ ] Time-travel debugging
- [ ] Cross-session analytics

## File Templates Location

All templates are in: `/Users/jacweath/grid/templates/grid-state/`

- `STATE.md` - Central state tracking
- `WARMTH.md` - Institutional knowledge
- `SCRATCHPAD.md` - Live discoveries
- `DECISIONS.md` - User decisions
- `BLOCKERS.md` - Blocker tracking
- `CHECKPOINT.md` - Interrupted thread state
- `config.json` - Grid configuration
- `BLOCK-SUMMARY.md` - Completed block record

## Integration Points

### Master Control (mc.md)

Master Control must:

1. **On wave complete:**
   - Update STATE.md with new position
   - Update progress_percent
   - Update updated_at timestamp

2. **On block complete:**
   - Aggregate WARMTH.md from executor's lessons_learned
   - Verify SUMMARY.md was written
   - Update STATE.md

3. **On checkpoint:**
   - Write CHECKPOINT.md with current thread state
   - Set STATE.md status to "checkpoint"
   - Wait for user response

4. **On session approaching exhaustion:**
   - Write CHECKPOINT.md with type: session_death
   - Set STATE.md status to "interrupted"
   - Include partial_work details

### Grid Executor (grid-executor.md)

Executors must:

1. **On thread complete:**
   - Commit work with clear message
   - Record commit hash

2. **On block complete:**
   - Write SUMMARY.md to `.grid/phases/{phase}/`
   - Include all commits with hashes
   - Include lessons_learned for warmth aggregation
   - List all artifacts_created

3. **On scratchpad entry:**
   - Write discovery to SCRATCHPAD.md
   - Include timestamp and program-id
   - Follow standard format

4. **On blocker encountered:**
   - Write to BLOCKERS.md
   - Include type, description, position
   - Mark as ACTIVE

### Resume Command (resume.md)

The `/grid:resume` command must:

1. **Detect state:**
   - Check if STATE.md exists
   - Parse status and position
   - Determine resume strategy

2. **Validate state:**
   - Verify commits exist in git
   - Verify claimed files exist
   - Check for conflicts

3. **Reconstruct context:**
   - Load STATE.md
   - Load WARMTH.md
   - Load DECISIONS.md
   - Load CHECKPOINT.md if exists
   - Collect all SUMMARY.md files
   - Build execution context

4. **Spawn continuation:**
   - Inject warmth into executor prompt
   - Provide completed_threads table
   - Provide pending plan
   - Set resume_point

## State Update Protocol

### Atomic Updates

Always use atomic writes to prevent corruption:

```python
# 1. Read current state
current = parse_yaml(read(".grid/STATE.md"))

# 2. Apply updates
merged = deep_merge(current, updates)
merged["updated_at"] = datetime.now().isoformat()

# 3. Write to temp file
write(".grid/STATE.md.tmp", to_yaml(merged))

# 4. Atomic rename
rename(".grid/STATE.md.tmp", ".grid/STATE.md")
```

### Update Triggers

| Event | File | Field Updated |
|-------|------|---------------|
| Wave starts | STATE.md | status: active, wave: N |
| Wave completes | STATE.md | wave: N+1, progress_percent |
| Block completes | STATE.md | block: N+1 |
| Checkpoint hit | STATE.md, CHECKPOINT.md | status: checkpoint |
| Session ending | STATE.md, CHECKPOINT.md | status: interrupted |
| Mission complete | STATE.md | status: completed |
| Blocker found | BLOCKERS.md | New entry |
| User decision | DECISIONS.md | New entry |
| Discovery made | SCRATCHPAD.md | New entry |

## Warmth Aggregation

After each block completes, aggregate warmth:

```python
def aggregate_warmth(block_summary_path):
    # 1. Parse block summary
    summary = parse_yaml(read(block_summary_path))
    block_warmth = summary.get("lessons_learned", {})

    # 2. Load existing warmth
    if file_exists(".grid/WARMTH.md"):
        existing = parse_yaml(read(".grid/WARMTH.md"))
    else:
        existing = {
            "codebase_patterns": [],
            "gotchas": [],
            "user_preferences": [],
            "decisions_made": [],
            "almost_did": [],
        }

    # 3. Merge (deduplicate)
    for category in ["codebase_patterns", "gotchas", "user_preferences", "almost_did"]:
        new_items = block_warmth.get(category, [])
        for item in new_items:
            if item not in existing[category]:
                existing[category].append(item)

    # 4. Write aggregated warmth
    write(".grid/WARMTH.md", to_yaml(existing))
```

## Context Reconstruction

When resuming, rebuild complete context:

```python
def reconstruct_context():
    context = {
        "cluster": None,
        "position": None,
        "completed_blocks": [],
        "completed_threads": [],
        "pending_plans": [],
        "warmth": None,
        "decisions": [],
        "blockers": [],
        "checkpoint": None,
    }

    # 1. Load central state
    state = parse_yaml(read(".grid/STATE.md"))
    context["cluster"] = state["cluster"]
    context["position"] = state["position"]

    # 2. Collect completed work
    for summary_path in glob(".grid/phases/*/SUMMARY.md"):
        summary = parse_yaml(read(summary_path))
        context["completed_blocks"].append(summary)

    # 3. Load warmth
    if file_exists(".grid/WARMTH.md"):
        context["warmth"] = read(".grid/WARMTH.md")

    # 4. Load decisions
    if file_exists(".grid/DECISIONS.md"):
        context["decisions"] = parse_decisions(".grid/DECISIONS.md")

    # 5. Load checkpoint if exists
    if file_exists(".grid/CHECKPOINT.md"):
        context["checkpoint"] = parse_yaml(read(".grid/CHECKPOINT.md"))

    # 6. Identify pending plans
    for plan_path in glob(".grid/plans/*-block-*.md"):
        block_num = extract_block_number(plan_path)
        if block_num not in [b["block"] for b in context["completed_blocks"]]:
            context["pending_plans"].append({
                "path": plan_path,
                "block": block_num,
                "content": read(plan_path),
            })

    return context
```

## Checkpoint Types

### Human Verify Checkpoint

```yaml
type: human_verify
checkpoint_details:
  verification_instructions: |
    1. Run: npm run dev
    2. Visit: http://localhost:4321
    3. Click dark mode toggle
    4. Verify theme persists on refresh
awaiting: "User to respond 'approved' or describe issues"
```

### Decision Checkpoint

```yaml
type: decision
checkpoint_details:
  question: "Deploy to Vercel or Netlify?"
  options:
    - id: vercel
      description: "Native Astro support, edge functions"
    - id: netlify
      description: "Simpler config, build plugins"
awaiting: "User to choose option"
```

### Human Action Checkpoint

```yaml
type: human_action
checkpoint_details:
  required_action: "Run: vercel login"
  reason: "Vercel CLI authentication needed"
awaiting: "User to complete action and confirm"
```

### Session Death Checkpoint

```yaml
type: session_death
checkpoint_details:
  last_action: "Writing localStorage persistence logic"
  partial_work:
    files_created: ["src/components/DarkModeToggle.astro"]
    files_modified: ["src/layouts/BaseLayout.astro"]
    staged_changes: true
awaiting: "Automatic resume on next session"
```

## Testing Persistence

### Test 1: Clean Checkpoint Resume

1. Start mission: `/grid`
2. Wait for checkpoint
3. Approve checkpoint
4. Close terminal (simulate session death)
5. New session: `/grid:resume`
6. **Expected:** Continues from approved checkpoint

### Test 2: Session Death Recovery

1. Start mission: `/grid`
2. During execution, kill terminal (SIGKILL)
3. New session: `/grid:resume`
4. **Expected:** Detects stale state, reconstructs from scratchpad + git

### Test 3: Failure Recovery

1. Start mission that will fail (e.g., missing API key)
2. Executor returns failure
3. New session: `/grid:resume`
4. **Expected:** Presents failure report with recovery options

## Next Steps

1. **Implement STATE.md updates in mc.md**
   - Add wave complete handler
   - Add block complete handler
   - Add checkpoint handler

2. **Implement SUMMARY.md writes in grid-executor.md**
   - Add block complete handler
   - Include lessons_learned section

3. **Complete /grid:resume implementation**
   - Add state validation
   - Add context reconstruction
   - Add continuation spawning

4. **Test end-to-end persistence**
   - Run full mission with interruption
   - Verify resume works correctly

End of Line.