# Troubleshooting Analysis for Grid Integration

## Executive Summary

Claude Code's troubleshooting documentation reveals a mature diagnostic ecosystem that The Grid currently underutilizes. The most impactful opportunities are: (1) implementing a Grid-specific `/grid:doctor` command modeled on Claude's `/doctor`, (2) establishing structured configuration reset procedures for common Grid failures, (3) creating self-healing patterns for agent spawn failures and context overflows, and (4) providing user-friendly error messages with actionable recovery steps. These improvements could reduce user-reported issues by 50%+ and enable The Grid to recover automatically from most transient failures.

---

## Mapping Grid Issues to Claude Code Troubleshooting

### 1. Agent Spawn Failures

**Claude Code Analog**: "Node not found errors", "Command not found" issues

**Grid Manifestation**:
- Task() calls fail silently or with cryptic errors
- Agent frontmatter not parsed correctly
- Missing agent files in `~/.claude/agents/`
- Permission issues preventing agent access

**Claude's Approach**:
- Check installation paths (`which npm`, `which node`)
- Verify environment configuration
- Provide specific remediation commands

**Grid Opportunity**:
```bash
# /grid:doctor could check:
ls ~/.claude/agents/grid-*.md          # Agent files exist
head -5 ~/.claude/agents/grid-executor.md  # YAML frontmatter valid
grep -l "name:" ~/.claude/agents/grid-*.md # All agents have name field
```

### 2. Context Overflows

**Claude Code Analog**: "High CPU or Memory Usage" section

**Grid Manifestation**:
- MC loses track of mission state
- Warmth transfer fails mid-conversation
- Agent responses truncated
- Autocompact corrupts critical context

**Claude's Approach**:
- Use `/compact` regularly
- Close and restart between major tasks
- Manage large directories via `.gitignore`

**Grid Opportunity**:
- Proactive context monitoring before hitting limits
- Automatic `/compact` trigger when approaching threshold
- Structured warmth checkpoints immune to compaction

### 3. Permission Issues

**Claude Code Analog**: "Repeated Permission Prompts", Authentication Issues

**Grid Manifestation**:
- Executor blocked from writing files
- Bash commands require repeated approval
- MCP tools unavailable to agents

**Claude's Approach**:
- Use `/permissions` to allow specific tools
- Clear and recreate auth state if corrupted

**Grid Opportunity**:
- Pre-flight permission check before spawning agents
- Recommend permission settings based on agent type
- Automatic permission mode suggestions

### 4. State Corruption

**Claude Code Analog**: "Resetting Configuration" section

**Grid Manifestation**:
- `.grid/STATE.md` becomes inconsistent
- Phase files reference non-existent blocks
- `config.json` contains invalid values
- Debug sessions reference deleted files

**Claude's Approach**:
```bash
# Reset all user settings and state
rm ~/.claude.json
rm -rf ~/.claude/

# Reset project-specific settings
rm -rf .claude/
rm .mcp.json
```

**Grid Opportunity**:
```bash
# /grid:reset (proposed)
# Levels of reset:
/grid:reset --soft     # Clear current session, keep config
/grid:reset --hard     # Clear all state, keep agents
/grid:reset --factory  # Nuclear option: reinstall everything
```

### 5. Budget Overruns

**Claude Code Analog**: Not directly addressed (Grid-specific)

**Grid Manifestation**:
- Agents exceed allocated tokens
- Opus used when Haiku would suffice
- Recursive spawning creates runaway costs

**Grid Opportunity**:
- Real-time budget monitoring via `/grid:budget`
- Circuit breaker pattern: auto-downgrade model when budget low
- Hard limits per agent type

---

## Diagnostic Tools for Grid

### 1. `/grid:doctor` Command

Modeled on Claude's `/doctor` command, this would check:

```
/grid:doctor
═══════════════════════════════════════════════════════════════════

GRID DIAGNOSTIC REPORT
──────────────────────

INSTALLATION
  ✓ Grid commands installed in ~/.claude/commands/grid/
  ✓ Grid agents installed in ~/.claude/agents/
  ✓ VERSION file present (1.7.x)
  ✗ Missing: grid-synth.md agent file
    → Run: cp /path/to/grid/agents/grid-synth.md ~/.claude/agents/

CONFIGURATION
  ✓ .grid/config.json exists and valid JSON
  ✓ Model setting: sonnet (valid)
  ✓ Budget setting: 1000000 tokens
  ⚠ Warning: No MCP servers configured for research agents

STATE
  ✓ .grid/STATE.md exists
  ✓ Current cluster: "API Refactor"
  ✗ State inconsistency: Block 2 marked complete but SUMMARY.md missing
    → Fix: Run /grid:resume to reconcile state

AGENTS (17 total)
  ✓ All agent files have valid YAML frontmatter
  ✓ All agents have name field
  ⚠ Warning: 5 agents missing 'model' field (using defaults)

PERMISSIONS
  ✓ Edit tool: allowed (with approval)
  ✓ Bash tool: allowed (with approval)
  ⚠ Recommendation: Add permissionMode to agent frontmatter for efficiency

CONTEXT
  Current usage: 45,000 / 200,000 tokens (22%)
  ✓ Context healthy
  Recommendation: /compact when > 70%

══════════════════════════════════════════════════════════════════
SUMMARY: 2 errors, 3 warnings
Run `/grid:doctor --fix` to auto-repair issues
══════════════════════════════════════════════════════════════════
```

### 2. Enhanced `/grid:status` with Health Indicators

Current `/grid:status` shows progress. Enhanced version adds health:

```
THE GRID - STATUS
══════════════════════════════════════════════════════════════════

SYSTEM HEALTH
─────────────
  Context:  [████░░░░░░] 42%     ✓ Healthy
  Budget:   [███████░░░] 68%     ⚠ Monitor
  State:    ✓ Consistent
  Agents:   ✓ All responsive

CLUSTER: API Refactor
...
```

### 3. Structured Error Logging

Grid could maintain an error log for post-mortem analysis:

```
.grid/
└── logs/
    ├── errors.jsonl           # Structured error log
    ├── spawns.jsonl           # Agent spawn history
    └── recoveries.jsonl       # Automatic recovery events
```

**Error log entry format**:
```json
{
  "timestamp": "2024-01-23T14:30:00Z",
  "error_type": "agent_spawn_failure",
  "agent": "grid-executor",
  "context": {
    "task_id": "02-03",
    "cluster": "API Refactor",
    "model": "sonnet"
  },
  "message": "Task() returned empty response",
  "recovery_attempted": true,
  "recovery_result": "success",
  "recovery_action": "retry_with_lower_model"
}
```

---

## Error Recovery Patterns

### 1. Agent Spawn Retry with Exponential Backoff

```python
# Pattern for MC to implement
def spawn_agent_with_retry(agent_type, prompt, max_retries=3):
    for attempt in range(max_retries):
        try:
            result = Task(
                prompt=prompt,
                subagent_type=agent_type,
                model=get_model_for_attempt(attempt)  # Downgrade on retry
            )
            if result and not is_error(result):
                return result
        except Exception as e:
            log_error(e, attempt)

        # Exponential backoff
        time.sleep(2 ** attempt)

    # All retries failed
    return graceful_degradation(prompt)
```

### 2. Context Overflow Prevention

```python
# Proactive context management
def check_context_before_spawn():
    usage = get_context_usage()

    if usage > 0.90:  # > 90%
        # Critical: Cannot spawn safely
        trigger_checkpoint()
        run_compact()
        return "compacted"

    elif usage > 0.70:  # > 70%
        # Warning: Spawn but prepare for compact
        log_warning("Context at {usage}%, compact soon")
        return "warn"

    else:
        return "healthy"
```

### 3. State Reconciliation

When STATE.md and phase files disagree:

```python
def reconcile_state():
    """Reconcile .grid/STATE.md with actual phase files."""

    state = parse_state_file()
    actual = scan_phase_directory()

    discrepancies = []

    for block_id, block_state in state['blocks'].items():
        actual_block = actual.get(block_id)

        if block_state['status'] == 'complete':
            if not actual_block or not actual_block.get('summary_exists'):
                discrepancies.append({
                    'type': 'missing_summary',
                    'block': block_id,
                    'action': 'mark_incomplete'
                })

        if block_state['status'] == 'pending':
            if actual_block and actual_block.get('summary_exists'):
                discrepancies.append({
                    'type': 'unexpected_complete',
                    'block': block_id,
                    'action': 'mark_complete'
                })

    if discrepancies:
        apply_reconciliation(discrepancies)
        log_reconciliation(discrepancies)
```

### 4. Permission Recovery

```python
def ensure_permissions_for_agent(agent_type):
    """Verify agent has required permissions before spawn."""

    required = get_required_permissions(agent_type)
    current = get_current_permissions()

    missing = required - current

    if missing:
        if agent_type in TRUSTED_AGENTS:
            # Suggest but don't block
            suggest_permission_additions(missing)
            return "proceed_with_prompts"
        else:
            # Block spawn until fixed
            raise PermissionError(
                f"Agent {agent_type} requires permissions: {missing}\n"
                f"Run: /permissions to add them"
            )
```

---

## Self-Healing Strategies

### 1. Automatic Agent File Recovery

If agent file is missing or corrupted:

```python
def heal_agent_file(agent_name):
    """Attempt to recover missing/corrupted agent file."""

    source_locations = [
        f"~/.claude/agents/{agent_name}.md.backup",
        f"/path/to/grid/agents/{agent_name}.md",
        # Fetch from npm package if installed
        get_npm_agent_path(agent_name)
    ]

    for source in source_locations:
        if file_exists(source) and is_valid_agent(source):
            copy_to_agents_dir(source, agent_name)
            log_recovery(f"Restored {agent_name} from {source}")
            return True

    return False
```

### 2. Session Auto-Checkpoint

Before risky operations, auto-save state:

```python
def risky_operation_wrapper(operation, *args):
    """Wrap risky operations with automatic checkpointing."""

    # Save state before operation
    checkpoint_id = save_checkpoint()

    try:
        result = operation(*args)
        return result
    except Exception as e:
        # Restore from checkpoint
        restore_checkpoint(checkpoint_id)
        log_error(f"Operation failed, restored to checkpoint {checkpoint_id}")
        raise
    finally:
        # Keep checkpoint for 1 hour, then auto-delete
        schedule_checkpoint_cleanup(checkpoint_id, hours=1)
```

### 3. Budget Circuit Breaker

```python
class BudgetCircuitBreaker:
    """Prevent runaway costs with automatic model downgrade."""

    def __init__(self, budget_limit):
        self.budget_limit = budget_limit
        self.used = 0
        self.state = "closed"  # closed = normal operation

    def check(self, estimated_cost):
        if self.state == "open":
            return "blocked"

        if self.used + estimated_cost > self.budget_limit * 0.9:
            self.state = "half-open"  # Trigger downgrade
            return "downgrade"

        return "proceed"

    def record(self, actual_cost):
        self.used += actual_cost

        if self.used > self.budget_limit:
            self.state = "open"  # Block further operations
            notify_user("Budget exceeded, Grid paused")
```

### 4. Context Compaction with Warmth Preservation

```python
def smart_compact():
    """Compact context while preserving Grid-critical state."""

    # Extract warmth before compact
    warmth = extract_warmth_payload()
    state_summary = summarize_grid_state()

    # Run compact
    run_compact()

    # Re-inject critical context
    inject_post_compact(f"""
## Grid Context Restoration

### Current State
{state_summary}

### Warmth Transfer
{warmth}

### Resume Instructions
Continue from where we left off. The following was preserved:
- Current cluster and position
- Active agent assignments
- Pending checkpoints
    """)
```

---

## User Guidance Improvements

### 1. Actionable Error Messages

**Current** (generic):
```
Error: Agent spawn failed
```

**Improved** (actionable):
```
═══════════════════════════════════════════════════════════════════
GRID ERROR: Agent Spawn Failure
═══════════════════════════════════════════════════════════════════

Agent: grid-executor
Task: Thread 02-03 (Implement API endpoint)

WHAT HAPPENED:
  The executor agent could not be spawned. This usually means:
  - Agent file is missing or corrupted
  - Insufficient context space for agent
  - Permission configuration issue

AUTOMATIC RECOVERY ATTEMPTED:
  ✗ Retry #1: Same error
  ✓ Retry #2: Downgraded to sonnet
  ✗ Retry #3: Context overflow detected

RECOMMENDED ACTIONS:
  1. Run /grid:doctor to diagnose
  2. Run /compact to free context
  3. Try /grid:resume to restart from last checkpoint

FULL ERROR LOG:
  .grid/logs/errors.jsonl (line 47)

═══════════════════════════════════════════════════════════════════
```

### 2. Interactive Recovery Wizard

```
═══════════════════════════════════════════════════════════════════
GRID RECOVERY WIZARD
═══════════════════════════════════════════════════════════════════

Issue detected: State file corruption

I found inconsistencies in .grid/STATE.md that need resolution.

OPTIONS:
  [1] Auto-repair (recommended)
      → Reconcile STATE.md with actual phase files
      → Keep completed work, mark inconsistent items for re-execution

  [2] Reset to last checkpoint
      → Restore state from 2 hours ago
      → Some work may need to be re-done

  [3] Manual intervention
      → Show me the discrepancies so I can fix them

  [4] Nuclear reset
      → Clear all state and start fresh
      → Preserves config.json settings

Enter choice (1-4):
═══════════════════════════════════════════════════════════════════
```

### 3. Contextual Help Integration

When errors occur, link to relevant documentation:

```python
ERROR_HELP_MAPPING = {
    "agent_spawn_failure": {
        "doc": "docs/troubleshooting/agent-spawn.md",
        "quick_tip": "Run /grid:doctor to diagnose agent issues",
        "community": "https://github.com/JamesWeatherhead/grid/issues?q=spawn"
    },
    "context_overflow": {
        "doc": "docs/troubleshooting/context.md",
        "quick_tip": "Run /compact before large operations",
        "community": "https://github.com/JamesWeatherhead/grid/issues?q=context"
    },
    "permission_denied": {
        "doc": "docs/troubleshooting/permissions.md",
        "quick_tip": "Use /permissions to configure tool access",
        "community": "https://github.com/JamesWeatherhead/grid/issues?q=permission"
    }
}
```

---

## Quick Wins (< 1 hour each)

### 1. Create `/grid:doctor` command (45 min)

Basic diagnostic command that checks:
- Agent files exist
- State file valid
- Config file parseable
- Context usage level

```yaml
# ~/.claude/commands/grid/doctor.md
---
name: grid:doctor
description: Diagnose Grid installation and state
allowed-tools:
  - Read
  - Glob
  - Grep
  - Bash
---
```

### 2. Add health indicators to `/grid:status` (20 min)

Extend current status to show:
- Context usage percentage
- Budget remaining
- Agent file health

### 3. Improve error messages with recovery hints (30 min)

Wrap common error paths with structured error objects:

```python
class GridError:
    def __init__(self, error_type, message, recovery_hints):
        self.error_type = error_type
        self.message = message
        self.recovery_hints = recovery_hints

    def display(self):
        return f"""
GRID ERROR: {self.error_type}
─────────────────────────────
{self.message}

TO FIX:
{chr(10).join(f'  {i+1}. {hint}' for i, hint in enumerate(self.recovery_hints))}
"""
```

### 4. Add configuration reset levels (15 min)

Document and implement reset options:

```bash
# Soft reset (current session only)
rm .grid/STATE.md

# Medium reset (all state, keep config)
rm -rf .grid/phases/ .grid/debug/ .grid/STATE.md

# Hard reset (everything except agents)
rm -rf .grid/

# Factory reset
rm -rf .grid/ ~/.claude/agents/grid-*.md ~/.claude/commands/grid/
```

### 5. Create error log infrastructure (30 min)

Simple JSONL logging for errors:

```python
def log_grid_error(error):
    log_path = ".grid/logs/errors.jsonl"
    os.makedirs(os.path.dirname(log_path), exist_ok=True)

    with open(log_path, "a") as f:
        f.write(json.dumps({
            "timestamp": datetime.utcnow().isoformat(),
            "error_type": error.error_type,
            "message": error.message,
            "context": error.context
        }) + "\n")
```

---

## Architecture Changes

### 1. Centralized Error Handling Layer

Instead of handling errors ad-hoc in each command, create a central error handler:

```
~/.claude/commands/grid/
├── _error_handler.md      # Shared error handling logic
├── _recovery_strategies.md # Recovery pattern library
└── ...other commands
```

MC would import these for consistent error handling across all operations.

### 2. Health Monitor Background Process

Optional daemon mode for continuous health monitoring:

```python
# /grid:daemon --health-monitor
while True:
    health = check_grid_health()

    if health.context_usage > 0.80:
        notify("Context at 80%, consider /compact")

    if health.budget_remaining < 0.20:
        notify("Budget at 20%, consider /grid:budget --add")

    if health.state_inconsistent:
        notify("State inconsistency detected, run /grid:doctor")

    sleep(300)  # Check every 5 minutes
```

### 3. Checkpoint System

Formalize checkpoint/restore for critical operations:

```
.grid/
└── checkpoints/
    ├── auto-20240123-143000.json   # Auto-checkpoint before risky op
    ├── manual-pre-refactor.json    # User-created checkpoint
    └── checkpoint-manifest.json    # Index of all checkpoints
```

### 4. Agent Health Registry

Track agent spawn success/failure rates:

```json
// .grid/agent-health.json
{
  "grid-executor": {
    "total_spawns": 47,
    "successful": 45,
    "failed": 2,
    "avg_tokens": 15000,
    "last_failure": "2024-01-22T10:30:00Z",
    "failure_reasons": ["context_overflow", "permission_denied"]
  }
}
```

This enables:
- Proactive warnings for problematic agents
- Automatic model downgrade for frequently-failing agents
- Usage analytics for optimization

---

## Specific Recommendations

### Immediate (This Week)

1. **Create `/grid:doctor` command** - Basic diagnostic that checks agent files, state validity, and context usage. Output in the format shown above.

2. **Add structured error messages** - Wrap all error paths with GridError class that includes error type, message, and recovery hints.

3. **Document reset procedures** - Add to `/grid:help` the different reset levels and when to use each.

4. **Add context monitoring to `/grid:status`** - Show context usage percentage with color-coded health indicator.

### Short-term (This Month)

5. **Implement error logging** - Create `.grid/logs/` structure and log all errors as JSONL for post-mortem analysis.

6. **Add agent spawn retry logic** - Implement exponential backoff with model downgrade on retry.

7. **Create checkpoint infrastructure** - Auto-checkpoint before risky operations (large spawns, state modifications).

8. **Build recovery wizard** - Interactive menu for common failure scenarios with guided recovery.

### Medium-term (Next Quarter)

9. **Implement self-healing patterns** - Agent file recovery, state reconciliation, budget circuit breaker.

10. **Add health monitor daemon** - Background process for proactive health monitoring and alerts.

11. **Create `/grid:bug` command** - Similar to Claude's `/bug`, automatically gather diagnostics and format for GitHub issue.

12. **Build analytics dashboard** - Track agent success rates, token usage, and failure patterns over time.

---

## Appendix: Error Code Registry

Standardized error codes for Grid issues:

| Code | Category | Description |
|------|----------|-------------|
| G001 | Agent | Agent file not found |
| G002 | Agent | Agent frontmatter invalid |
| G003 | Agent | Agent spawn timeout |
| G004 | Agent | Agent response empty |
| G010 | Context | Context overflow imminent |
| G011 | Context | Context overflow occurred |
| G012 | Context | Warmth transfer failed |
| G020 | State | STATE.md not found |
| G021 | State | STATE.md parse error |
| G022 | State | State/phase inconsistency |
| G030 | Permission | Tool permission denied |
| G031 | Permission | File write blocked |
| G040 | Budget | Budget limit approaching |
| G041 | Budget | Budget limit exceeded |
| G050 | Config | config.json invalid |
| G051 | Config | Unknown model specified |

---

## Conclusion

Claude Code's troubleshooting infrastructure provides an excellent template for Grid error handling. The key insights are:

1. **Diagnostics First** - The `/doctor` command pattern gives users confidence and self-service capability
2. **Actionable Errors** - Every error should include specific recovery steps
3. **Reset Levels** - Multiple reset options from soft to factory give users appropriate tools
4. **Self-Healing** - Automatic recovery attempts before surfacing errors to users

The immediate wins are:
- `/grid:doctor` for diagnostics
- Structured error messages with recovery hints
- Context monitoring in `/grid:status`

The deeper opportunities involve checkpoint infrastructure, agent health tracking, and self-healing patterns. These require more significant architecture but would make The Grid substantially more robust and user-friendly.

End of Line.