# Grid Blackboard System Specification

**Version:** 1.7.x
**Status:** Active
**Last Updated:** 2026-01-24

---

## Overview

The Blackboard is The Grid's shared memory system - a GPU-like architecture enabling parallel agents to communicate, coordinate, and persist knowledge across execution boundaries. Like a GPU's shared memory hierarchy, the Blackboard provides different memory tiers optimized for different access patterns and lifetimes.

```
┌─────────────────────────────────────────────────────────────────────┐
│                        THE BLACKBOARD                                │
├─────────────────────────────────────────────────────────────────────┤
│                                                                      │
│   L4: LEARNINGS.md         ← Persistent across missions (ROM-like)  │
│   ─────────────────────────────────────────────────────────────────  │
│   L3: STATE.md             ← Global mission state (shared registers) │
│   ─────────────────────────────────────────────────────────────────  │
│   L2: Phase Scratchpads    ← Per-phase communication (shared mem)   │
│   ─────────────────────────────────────────────────────────────────  │
│   L1: Agent Context        ← Private working memory (registers)     │
│                                                                      │
└─────────────────────────────────────────────────────────────────────┘
```

---

## 1. STATE.md Schema (L3 Global State)

The global state file serves as the mission's source of truth - the "shared registers" that all agents can read and MC can write.

### Location

```
.grid/STATE.md
```

### Schema

```yaml
---
# ═══════════════════════════════════════════════════════════════════
# GRID STATE - Machine Parseable Header
# ═══════════════════════════════════════════════════════════════════

# Metadata
version: "1.0"
session_id: "sess-{timestamp}-{random_6char}"
mission_id: "mission-{timestamp}-{slug}"
cluster: "{user_provided_name_or_inferred}"
created_at: "{ISO8601_timestamp}"
updated_at: "{ISO8601_timestamp}"

# Mission Status
status: "planning" | "in_progress" | "blocked" | "complete" | "failed"
mode: "autopilot" | "guided" | "hands_on"

# Phase Tracking
phases:
  total: {N}
  current: {N}
  completed: []  # List of completed phase IDs

  definitions:
    - id: "01-foundation"
      name: "Foundation"
      status: "complete" | "in_progress" | "pending"
      blocks: 3
      blocks_complete: 2
      started_at: "{ISO8601}"
      completed_at: "{ISO8601}" | null

    - id: "02-core"
      name: "Core Features"
      status: "pending"
      blocks: 4
      blocks_complete: 0
      started_at: null
      completed_at: null

# Position (Current Location in Execution)
position:
  phase: 1
  phase_id: "01-foundation"
  phase_name: "Foundation"
  block: 2
  block_id: "01-02"
  block_name: "Authentication"
  wave: 1
  thread: 3
  thread_total: 5

# Progress Metrics
progress:
  blocks_complete: 4
  blocks_total: 12
  threads_complete: 23
  threads_total: 48
  percent: 47.9

# Active Agents Registry
agents:
  active: []  # Currently executing agents
  # Example when running:
  # active:
  #   - id: "executor-001"
  #     type: "executor"
  #     spawned_at: "{ISO8601}"
  #     task: "Implementing auth middleware"
  #     phase: "01-foundation"
  #     block: "01-02"
  #     last_heartbeat: "{ISO8601}"

  completed: []  # Agents that finished this session
  # Example:
  # completed:
  #   - id: "planner-001"
  #     type: "planner"
  #     result: "success"
  #     duration_seconds: 45

# Cross-Phase Persistent Data
persistent:
  # Decisions that affect multiple phases
  decisions:
    - id: "dec-001"
      decision: "Using JWT with refresh rotation"
      phase_decided: "01-foundation"
      affects_phases: ["01-foundation", "02-core", "03-api"]
      rationale: "Stateless auth for horizontal scaling"

  # Shared artifacts
  artifacts:
    - id: "art-001"
      type: "schema"
      path: "prisma/schema.prisma"
      created_phase: "01-foundation"
      used_by_phases: ["01-foundation", "02-core"]

  # Tech stack (accumulated)
  tech_stack:
    runtime: "node-20"
    framework: "next-14"
    database: "postgresql"
    orm: "prisma-5"
    auth: "jose-jwt"
    added_packages:
      - name: "jose"
        added_phase: "01-foundation"
        purpose: "JWT handling"

# Resume Information
resume:
  checkpoint_file: ".grid/CHECKPOINT.md" | null
  last_agent: "executor-001" | null
  last_commit: "{git_short_hash}" | null
  can_resume: true | false
  resume_position:
    phase: 1
    block: 2
    thread: 3

# Budget Tracking (Summary)
budget:
  total_cost_usd: 2.45
  session_cost_usd: 0.89
  spawns_this_session: 12

# Warnings/Blockers
warnings: []
# Example:
# warnings:
#   - level: "warning"
#     message: "API rate limit approaching"
#     source: "executor-002"
#     timestamp: "{ISO8601}"

blockers: []
# Example:
# blockers:
#   - id: "blocker-001"
#     type: "missing_credential"
#     description: "Stripe API key not configured"
#     blocking_thread: "01-03-thread-2"
#     added_at: "{ISO8601}"

# Energy (Tron aesthetic, optional display)
energy: 9000
---

# THE GRID - Mission State

## Mission: {cluster}

**Status:** {status}
**Mode:** {mode}
**Progress:** [{progress_bar}] {percent}%

## Current Position

Phase {position.phase}/{phases.total}: {position.phase_name}
Block {position.block}: {position.block_name}
Thread {position.thread}/{position.thread_total}

## Recent Activity

| Time | Agent | Action | Result |
|------|-------|--------|--------|
| {time} | {agent} | {action} | {result} |

## Active Agents

{agent_list_or_none}

## Session Summary

- Started: {created_at}
- Last Update: {updated_at}
- Commits: {commit_count}
- Files Changed: {file_count}

---

End of Line.
```

### Access Rules

| Agent Type | Read | Write |
|------------|------|-------|
| MC | YES | YES (full) |
| Executor | YES | YES (position, agents, warnings only) |
| Planner | YES | YES (phases, position only) |
| Recognizer | YES | YES (warnings, blockers only) |
| Scout | YES | NO |
| Memory | YES | YES (persistent section only) |

### Update Protocol

```python
def update_state(updates: dict, agent_id: str, agent_type: str):
    """
    Update STATE.md with proper locking and validation.

    Args:
        updates: Dict of fields to update
        agent_id: ID of updating agent
        agent_type: Type for permission checking
    """
    # 1. Acquire advisory lock (file-based)
    lock_file = ".grid/.state.lock"
    acquire_lock(lock_file, timeout=5)

    try:
        # 2. Read current state
        state = read_yaml_frontmatter(".grid/STATE.md")

        # 3. Validate permissions
        allowed_fields = get_allowed_fields(agent_type)
        for field in updates.keys():
            if field not in allowed_fields:
                raise PermissionError(f"{agent_type} cannot update {field}")

        # 4. Apply updates
        state = deep_merge(state, updates)
        state["updated_at"] = now_iso8601()

        # 5. Emit write notification
        emit_blackboard_event("state_updated", {
            "agent": agent_id,
            "fields": list(updates.keys()),
            "timestamp": state["updated_at"]
        })

        # 6. Write back
        write_yaml_frontmatter(".grid/STATE.md", state)

    finally:
        release_lock(lock_file)
```

---

## 2. Per-Phase Scratchpad (L2 Shared Memory)

Phase scratchpads enable fast inter-agent communication within a phase. They're the "shared memory" that agents in the same compute unit can access.

### Location

```
.grid/phases/{phase_id}/scratchpad.md
```

Example:
```
.grid/phases/01-foundation/scratchpad.md
.grid/phases/02-core/scratchpad.md
```

### Schema

```yaml
---
# ═══════════════════════════════════════════════════════════════════
# PHASE SCRATCHPAD - Inter-Agent Communication
# ═══════════════════════════════════════════════════════════════════

phase_id: "01-foundation"
phase_name: "Foundation"
created_at: "{ISO8601}"
last_updated: "{ISO8601}"

# Index for Fast Lookup (updated on each write)
index:
  entry_count: 15
  max_entries: 50  # Archive threshold

  by_topic:
    auth: [1, 5, 12]
    database: [2, 3, 8]
    api: [4, 6, 7, 9]

  by_agent:
    executor-001: [1, 2, 3]
    executor-002: [4, 5, 6]
    scout-001: [7, 8]

  by_category:
    discovery: [1, 4, 7]
    decision: [2, 5]
    gotcha: [3, 6]
    pattern: [8, 9]
    warning: [10]
    question: [11]
    blocker: [12]
    handoff: [13, 14, 15]

  by_relevance:
    HIGH: [1, 5, 9, 12]
    MEDIUM: [2, 3, 4, 6, 7, 8]
    LOW: [10, 11, 13, 14, 15]

  recent: [15, 14, 13, 12, 11, 10, 9, 8, 7, 6]  # Most recent first

# Subscription Registry (for write notifications)
subscriptions:
  - agent_id: "executor-002"
    topics: ["auth", "database"]
    relevance_min: "MEDIUM"

  - agent_id: "memory-001"
    topics: ["*"]  # All topics
    relevance_min: "HIGH"

# Archive Info
archive:
  archived_count: 25
  last_archive: "{ISO8601}"
  archive_file: ".grid/phases/01-foundation/scratchpad_archive.md"
---

# Phase Scratchpad: {phase_name}

## Entries

### [1] [2026-01-24T14:30:00Z] executor-001 | discovery | auth

**Topic:** Authentication
**Tags:** auth, middleware, prisma, jwt
**Relevance:** HIGH
**Block:** 01-02

Found: Auth middleware must run AFTER body parsing middleware.
Impact: All protected routes affected.
Action: Updated middleware order in app.ts.

Evidence:
- File: src/middleware/index.ts
- Line: 23-45
- Commit: abc123

---

### [2] [2026-01-24T14:35:00Z] executor-001 | decision | database

**Topic:** Database Schema
**Tags:** database, prisma, schema
**Relevance:** MEDIUM
**Block:** 01-02

Decision: Using soft deletes (deletedAt timestamp) instead of hard deletes.
Rationale: Audit trail requirements, data recovery capability.
Affects: All entity models in Prisma schema.

---

### [3] [2026-01-24T14:40:00Z] executor-002 | gotcha | api

**Topic:** API Conventions
**Tags:** api, request, body, parsing
**Relevance:** HIGH
**Block:** 01-03

Gotcha: Next.js App Router uses `req.json()` not `req.body` for JSON parsing.
First hit: POST /api/auth/login returned undefined body.
Fix: Changed all route handlers to await req.json().

---

### [4] [2026-01-24T14:45:00Z] executor-001 | handoff | auth

**Topic:** Auth Implementation Handoff
**Tags:** auth, handoff, continuation
**Relevance:** MEDIUM
**Block:** 01-02 -> 01-03

Handoff: JWT implementation complete.

**Completed:**
- Access token generation (15min expiry)
- Refresh token rotation (7day expiry)
- Token validation middleware

**For Next Agent:**
- Refresh endpoint at POST /api/auth/refresh
- Use jose library for verification
- HttpOnly cookies already configured

**Files to Review:**
- src/lib/auth/jwt.ts
- src/middleware/auth.ts

---

### [5] [2026-01-24T14:50:00Z] executor-002 | blocker | external

**Topic:** External Service Blocker
**Tags:** stripe, api, credentials
**Relevance:** HIGH
**Block:** 01-04

Blocker: Stripe API key not configured.
Type: missing_credential
Required: STRIPE_SECRET_KEY environment variable.
Blocking: Payment endpoint implementation.

Action Required: User must add Stripe credentials to .env.local

---
```

### Entry Format

```markdown
### [{entry_id}] [{ISO8601_timestamp}] {agent_id} | {category} | {topic}

**Topic:** {Human readable topic name}
**Tags:** {comma, separated, tags}
**Relevance:** HIGH | MEDIUM | LOW
**Block:** {block_id}

{Entry content - varies by category}

---
```

### Categories

| Category | Purpose | Required Fields |
|----------|---------|-----------------|
| `discovery` | New finding about codebase | Finding, Impact, Evidence |
| `decision` | Choice made during execution | Decision, Rationale, Affects |
| `gotcha` | Trap or pitfall found | Gotcha, First Hit, Fix |
| `pattern` | Recognized codebase pattern | Pattern, Where Found, Usage |
| `warning` | Potential issue for others | Warning, Risk Level, Mitigation |
| `question` | Needs resolution | Question, Context, Urgency |
| `blocker` | Execution blocked | Blocker, Type, Action Required |
| `handoff` | Work transfer to next agent | Completed, For Next Agent, Files |
| `heartbeat` | Progress update | Status, Progress, Current Action |

### Write Protocol

```python
def write_scratchpad_entry(
    phase_id: str,
    agent_id: str,
    category: str,
    topic: str,
    content: str,
    tags: list[str],
    relevance: str,
    block_id: str
):
    """
    Write entry to phase scratchpad with index update and notification.
    """
    scratchpad_path = f".grid/phases/{phase_id}/scratchpad.md"

    # 1. Read current state
    scratchpad = read_file(scratchpad_path)
    frontmatter = parse_yaml_frontmatter(scratchpad)

    # 2. Get next entry ID
    entry_id = frontmatter["index"]["entry_count"] + 1
    timestamp = now_iso8601()

    # 3. Check archive threshold
    if entry_id > frontmatter["index"]["max_entries"]:
        archive_old_entries(scratchpad_path, keep_recent=25)
        entry_id = frontmatter["index"]["entry_count"] + 1

    # 4. Format entry
    entry = format_entry(
        entry_id=entry_id,
        timestamp=timestamp,
        agent_id=agent_id,
        category=category,
        topic=topic,
        content=content,
        tags=tags,
        relevance=relevance,
        block_id=block_id
    )

    # 5. Update index
    frontmatter["index"]["entry_count"] = entry_id
    frontmatter["last_updated"] = timestamp

    # Add to topic index
    if topic not in frontmatter["index"]["by_topic"]:
        frontmatter["index"]["by_topic"][topic] = []
    frontmatter["index"]["by_topic"][topic].append(entry_id)

    # Add to agent index
    if agent_id not in frontmatter["index"]["by_agent"]:
        frontmatter["index"]["by_agent"][agent_id] = []
    frontmatter["index"]["by_agent"][agent_id].append(entry_id)

    # Add to category index
    if category not in frontmatter["index"]["by_category"]:
        frontmatter["index"]["by_category"][category] = []
    frontmatter["index"]["by_category"][category].append(entry_id)

    # Add to relevance index
    frontmatter["index"]["by_relevance"][relevance].append(entry_id)

    # Update recent (keep last 10)
    frontmatter["index"]["recent"].insert(0, entry_id)
    frontmatter["index"]["recent"] = frontmatter["index"]["recent"][:10]

    # 6. Append entry to file
    append_entry(scratchpad_path, entry, frontmatter)

    # 7. Emit write notification
    notify_subscribers(
        phase_id=phase_id,
        entry_id=entry_id,
        topic=topic,
        relevance=relevance,
        agent_id=agent_id,
        subscriptions=frontmatter["subscriptions"]
    )

    return entry_id
```

### Phase Lifecycle

```
Phase Start:
  └─> Create scratchpad.md with empty index

During Phase:
  └─> Agents write entries
  └─> Index updated on each write
  └─> Subscribers notified
  └─> Auto-archive when > 50 entries

Phase Complete:
  └─> Archive remaining entries
  └─> Extract high-value to LEARNINGS.md
  └─> Compress scratchpad to summary
  └─> Move to archive: .grid/phases/{phase_id}/scratchpad_archived.md
```

### Archive Protocol

```python
def archive_phase_scratchpad(phase_id: str):
    """
    Archive scratchpad at phase completion.
    """
    scratchpad_path = f".grid/phases/{phase_id}/scratchpad.md"
    archive_path = f".grid/phases/{phase_id}/scratchpad_archived.md"

    # 1. Read all entries
    scratchpad = read_file(scratchpad_path)
    entries = parse_entries(scratchpad)
    frontmatter = parse_yaml_frontmatter(scratchpad)

    # 2. Extract high-value entries for LEARNINGS.md
    high_value = [e for e in entries if e.relevance == "HIGH"]
    for entry in high_value:
        extract_to_learnings(entry)

    # 3. Create archive with summary
    archive_content = f"""---
phase_id: {phase_id}
archived_at: {now_iso8601()}
total_entries: {len(entries)}
high_value_extracted: {len(high_value)}
---

# Phase {phase_id} Scratchpad Archive

## Summary
- Entries: {len(entries)}
- Discoveries: {count_by_category(entries, 'discovery')}
- Decisions: {count_by_category(entries, 'decision')}
- Gotchas: {count_by_category(entries, 'gotcha')}
- Blockers Resolved: {count_by_category(entries, 'blocker')}

## All Entries

{format_all_entries(entries)}
"""

    write_file(archive_path, archive_content)

    # 4. Clear active scratchpad for potential reuse
    write_file(scratchpad_path, create_empty_scratchpad(phase_id))
```

---

## 3. Write Notification System

The notification system enables real-time coordination between agents without polling.

### Event Types

```yaml
events:
  # State events (L3)
  - type: "state_updated"
    fields: ["position", "status", "agents"]
    agent: "executor-001"

  - type: "phase_changed"
    old_phase: "01-foundation"
    new_phase: "02-core"

  - type: "blocker_added"
    blocker_id: "blocker-001"
    blocking_thread: "01-03-thread-2"

  # Scratchpad events (L2)
  - type: "entry_written"
    phase_id: "01-foundation"
    entry_id: 15
    topic: "auth"
    relevance: "HIGH"
    agent: "executor-002"

  - type: "handoff_posted"
    from_agent: "executor-001"
    to_block: "01-03"
    entry_id: 14

  # Agent events
  - type: "agent_spawned"
    agent_id: "executor-003"
    task: "Implementing search"

  - type: "agent_complete"
    agent_id: "executor-001"
    result: "success"
    duration: 45
```

### Notification Mechanism

```python
# Event file approach (filesystem-based pub/sub)
EVENT_FILE = ".grid/events/current.jsonl"

def emit_blackboard_event(event_type: str, payload: dict):
    """
    Emit event to the blackboard event stream.
    """
    event = {
        "type": event_type,
        "timestamp": now_iso8601(),
        "payload": payload,
        "id": generate_event_id()
    }

    # Append to event file (JSONL format)
    with open(EVENT_FILE, "a") as f:
        f.write(json.dumps(event) + "\n")

    # Also write to latest event file for quick polling
    with open(".grid/events/latest.json", "w") as f:
        json.dump(event, f)


def subscribe_to_events(
    agent_id: str,
    event_types: list[str] = None,
    topics: list[str] = None,
    callback: callable = None
):
    """
    Subscribe agent to blackboard events.

    For long-running agents, poll the event file.
    For quick checks, read latest.json.
    """
    # Record subscription in scratchpad
    if topics:
        phase_id = get_current_phase()
        update_scratchpad_subscriptions(phase_id, agent_id, topics)

    # Return event stream reader
    return EventStreamReader(
        event_file=EVENT_FILE,
        filter_types=event_types,
        filter_topics=topics,
        callback=callback
    )


class EventStreamReader:
    """Read events from blackboard event stream."""

    def __init__(self, event_file, filter_types=None, filter_topics=None, callback=None):
        self.event_file = event_file
        self.filter_types = filter_types
        self.filter_topics = filter_topics
        self.callback = callback
        self.last_position = 0

    def poll(self) -> list[dict]:
        """Poll for new events since last check."""
        events = []

        with open(self.event_file, "r") as f:
            f.seek(self.last_position)
            for line in f:
                event = json.loads(line.strip())

                # Apply filters
                if self.filter_types and event["type"] not in self.filter_types:
                    continue
                if self.filter_topics:
                    event_topic = event.get("payload", {}).get("topic")
                    if event_topic and event_topic not in self.filter_topics:
                        continue

                events.append(event)

                if self.callback:
                    self.callback(event)

            self.last_position = f.tell()

        return events
```

### Conflict Detection

```python
def detect_write_conflict(
    file_path: str,
    agent_id: str,
    operation: str
) -> ConflictResult:
    """
    Detect if multiple agents are modifying the same resource.
    """
    # Check recent events for same file
    recent_events = read_recent_events(minutes=5)

    conflicts = []
    for event in recent_events:
        if event["type"] == "file_modified":
            if event["payload"]["path"] == file_path:
                if event["payload"]["agent"] != agent_id:
                    conflicts.append({
                        "conflicting_agent": event["payload"]["agent"],
                        "operation": event["payload"]["operation"],
                        "timestamp": event["timestamp"]
                    })

    if conflicts:
        return ConflictResult(
            has_conflict=True,
            file_path=file_path,
            conflicts=conflicts,
            resolution="checkpoint"  # Default: create checkpoint for manual resolution
        )

    return ConflictResult(has_conflict=False)


def handle_scratchpad_conflict(
    phase_id: str,
    entry: dict,
    existing_entry_id: int
):
    """
    Handle conflict when two agents write about same topic.

    Strategy: Merge rather than reject (additive, not overwrite).
    """
    # Read existing entry
    existing = get_scratchpad_entry(phase_id, existing_entry_id)

    # If same category and topic within 5 minutes, merge
    if (
        existing["category"] == entry["category"] and
        existing["topic"] == entry["topic"] and
        time_since(existing["timestamp"]) < timedelta(minutes=5)
    ):
        # Create merged entry
        merged = {
            **existing,
            "content": f"{existing['content']}\n\n---\n\n**Update from {entry['agent_id']}:**\n{entry['content']}",
            "tags": list(set(existing["tags"] + entry["tags"])),
            "last_updated": now_iso8601(),
            "contributors": existing.get("contributors", [existing["agent_id"]]) + [entry["agent_id"]]
        }

        update_scratchpad_entry(phase_id, existing_entry_id, merged)

        emit_blackboard_event("entry_merged", {
            "phase_id": phase_id,
            "entry_id": existing_entry_id,
            "merged_from": entry["agent_id"]
        })

        return existing_entry_id

    # Otherwise, create new entry (no conflict)
    return None
```

---

## 4. Memory Hierarchy

### L1: Agent-Local Context (Private)

Each agent has private working memory within its context window. This is NOT persisted to the blackboard.

```yaml
# Conceptual - lives only in agent's context
agent_local:
  agent_id: "executor-001"

  # What agent is working on
  current_task:
    thread_id: "01-02-thread-3"
    started_at: "{ISO8601}"
    files_touched: []

  # Agent's internal state
  working_memory:
    - "Implementing JWT validation"
    - "Using jose library per warmth"
    - "Need to handle token refresh"

  # Recent tool outputs (not persisted)
  recent_outputs:
    - tool: "grep"
      result: "Found 3 usages of deprecated method"

  # Scratch calculations
  scratch:
    estimated_remaining: "15 minutes"
    complexity_assessment: "medium"
```

**Rules for L1:**
- Never persisted
- Dies with agent
- Use for working state only
- Transfer important findings to L2 (scratchpad)

### L2: Phase Scratchpad (Shared Within Phase)

Fast communication channel for agents working in the same phase.

```
Access: All agents in current phase
Lifetime: Duration of phase (archived at phase end)
Location: .grid/phases/{phase_id}/scratchpad.md
```

**When to write to L2:**
- Discovery that affects other agents in phase
- Decision that other agents need to know
- Gotcha/trap that saves others time
- Handoff information for continuation
- Heartbeat/progress updates

### L3: STATE.md (Global)

Mission-wide state visible to all agents.

```
Access: All agents (write restricted by type)
Lifetime: Duration of mission
Location: .grid/STATE.md
```

**When to write to L3:**
- Position changes (thread/block/phase complete)
- Status changes (blocked, complete)
- Agent registry updates
- Cross-phase decisions
- Blockers that affect mission

### L4: LEARNINGS.md (Persistent)

Knowledge that survives across missions.

```
Access: Read: All | Write: Memory Agent
Lifetime: Permanent (until manual cleanup)
Location: .grid/LEARNINGS.md
```

**What goes in L4:**
- Validated patterns (3+ observations)
- User preferences (explicit or inferred)
- Codebase conventions
- Architectural decisions
- Failure patterns to avoid

### Memory Hierarchy Flow

```
Agent discovers something important
    │
    ▼
┌─────────────────────────────────────────────────────────┐
│ Is it relevant to OTHER agents in THIS phase?           │
│ YES → Write to L2 (Phase Scratchpad)                    │
│ NO  → Keep in L1 (local context)                        │
└─────────────────────────────────────────────────────────┘
    │
    ▼
┌─────────────────────────────────────────────────────────┐
│ Does it affect MISSION STATE or OTHER phases?           │
│ YES → Also update L3 (STATE.md)                         │
│ NO  → L2 is sufficient                                  │
└─────────────────────────────────────────────────────────┘
    │
    ▼
┌─────────────────────────────────────────────────────────┐
│ At phase end, Memory Agent evaluates L2 entries:        │
│ HIGH relevance + 3+ validations → Promote to L4         │
│ Otherwise → Archive only                                │
└─────────────────────────────────────────────────────────┘
```

### Promotion Rules

```python
def should_promote_to_learnings(entry: dict) -> bool:
    """
    Determine if scratchpad entry should be promoted to LEARNINGS.md.
    """
    # Always promote user preferences
    if "user_preference" in entry.get("tags", []):
        return True

    # Always promote architectural decisions
    if entry["category"] == "decision" and "architectural" in entry.get("tags", []):
        return True

    # Promote high-relevance patterns with evidence
    if entry["relevance"] == "HIGH":
        if entry["category"] in ["pattern", "gotcha", "discovery"]:
            return True

    # Promote anything with explicit "promote" tag
    if "promote" in entry.get("tags", []):
        return True

    return False
```

---

## 5. File Structure

```
.grid/
├── STATE.md                          # L3 - Global mission state
├── LEARNINGS.md                      # L4 - Persistent knowledge
├── CHECKPOINT.md                     # Current checkpoint (if any)
├── config.json                       # User configuration
├── budget.json                       # Cost tracking
│
├── events/                           # Event stream
│   ├── current.jsonl                 # Event log (JSONL)
│   └── latest.json                   # Most recent event
│
├── phases/                           # Per-phase data
│   ├── 01-foundation/
│   │   ├── scratchpad.md             # L2 - Phase scratchpad
│   │   ├── scratchpad_archived.md    # Archived entries
│   │   ├── 01-PLAN.md                # Block plans
│   │   ├── 02-PLAN.md
│   │   ├── 01-SUMMARY.md             # Block summaries
│   │   └── 02-SUMMARY.md
│   │
│   └── 02-core/
│       ├── scratchpad.md
│       └── ...
│
├── memory/                           # Memory Agent workspace
│   ├── INDEX.md                      # Fast lookup index
│   ├── patterns.md                   # Codebase patterns
│   ├── gotchas.md                    # Traps to avoid
│   ├── decisions.md                  # Architectural decisions
│   ├── tech-context.md               # Tech stack knowledge
│   └── compressed/                   # Compressed old phases
│       ├── phase-01.md
│       └── phase-02.md
│
└── debug/                            # Debug sessions
    └── {session_id}/
        └── investigation.md
```

---

## 6. Implementation Checklist

### Phase 1: Core Infrastructure
- [ ] STATE.md schema implementation
- [ ] YAML frontmatter parser
- [ ] File locking mechanism
- [ ] Basic read/write operations

### Phase 2: Scratchpad System
- [ ] Per-phase scratchpad creation
- [ ] Entry writing with index update
- [ ] Lookup helpers (by topic, agent, relevance)
- [ ] Auto-archive mechanism

### Phase 3: Notification System
- [ ] Event file creation
- [ ] emit_blackboard_event function
- [ ] EventStreamReader class
- [ ] Subscription registration

### Phase 4: Memory Hierarchy
- [ ] L1 agent context guidelines
- [ ] L2 -> L3 promotion rules
- [ ] L2 -> L4 extraction
- [ ] Memory agent integration

### Phase 5: Conflict Handling
- [ ] Write conflict detection
- [ ] Merge protocol for scratchpad
- [ ] Checkpoint creation on file conflicts

---

## 7. Examples

### Example: Executor Writes Discovery

```python
# Executor discovers that auth middleware order matters
write_scratchpad_entry(
    phase_id="01-foundation",
    agent_id="executor-001",
    category="gotcha",
    topic="auth",
    content="""Gotcha: Auth middleware must run AFTER body parsing middleware.

First hit: Validation errors were leaking auth state because auth ran first.

Fix: Reorder middleware in src/middleware/index.ts:
1. bodyParser
2. cors
3. auth  <-- moved after bodyParser
4. validation

Evidence:
- File: src/middleware/index.ts
- Lines: 15-30
- Commit: abc123""",
    tags=["auth", "middleware", "order", "body-parser"],
    relevance="HIGH",
    block_id="01-02"
)
```

### Example: Reading Relevant Context

```python
# Scout preparing context for planner
def gather_context_for_planning(phase_id: str, work_description: str) -> dict:
    """
    Gather relevant context from blackboard for planning.
    """
    # Extract keywords from work description
    keywords = extract_keywords(work_description)  # ["auth", "api", "user"]

    # Read global state
    state = read_state()

    # Read relevant learnings
    learnings = filter_learnings_for_context(
        read_learnings(),
        work_context=work_description,
        max_entries=10
    )

    # Read phase scratchpad entries by topic
    scratchpad_entries = []
    for keyword in keywords:
        entries = lookup_scratchpad_by_topic(phase_id, keyword)
        scratchpad_entries.extend(entries)

    # Deduplicate and sort by relevance
    scratchpad_entries = deduplicate(scratchpad_entries)
    scratchpad_entries.sort(key=lambda e: relevance_score(e), reverse=True)

    return {
        "state": state,
        "learnings": learnings,
        "phase_context": scratchpad_entries[:15],  # Top 15 relevant
        "recent_decisions": [e for e in scratchpad_entries if e["category"] == "decision"][:5]
    }
```

### Example: Phase Transition

```python
def transition_to_next_phase(completed_phase_id: str, next_phase_id: str):
    """
    Handle phase transition with blackboard updates.
    """
    # 1. Archive completed phase scratchpad
    archive_phase_scratchpad(completed_phase_id)

    # 2. Extract learnings
    high_value_entries = get_high_value_entries(completed_phase_id)
    for entry in high_value_entries:
        extract_to_learnings(entry)

    # 3. Update global state
    update_state({
        "phases": {
            "current": int(next_phase_id.split("-")[0]),
            "completed": get_completed_phases() + [completed_phase_id]
        },
        "position": {
            "phase": int(next_phase_id.split("-")[0]),
            "phase_id": next_phase_id,
            "block": 1,
            "wave": 1,
            "thread": 1
        }
    }, agent_id="mc", agent_type="mc")

    # 4. Create new phase scratchpad
    create_phase_scratchpad(next_phase_id)

    # 5. Emit phase transition event
    emit_blackboard_event("phase_changed", {
        "old_phase": completed_phase_id,
        "new_phase": next_phase_id
    })

    # 6. Carry forward critical context
    carry_forward_context(
        from_phase=completed_phase_id,
        to_phase=next_phase_id,
        context_types=["handoff", "blocker", "warning"]
    )
```

---

## 8. Performance Considerations

### File Access Patterns

| Operation | Expected Latency | Optimization |
|-----------|-----------------|--------------|
| Read STATE.md | <10ms | Cached in memory during execution |
| Write scratchpad entry | <50ms | Append-only, async index update |
| Lookup by topic | <20ms | YAML frontmatter index |
| Full scratchpad scan | <100ms | Avoid, use index |
| Archive scratchpad | <500ms | Batched, end of phase only |

### Concurrency Guidelines

1. **STATE.md**: Use advisory file lock for writes
2. **Scratchpad**: Append-only design avoids most conflicts
3. **Events**: JSONL append is naturally concurrent
4. **LEARNINGS.md**: Single writer (Memory Agent)

### Memory Budget

```yaml
# Approximate token costs for blackboard reads
token_estimates:
  state_md_header: 500      # YAML frontmatter
  state_md_full: 1500       # Full file
  scratchpad_index: 200     # YAML index only
  scratchpad_entry: 100     # Single entry
  scratchpad_full: 2000     # 50 entries
  learnings_filtered: 500   # Relevant subset
  learnings_full: 3000      # Full file

# Budget allocation per agent
agent_blackboard_budget:
  executor: 2000   # State + scratchpad entries
  planner: 3000    # State + learnings + scratchpad
  scout: 1000      # Read-only, minimal
  memory: 5000     # Full access for compression
```

---

## 9. Security and Privacy

### Access Control Matrix

| Resource | MC | Planner | Executor | Scout | Recognizer | Memory |
|----------|-----|---------|----------|-------|------------|--------|
| STATE.md (read) | Y | Y | Y | Y | Y | Y |
| STATE.md (write) | Y | Y* | Y* | N | Y* | Y* |
| Scratchpad (read) | Y | Y | Y | Y | Y | Y |
| Scratchpad (write) | Y | N | Y | N | Y | Y |
| LEARNINGS (read) | Y | Y | Y | Y | Y | Y |
| LEARNINGS (write) | N | N | N | N | N | Y |
| Events (emit) | Y | Y | Y | Y | Y | Y |

*Limited to specific fields (see STATE.md schema)

### Sensitive Data Handling

```python
SENSITIVE_PATTERNS = [
    r"api[_-]?key",
    r"secret",
    r"password",
    r"token",
    r"credential",
    r"private[_-]?key"
]

def sanitize_scratchpad_entry(entry: dict) -> dict:
    """
    Remove sensitive data before writing to scratchpad.
    """
    content = entry["content"]

    for pattern in SENSITIVE_PATTERNS:
        # Replace values, not keys
        content = re.sub(
            rf'({pattern})\s*[:=]\s*["\']?[\w\-\.]+["\']?',
            r'\1: [REDACTED]',
            content,
            flags=re.IGNORECASE
        )

    entry["content"] = content
    return entry
```

---

*End of Blackboard Specification. End of Line.*