# Grid Scratchpad Protocol **Version:** 1.7.x **Status:** Active **Last Updated:** 2026-01-24 --- ## Overview Phase Scratchpads are the **L2 Shared Memory** layer of The Grid's Blackboard system. They enable fast inter-agent communication within a phase - the "shared memory" that agents in the same compute unit can access. This document specifies the scratchpad format, entry protocol, and memory hierarchy integration. ``` Location: .grid/phases/{phase_id}/scratchpad.md Access: All agents in phase (read/write) Lifetime: Duration of phase (archived on completion) Format: YAML frontmatter (index) + Markdown entries ``` --- ## 1. File Location and Structure ### 1.1 Directory Layout ``` .grid/ ├── STATE.md # L3 - Global mission state ├── LEARNINGS.md # L4 - Persistent knowledge │ └── phases/ # Per-phase directories ├── 01-foundation/ │ ├── scratchpad.md # Active scratchpad │ ├── scratchpad_archived.md # Archived entries (post-phase) │ ├── 01-PLAN.md # Block plans │ ├── 02-PLAN.md │ ├── 01-SUMMARY.md # Block summaries │ └── 02-SUMMARY.md │ ├── 02-core/ │ ├── scratchpad.md │ └── ... │ └── 03-polish/ ├── scratchpad.md └── ... ``` ### 1.2 Scratchpad File Structure ```markdown --- # YAML Frontmatter (Index) phase_id: "01-foundation" phase_name: "Foundation" ... --- # Markdown Body (Entries) ## Entries ### [1] [TIMESTAMP] agent_id | category | topic ... ### [2] [TIMESTAMP] agent_id | category | topic ... ``` --- ## 2. YAML Frontmatter Schema ### 2.1 Complete Schema ```yaml --- # ═══════════════════════════════════════════════════════════════════════════════ # PHASE SCRATCHPAD - Inter-Agent Communication # ═══════════════════════════════════════════════════════════════════════════════ # Metadata phase_id: string # Required. Format: "{NN}-{slug}" (e.g., "01-foundation") phase_name: string # Required. Human-readable phase name created_at: string # Required. ISO 8601 timestamp last_updated: string # Required. ISO 8601 timestamp (auto-updated) # ───────────────────────────────────────────────────────────────────────────────── # INDEX (Fast Lookup) # ───────────────────────────────────────────────────────────────────────────────── index: entry_count: integer # Total entries (current, not archived) max_entries: integer # Archive threshold (default: 50) # Lookup by topic keyword by_topic: auth: [1, 5, 12] # Entry IDs related to "auth" database: [2, 3, 8] api: [4, 6, 7, 9] # ... more topics # Lookup by agent by_agent: executor-001: [1, 2, 3] executor-002: [4, 5, 6] scout-001: [7, 8] # ... more agents # Lookup by category by_category: discovery: [1, 4, 7] decision: [2, 5] gotcha: [3, 6] pattern: [8, 9] warning: [10] question: [11] blocker: [12] handoff: [13, 14, 15] heartbeat: [16, 17] # Lookup by relevance level by_relevance: HIGH: [1, 5, 9, 12] MEDIUM: [2, 3, 4, 6, 7, 8] LOW: [10, 11, 13, 14, 15] # Most recent entries (newest first, max 10) recent: [17, 16, 15, 14, 13, 12, 11, 10, 9, 8] # ───────────────────────────────────────────────────────────────────────────────── # SUBSCRIPTIONS (Write Notifications) # ───────────────────────────────────────────────────────────────────────────────── subscriptions: - agent_id: string # Agent to notify topics: array # Topics to monitor (["auth", "database"] or ["*"] for all) relevance_min: enum # Minimum relevance: HIGH | MEDIUM | LOW # ───────────────────────────────────────────────────────────────────────────────── # ARCHIVE INFO # ───────────────────────────────────────────────────────────────────────────────── archive: archived_count: integer # Total entries archived last_archive: string # ISO 8601 timestamp of last archive archive_file: string # Path to archive file --- ``` ### 2.2 Default Values ```yaml --- phase_id: "{phase_id}" phase_name: "{phase_name}" created_at: "{ISO8601}" last_updated: "{ISO8601}" index: entry_count: 0 max_entries: 50 by_topic: {} by_agent: {} by_category: discovery: [] decision: [] gotcha: [] pattern: [] warning: [] question: [] blocker: [] handoff: [] heartbeat: [] by_relevance: HIGH: [] MEDIUM: [] LOW: [] recent: [] subscriptions: [] archive: archived_count: 0 last_archive: null archive_file: ".grid/phases/{phase_id}/scratchpad_archived.md" --- ``` --- ## 3. Entry Format ### 3.1 Entry Header ```markdown ### [{entry_id}] [{ISO8601_timestamp}] {agent_id} | {category} | {topic} ``` **Components:** - `entry_id`: Sequential integer (1, 2, 3, ...) - `ISO8601_timestamp`: Format: `2026-01-24T14:30:00Z` - `agent_id`: Format: `{type}-{NNN}` (e.g., `executor-001`) - `category`: One of the defined categories (see 3.2) - `topic`: Lowercase keyword (e.g., `auth`, `database`, `api`) ### 3.2 Categories | Category | Purpose | When to Use | |----------|---------|-------------| | `discovery` | New finding about codebase | Found unexpected behavior, hidden config, undocumented API | | `decision` | Choice made during execution | Chose library A over B, decided on architecture pattern | | `gotcha` | Trap or pitfall found | Bug that was hard to diagnose, confusing behavior | | `pattern` | Recognized codebase pattern | Consistent style, naming conventions, architectural patterns | | `warning` | Potential issue for others | Non-blocking concern, risk to monitor | | `question` | Needs resolution | Ambiguity needing clarification, missing context | | `blocker` | Execution blocked | Cannot proceed without external input | | `handoff` | Work transfer to next agent | Summary for continuation agent | | `heartbeat` | Progress update | Status check, alive signal (see Heartbeat section) | ### 3.3 Entry Body Template ```markdown ### [{id}] [{timestamp}] {agent_id} | {category} | {topic} **Topic:** {Human readable topic name} **Tags:** {comma, separated, tags} **Relevance:** HIGH | MEDIUM | LOW **Block:** {block_id} {Category-specific content - see 3.4} --- ``` ### 3.4 Category-Specific Content #### Discovery ```markdown Found: {What was discovered} Impact: {Who/what is affected} Action: {What was done about it} Evidence: - File: {filepath} - Line: {line numbers} - Commit: {commit hash} ``` #### Decision ```markdown Decision: {What was decided} Rationale: {Why this choice} Affects: {Components/files affected} Alternatives: {What was considered but rejected} ``` #### Gotcha ```markdown Gotcha: {The trap/pitfall} First Hit: {How it was discovered - symptoms} Fix: {How to avoid/fix it} Evidence: {Where in code} ``` #### Pattern ```markdown Pattern: {Pattern name or description} Where Found: {Files/components where observed} Usage: {How to apply this pattern} Examples: - {Example 1} - {Example 2} ``` #### Warning ```markdown Warning: {The concern} Risk Level: {low|medium|high} Mitigation: {How to address} Monitor: {What to watch for} ``` #### Question ```markdown Question: {The question} Context: {Why this question arose} Urgency: {low|medium|high} Attempted: {What was tried to resolve} ``` #### Blocker ```markdown Blocker: {What is blocked} Type: {missing_credential|external_api|permission|dependency} Required: {What is needed} Blocking: {Thread/task being blocked} Action Required: {Specific action user must take} ``` #### Handoff ```markdown Handoff: {What is being handed off} **Completed:** - {Task 1 done} - {Task 2 done} **For Next Agent:** - {Key info 1} - {Key info 2} **Files to Review:** - {filepath1} - {filepath2} **Watch Out For:** - {Known issue} ``` #### Heartbeat ```markdown **Status:** {Current task/thread} **Progress:** {percent}% **Current Action:** {What agent is doing right now} **Files Touched:** {Files modified this session} **Duration:** {Time since last heartbeat} ``` --- ## 4. Relevance Levels ### 4.1 Level Definitions | Level | Definition | Promotion to L4 | Examples | |-------|------------|-----------------|----------| | **HIGH** | Affects multiple systems, blocks progress, or reveals critical pattern | Automatic candidate | Auth middleware order bug, architectural decisions, user preferences | | **MEDIUM** | Affects current subsystem or block | Manual review | Local gotchas, block-specific patterns, non-blocking warnings | | **LOW** | Nice to know, minor detail | Rarely promoted | Heartbeats, minor observations, style preferences | ### 4.2 Relevance Assignment Guidelines ```python def determine_relevance(entry: dict) -> str: """ Determine relevance level for a scratchpad entry. """ category = entry["category"] content = entry["content"] tags = entry.get("tags", []) # HIGH relevance triggers if category == "blocker": return "HIGH" if category == "decision" and "architectural" in tags: return "HIGH" if "user_preference" in tags: return "HIGH" if "security" in tags or "auth" in tags: return "HIGH" if entry.get("affects_multiple", False): return "HIGH" # MEDIUM relevance triggers if category in ["gotcha", "pattern", "discovery"]: return "MEDIUM" if category == "decision": return "MEDIUM" if category == "warning": return "MEDIUM" # LOW by default return "LOW" ``` --- ## 5. Write Protocol ### 5.1 Write Entry Function ```python from datetime import datetime from pathlib import Path import yaml import re def write_scratchpad_entry( phase_id: str, agent_id: str, category: str, topic: str, content: str, tags: list[str], relevance: str, block_id: str ) -> int: """ Write entry to phase scratchpad with index update and notification. Args: phase_id: Phase identifier (e.g., "01-foundation") agent_id: Writing agent's ID (e.g., "executor-001") category: Entry category (discovery, decision, etc.) topic: Topic keyword (e.g., "auth", "database") content: Entry body content tags: List of tag keywords relevance: Relevance level (HIGH, MEDIUM, LOW) block_id: Current block ID Returns: Entry ID of the new entry """ scratchpad_path = Path(f".grid/phases/{phase_id}/scratchpad.md") # 1. Ensure scratchpad exists if not scratchpad_path.exists(): create_phase_scratchpad(phase_id) # 2. Read current state scratchpad_content = scratchpad_path.read_text() frontmatter, body = parse_scratchpad(scratchpad_content) # 3. Get next entry ID entry_id = frontmatter["index"]["entry_count"] + 1 timestamp = datetime.utcnow().strftime("%Y-%m-%dT%H:%M:%SZ") # 4. Check archive threshold if entry_id > frontmatter["index"]["max_entries"]: archive_old_entries(phase_id, keep_recent=25) # Re-read after archive scratchpad_content = scratchpad_path.read_text() frontmatter, body = parse_scratchpad(scratchpad_content) entry_id = frontmatter["index"]["entry_count"] + 1 # 5. Format entry entry = format_entry( entry_id=entry_id, timestamp=timestamp, agent_id=agent_id, category=category, topic=topic, content=content, tags=tags, relevance=relevance, block_id=block_id ) # 6. Update index update_index(frontmatter, entry_id, agent_id, category, topic, relevance) frontmatter["index"]["entry_count"] = entry_id frontmatter["last_updated"] = timestamp # 7. Write back new_content = format_scratchpad_file(frontmatter, body + entry) scratchpad_path.write_text(new_content) # 8. Notify subscribers notify_subscribers( phase_id=phase_id, entry_id=entry_id, topic=topic, relevance=relevance, agent_id=agent_id, subscriptions=frontmatter.get("subscriptions", []) ) # 9. Emit event emit_blackboard_event("entry_written", { "phase_id": phase_id, "entry_id": entry_id, "topic": topic, "relevance": relevance, "agent": agent_id, "category": category }) return entry_id def update_index(frontmatter: dict, entry_id: int, agent_id: str, category: str, topic: str, relevance: str): """Update all index sections with new entry.""" idx = frontmatter["index"] # by_topic if topic not in idx["by_topic"]: idx["by_topic"][topic] = [] idx["by_topic"][topic].append(entry_id) # by_agent if agent_id not in idx["by_agent"]: idx["by_agent"][agent_id] = [] idx["by_agent"][agent_id].append(entry_id) # by_category if category not in idx["by_category"]: idx["by_category"][category] = [] idx["by_category"][category].append(entry_id) # by_relevance idx["by_relevance"][relevance].append(entry_id) # recent (keep max 10, newest first) idx["recent"].insert(0, entry_id) idx["recent"] = idx["recent"][:10] def format_entry(entry_id: int, timestamp: str, agent_id: str, category: str, topic: str, content: str, tags: list[str], relevance: str, block_id: str) -> str: """Format entry as markdown.""" tags_str = ", ".join(tags) return f""" ### [{entry_id}] [{timestamp}] {agent_id} | {category} | {topic} **Topic:** {topic.title().replace("_", " ")} **Tags:** {tags_str} **Relevance:** {relevance} **Block:** {block_id} {content} --- """ ``` ### 5.2 Read Entry Function ```python def read_scratchpad_entries( phase_id: str, topic: str = None, category: str = None, relevance: str = None, agent_id: str = None, limit: int = None ) -> list[dict]: """ Read scratchpad entries with optional filtering. Args: phase_id: Phase to read from topic: Filter by topic keyword category: Filter by category relevance: Filter by minimum relevance (HIGH returns only HIGH, MEDIUM returns HIGH+MEDIUM, LOW returns all) agent_id: Filter by writing agent limit: Maximum entries to return Returns: List of entry dictionaries """ scratchpad_path = Path(f".grid/phases/{phase_id}/scratchpad.md") if not scratchpad_path.exists(): return [] content = scratchpad_path.read_text() frontmatter, body = parse_scratchpad(content) idx = frontmatter["index"] # Determine entry IDs to fetch entry_ids = set(range(1, idx["entry_count"] + 1)) # Apply filters using index if topic and topic in idx["by_topic"]: entry_ids &= set(idx["by_topic"][topic]) elif topic: return [] # Topic not found if category and category in idx["by_category"]: entry_ids &= set(idx["by_category"][category]) elif category: return [] if agent_id and agent_id in idx["by_agent"]: entry_ids &= set(idx["by_agent"][agent_id]) elif agent_id: return [] if relevance: relevance_ids = set() if relevance == "LOW": relevance_ids = set(idx["by_relevance"]["HIGH"] + idx["by_relevance"]["MEDIUM"] + idx["by_relevance"]["LOW"]) elif relevance == "MEDIUM": relevance_ids = set(idx["by_relevance"]["HIGH"] + idx["by_relevance"]["MEDIUM"]) else: # HIGH relevance_ids = set(idx["by_relevance"]["HIGH"]) entry_ids &= relevance_ids # Parse entries from body entries = parse_entries_from_body(body) filtered = [e for e in entries if e["id"] in entry_ids] # Sort by recency (newest first) filtered.sort(key=lambda e: e["timestamp"], reverse=True) if limit: filtered = filtered[:limit] return filtered ``` ### 5.3 Lookup Helpers ```python def lookup_by_topic(phase_id: str, topic: str) -> list[dict]: """Get all entries for a topic.""" return read_scratchpad_entries(phase_id, topic=topic) def lookup_by_category(phase_id: str, category: str) -> list[dict]: """Get all entries of a category.""" return read_scratchpad_entries(phase_id, category=category) def get_recent_entries(phase_id: str, count: int = 10) -> list[dict]: """Get most recent entries.""" return read_scratchpad_entries(phase_id, limit=count) def get_high_relevance(phase_id: str) -> list[dict]: """Get all HIGH relevance entries.""" return read_scratchpad_entries(phase_id, relevance="HIGH") def get_agent_entries(phase_id: str, agent_id: str) -> list[dict]: """Get all entries from a specific agent.""" return read_scratchpad_entries(phase_id, agent_id=agent_id) def get_blockers(phase_id: str) -> list[dict]: """Get all blocker entries.""" return read_scratchpad_entries(phase_id, category="blocker") def get_handoffs(phase_id: str) -> list[dict]: """Get all handoff entries.""" return read_scratchpad_entries(phase_id, category="handoff") ``` --- ## 6. Heartbeat Protocol ### 6.1 Purpose Heartbeats enable staleness detection by MC. If an executor hasn't written a heartbeat in 10+ minutes, MC considers it stale and may intervene. ### 6.2 Heartbeat Frequency | Trigger | When | |---------|------| | Timer | Every 5 minutes during execution | | File created | After each file write | | Commit made | After successful git commit | | Long operation start | Before npm install, large generation | | Verification complete | After test pass/fail | ### 6.3 Heartbeat Entry Format ```markdown ### [17] [2026-01-24T16:30:00Z] executor-001 | heartbeat | progress **Topic:** Heartbeat **Tags:** heartbeat, progress, status **Relevance:** LOW **Block:** 01-03 **Status:** Working on thread 2 **Progress:** 60% **Current Action:** Writing POST handler for /api/auth **Files Touched:** src/api/auth/route.ts, src/types/user.ts **Duration:** 5 minutes since last heartbeat --- ``` ### 6.4 Heartbeat Function ```python def write_heartbeat( phase_id: str, agent_id: str, block_id: str, thread_info: str, progress_percent: int, current_action: str, files_touched: list[str] ): """Write heartbeat entry to scratchpad.""" files_str = ", ".join(files_touched) if files_touched else "None yet" content = f"""**Status:** Working on {thread_info} **Progress:** {progress_percent}% **Current Action:** {current_action} **Files Touched:** {files_str} **Duration:** 5 minutes since last heartbeat""" write_scratchpad_entry( phase_id=phase_id, agent_id=agent_id, category="heartbeat", topic="progress", content=content, tags=["heartbeat", "progress", "status"], relevance="LOW", block_id=block_id ) ``` ### 6.5 Staleness Detection ```python def check_agent_staleness(phase_id: str, agent_id: str) -> dict: """ Check if an agent is stale based on heartbeats. Returns: { "stale": bool, "last_heartbeat": str|None, "minutes_since": int|None, "warning_level": "ok"|"warning"|"stale" } """ heartbeats = read_scratchpad_entries( phase_id=phase_id, agent_id=agent_id, category="heartbeat", limit=1 ) if not heartbeats: return { "stale": True, "last_heartbeat": None, "minutes_since": None, "warning_level": "stale" } last = heartbeats[0] last_time = datetime.fromisoformat(last["timestamp"].replace("Z", "+00:00")) now = datetime.now(timezone.utc) minutes_since = (now - last_time).total_seconds() / 60 if minutes_since > 10: level = "stale" stale = True elif minutes_since > 5: level = "warning" stale = False else: level = "ok" stale = False return { "stale": stale, "last_heartbeat": last["timestamp"], "minutes_since": int(minutes_since), "warning_level": level } ``` --- ## 7. Archive Protocol ### 7.1 When to Archive | Trigger | Action | |---------|--------| | `entry_count > max_entries` | Archive oldest 25 entries | | Phase complete | Archive all entries, extract to L4 | | Manual cleanup | Archive on command | ### 7.2 Mid-Phase Archive (Auto) ```python def archive_old_entries(phase_id: str, keep_recent: int = 25): """ Archive old entries when threshold exceeded. Called automatically when entry_count > max_entries. """ scratchpad_path = Path(f".grid/phases/{phase_id}/scratchpad.md") archive_path = Path(f".grid/phases/{phase_id}/scratchpad_archived.md") # Read current content = scratchpad_path.read_text() frontmatter, body = parse_scratchpad(content) # Parse all entries entries = parse_entries_from_body(body) total = len(entries) # Keep most recent to_keep = entries[-keep_recent:] to_archive = entries[:-keep_recent] # Append to archive file if archive_path.exists(): archive_content = archive_path.read_text() else: archive_content = f"""--- phase_id: {phase_id} archived_at: {datetime.utcnow().strftime("%Y-%m-%dT%H:%M:%SZ")} --- # Phase {phase_id} Scratchpad Archive ## Archived Entries """ for entry in to_archive: archive_content += format_archived_entry(entry) archive_path.write_text(archive_content) # Rebuild scratchpad with remaining entries new_frontmatter = rebuild_frontmatter(phase_id, to_keep) new_frontmatter["archive"]["archived_count"] += len(to_archive) new_frontmatter["archive"]["last_archive"] = datetime.utcnow().strftime("%Y-%m-%dT%H:%M:%SZ") new_body = "" for i, entry in enumerate(to_keep, start=1): entry["id"] = i # Renumber new_body += format_entry_from_dict(entry) scratchpad_path.write_text(format_scratchpad_file(new_frontmatter, new_body)) print(f"[Scratchpad] Archived {len(to_archive)} entries, kept {len(to_keep)}") ``` ### 7.3 Phase End Archive ```python def archive_phase_scratchpad(phase_id: str): """ Archive entire scratchpad at phase completion. Extracts high-value entries to LEARNINGS.md. """ scratchpad_path = Path(f".grid/phases/{phase_id}/scratchpad.md") archive_path = Path(f".grid/phases/{phase_id}/scratchpad_archived.md") if not scratchpad_path.exists(): return content = scratchpad_path.read_text() frontmatter, body = parse_scratchpad(content) entries = parse_entries_from_body(body) # 1. Extract high-value entries for LEARNINGS.md high_value = [e for e in entries if should_promote_to_learnings(e)] for entry in high_value: extract_to_learnings(entry) # 2. Create comprehensive archive archive_content = f"""--- phase_id: {phase_id} phase_name: {frontmatter["phase_name"]} archived_at: {datetime.utcnow().strftime("%Y-%m-%dT%H:%M:%SZ")} total_entries: {len(entries)} high_value_extracted: {len(high_value)} categories: discovery: {len([e for e in entries if e["category"] == "discovery"])} decision: {len([e for e in entries if e["category"] == "decision"])} gotcha: {len([e for e in entries if e["category"] == "gotcha"])} pattern: {len([e for e in entries if e["category"] == "pattern"])} warning: {len([e for e in entries if e["category"] == "warning"])} question: {len([e for e in entries if e["category"] == "question"])} blocker: {len([e for e in entries if e["category"] == "blocker"])} handoff: {len([e for e in entries if e["category"] == "handoff"])} heartbeat: {len([e for e in entries if e["category"] == "heartbeat"])} --- # Phase {phase_id} - {frontmatter["phase_name"]} - Scratchpad Archive ## Summary - **Total Entries:** {len(entries)} - **High-Value Extracted to LEARNINGS.md:** {len(high_value)} - **Phase Duration:** {frontmatter["created_at"]} to {frontmatter["last_updated"]} ## Extracted to LEARNINGS.md {format_extraction_summary(high_value)} ## All Entries {format_all_entries(entries)} """ # Write archive archive_path.write_text(archive_content) # Clear active scratchpad (reset for potential reuse) create_phase_scratchpad(phase_id, frontmatter["phase_name"]) # Emit event emit_blackboard_event("phase_archived", { "phase_id": phase_id, "entries_archived": len(entries), "entries_promoted": len(high_value) }) print(f"[Scratchpad] Phase {phase_id} archived: {len(entries)} entries, {len(high_value)} promoted to L4") ``` --- ## 8. Memory Hierarchy The Grid uses a four-level memory hierarchy, similar to a GPU's shared memory architecture. Each level has different access patterns, lifetimes, and purposes. ### 8.1 Hierarchy Overview ``` ┌─────────────────────────────────────────────────────────────────────────────┐ │ MEMORY HIERARCHY │ ├─────────────────────────────────────────────────────────────────────────────┤ │ │ │ L4: LEARNINGS.md ← Persistent knowledge (ROM-like) │ │ ───────────────── Lifetime: Permanent (survives missions) │ │ Location: .grid/LEARNINGS.md │ │ Access: Read=All | Write=Memory Agent only │ │ │ │ L3: STATE.md ← Global mission state (shared registers) │ │ ───────────── Lifetime: Mission duration │ │ Location: .grid/STATE.md │ │ Access: Read=All | Write=Restricted by agent type │ │ │ │ L2: Phase Scratchpad ← Inter-agent communication (shared memory) │ │ ───────────────────── Lifetime: Phase duration (archived at end) │ │ Location: .grid/phases/{phase_id}/scratchpad.md │ │ Access: Read=All | Write=All agents in phase │ │ │ │ L1: Agent Context ← Private working memory (registers) │ │ ───────────────── Lifetime: Agent execution (dies with agent) │ │ Location: In-memory only (not persisted) │ │ Access: Private to agent │ │ │ └─────────────────────────────────────────────────────────────────────────────┘ ``` ### 8.2 L1: Agent Context (Private) **Description:** Each agent's private working memory within its context window. NOT persisted to disk. **Lifetime:** Exists only during agent execution. Dies when agent completes. **Contents:** ```yaml # Conceptual - lives only in agent's context window agent_local: agent_id: "executor-001" # What agent is working on current_task: thread_id: "01-02-thread-3" started_at: "2026-01-24T14:30:00Z" files_touched: [] # Agent's internal reasoning state working_memory: - "Implementing JWT validation" - "Using jose library per warmth" - "Need to handle token refresh" # Recent tool outputs (not persisted) recent_outputs: - tool: "grep" result: "Found 3 usages of deprecated method" # Scratch calculations scratch: estimated_remaining: "15 minutes" complexity_assessment: "medium" ``` **Rules:** - Never persisted to disk - Dies with agent - Use for working state only - Transfer important findings to L2 (scratchpad) **When to promote L1 to L2:** - Discovery affects other agents in phase - Decision other agents need to know - Gotcha/trap that saves others time - Progress that enables continuation ### 8.3 L2: Phase Scratchpad (Shared Within Phase) **Description:** Fast communication channel for agents working in the same phase. This document's primary focus. **Lifetime:** Duration of phase. Archived when phase completes. **Location:** `.grid/phases/{phase_id}/scratchpad.md` **Access:** All agents can read and write. **When to write to L2:** - Discovery that affects other agents in phase - Decision that other agents need to know - Gotcha/trap that saves others time - Handoff information for continuation - Heartbeat/progress updates - Questions needing resolution - Blockers requiring intervention **When NOT to write to L2:** - Private working notes - Verbose tool output - Already documented in code - Duplicate of existing entry ### 8.4 L3: STATE.md (Global) **Description:** Mission-wide state visible to all agents. The single source of truth for mission progress. **Lifetime:** Duration of mission. **Location:** `.grid/STATE.md` **Access:** All agents read. Write restricted by agent type (see STATE_SCHEMA.md). **When to write to L3:** - Position changes (thread/block/phase complete) - Status changes (blocked, complete) - Agent registry updates - Cross-phase decisions - Blockers that affect mission - Budget updates **When NOT to write to L3:** - Phase-local discoveries (use L2) - Temporary observations - Duplicate of L2 content ### 8.5 L4: LEARNINGS.md (Persistent) **Description:** Knowledge that survives across missions. Accumulated patterns, preferences, and architectural decisions. **Lifetime:** Permanent (until manual cleanup). **Location:** `.grid/LEARNINGS.md` **Access:** Read: All | Write: Memory Agent only **What goes in L4:** - Validated patterns (3+ observations) - User preferences (explicit or inferred) - Codebase conventions - Architectural decisions - Failure patterns to avoid - Tech stack knowledge **What does NOT go in L4:** - Temporary observations - Mission-specific details - Unvalidated hypotheses - Redundant information ### 8.6 Memory Flow Diagram ``` Agent discovers something │ ▼ ┌─────────────────────────────────────────────────────────────────┐ │ Is it relevant to OTHER agents in THIS phase? │ │ YES → Write to L2 (Phase Scratchpad) │ │ NO → Keep in L1 (local context) │ └────────────────────────────────────────────────────────────────┘ │ ▼ ┌─────────────────────────────────────────────────────────────────┐ │ Does it affect MISSION STATE or OTHER phases? │ │ YES → Also update L3 (STATE.md) │ │ NO → L2 is sufficient │ └────────────────────────────────────────────────────────────────┘ │ ▼ ┌─────────────────────────────────────────────────────────────────┐ │ At phase end, Memory Agent evaluates L2 entries: │ │ HIGH relevance + promotion criteria → Promote to L4 │ │ Otherwise → Archive only │ └────────────────────────────────────────────────────────────────┘ ``` ### 8.7 Promotion Rules (L2 to L4) ```python def should_promote_to_learnings(entry: dict) -> bool: """ Determine if scratchpad entry should be promoted to LEARNINGS.md. Promotion criteria (any of these): 1. User preference discovered 2. Architectural decision with long-term impact 3. HIGH relevance pattern/gotcha/discovery 4. Explicit "promote" tag """ tags = entry.get("tags", []) category = entry.get("category", "") relevance = entry.get("relevance", "LOW") # Always promote user preferences if "user_preference" in tags: return True # Always promote architectural decisions if category == "decision" and "architectural" in tags: return True # Promote HIGH relevance patterns, gotchas, discoveries if relevance == "HIGH": if category in ["pattern", "gotcha", "discovery"]: return True # Promote explicitly tagged entries if "promote" in tags: return True # Don't promote low relevance or transient entries return False def extract_to_learnings(entry: dict): """ Extract scratchpad entry to LEARNINGS.md. """ learnings_path = Path(".grid/LEARNINGS.md") # Determine learning category if "user_preference" in entry.get("tags", []): learning_category = "user_preferences" elif entry["category"] == "decision": learning_category = "decisions" elif entry["category"] == "gotcha": learning_category = "gotchas" elif entry["category"] == "pattern": learning_category = "patterns" else: learning_category = "discoveries" # Format learning entry learning = format_learning_entry(entry, learning_category) # Append to LEARNINGS.md append_to_learnings(learnings_path, learning, learning_category) # Emit event emit_blackboard_event("learning_extracted", { "from_phase": entry.get("phase_id"), "entry_id": entry.get("id"), "category": learning_category }) ``` ### 8.8 Memory Budget (Token Estimates) ```yaml # Approximate token costs for blackboard reads token_estimates: # L4 - LEARNINGS.md learnings_full: 3000 # Entire file learnings_filtered: 500 # Relevant subset # L3 - STATE.md state_full: 1500 # Full file state_header: 500 # YAML frontmatter only # L2 - Phase Scratchpad scratchpad_full: 2000 # 50 entries scratchpad_index: 200 # YAML index only scratchpad_entry: 100 # Single entry scratchpad_filtered: 500 # Topic/category subset # Budget allocation per agent type agent_blackboard_budget: executor: 2000 # State + scratchpad entries planner: 3000 # State + learnings + scratchpad scout: 1000 # Read-only, minimal recognizer: 1500 # State + warnings memory: 5000 # Full access for compression ``` --- ## 9. Phase Lifecycle ### 9.1 Phase Start ```python def initialize_phase(phase_id: str, phase_name: str): """ Initialize a new phase with empty scratchpad. Called by MC when starting a phase. """ phase_dir = Path(f".grid/phases/{phase_id}") phase_dir.mkdir(parents=True, exist_ok=True) create_phase_scratchpad(phase_id, phase_name) emit_blackboard_event("phase_started", { "phase_id": phase_id, "phase_name": phase_name }) def create_phase_scratchpad(phase_id: str, phase_name: str = None): """Create empty scratchpad for phase.""" if phase_name is None: phase_name = phase_id.replace("-", " ").title() timestamp = datetime.utcnow().strftime("%Y-%m-%dT%H:%M:%SZ") content = f"""--- # ═══════════════════════════════════════════════════════════════════════════════ # PHASE SCRATCHPAD - Inter-Agent Communication # ═══════════════════════════════════════════════════════════════════════════════ phase_id: "{phase_id}" phase_name: "{phase_name}" created_at: "{timestamp}" last_updated: "{timestamp}" index: entry_count: 0 max_entries: 50 by_topic: {{}} by_agent: {{}} by_category: discovery: [] decision: [] gotcha: [] pattern: [] warning: [] question: [] blocker: [] handoff: [] heartbeat: [] by_relevance: HIGH: [] MEDIUM: [] LOW: [] recent: [] subscriptions: [] archive: archived_count: 0 last_archive: null archive_file: ".grid/phases/{phase_id}/scratchpad_archived.md" --- # Phase Scratchpad: {phase_name} ## Entries """ scratchpad_path = Path(f".grid/phases/{phase_id}/scratchpad.md") scratchpad_path.write_text(content) ``` ### 9.2 During Phase ``` ┌─────────────────────────────────────────────────────────────────┐ │ PHASE EXECUTION │ ├─────────────────────────────────────────────────────────────────┤ │ │ │ Agents write entries: │ │ └─> Index updated on each write │ │ └─> Subscribers notified │ │ └─> Events emitted │ │ │ │ Auto-archive when entry_count > max_entries: │ │ └─> Oldest entries moved to archive │ │ └─> Index rebuilt │ │ └─> Entries renumbered │ │ │ │ Agents read entries: │ │ └─> Use index for fast lookup │ │ └─> Filter by topic/category/relevance │ │ │ └─────────────────────────────────────────────────────────────────┘ ``` ### 9.3 Phase Complete ```python def complete_phase(phase_id: str, next_phase_id: str = None): """ Complete a phase: archive scratchpad, extract learnings, transition. Called by MC when all blocks in phase complete. """ # 1. Archive scratchpad and extract to L4 archive_phase_scratchpad(phase_id) # 2. Carry forward critical context if there's a next phase if next_phase_id: carry_forward_context(phase_id, next_phase_id) initialize_phase(next_phase_id, get_phase_name(next_phase_id)) emit_blackboard_event("phase_completed", { "phase_id": phase_id, "next_phase_id": next_phase_id }) def carry_forward_context(from_phase: str, to_phase: str): """ Carry forward critical entries to new phase. """ # Get unresolved blockers, warnings, and handoffs critical = read_scratchpad_entries( from_phase, category="blocker" ) + read_scratchpad_entries( from_phase, category="warning" ) + read_scratchpad_entries( from_phase, category="handoff", limit=5 # Most recent handoffs ) # Write carry-forward entries to new phase for entry in critical: if entry["category"] == "blocker": # Only carry unresolved blockers if not entry.get("resolved", False): write_scratchpad_entry( phase_id=to_phase, agent_id="system", category="blocker", topic=entry["topic"], content=f"[Carried from {from_phase}]\n\n{entry['content']}", tags=entry.get("tags", []) + ["carried_forward"], relevance="HIGH", block_id=f"{to_phase.split('-')[0]}-01" ) elif entry["category"] == "handoff": write_scratchpad_entry( phase_id=to_phase, agent_id="system", category="handoff", topic=entry["topic"], content=f"[Context from {from_phase}]\n\n{entry['content']}", tags=entry.get("tags", []) + ["context"], relevance="MEDIUM", block_id=f"{to_phase.split('-')[0]}-01" ) ``` --- ## 10. Best Practices ### 10.1 Writing Entries **DO:** - Write entries promptly when discoveries are made - Use specific, searchable topics - Include evidence (file paths, line numbers, commits) - Tag with relevant keywords - Set appropriate relevance level - Use structured content for category **DON'T:** - Write verbose tool output - Duplicate existing entries - Use vague topics like "stuff" or "misc" - Over-tag with irrelevant keywords - Mark everything as HIGH relevance - Write heartbeats more frequently than every 5 minutes ### 10.2 Reading Entries **DO:** - Use index-based lookups (by_topic, by_category) - Filter by relevance when context budget is tight - Read recent entries first for current context - Check for blockers before starting work - Read handoffs when continuing another agent's work **DON'T:** - Read entire scratchpad when index lookup suffices - Ignore relevance levels - Skip reading handoffs for continuation tasks ### 10.3 Categories **DO:** - Use `discovery` for new findings - Use `decision` for choices with rationale - Use `gotcha` for traps that cost you time - Use `pattern` for consistent conventions - Use `blocker` only for actual blockers - Use `handoff` at end of work session **DON'T:** - Use `blocker` for warnings (use `warning`) - Use `decision` for discoveries - Mix multiple categories in one entry - Skip `handoff` when continuation is expected ### 10.4 Memory Hierarchy **DO:** - Keep L1 for private working state - Promote to L2 when others need to know - Update L3 for position/status changes - Let Memory Agent manage L4 promotion - Be honest about relevance levels **DON'T:** - Write everything to L4 directly - Skip L2 and go straight to L3 - Treat L2 as private notes - Forget to write heartbeats --- ## 11. Complete Example Scratchpad ```markdown --- # ═══════════════════════════════════════════════════════════════════════════════ # PHASE SCRATCHPAD - Inter-Agent Communication # ═══════════════════════════════════════════════════════════════════════════════ phase_id: "01-foundation" phase_name: "Foundation" created_at: "2026-01-24T14:00:00Z" last_updated: "2026-01-24T16:45:00Z" index: entry_count: 8 max_entries: 50 by_topic: auth: [1, 3, 6] database: [2] api: [4, 5] validation: [5] progress: [7, 8] by_agent: executor-001: [1, 2, 3, 7] executor-002: [4, 5, 6, 8] by_category: discovery: [1] decision: [2] gotcha: [3, 4] pattern: [5] warning: [] question: [] blocker: [] handoff: [6] heartbeat: [7, 8] by_relevance: HIGH: [1, 3, 4] MEDIUM: [2, 5, 6] LOW: [7, 8] recent: [8, 7, 6, 5, 4, 3, 2, 1] subscriptions: - agent_id: "executor-002" topics: ["auth"] relevance_min: "MEDIUM" archive: archived_count: 0 last_archive: null archive_file: ".grid/phases/01-foundation/scratchpad_archived.md" --- # Phase Scratchpad: Foundation ## Entries ### [1] [2026-01-24T14:30:00Z] executor-001 | discovery | auth **Topic:** Authentication **Tags:** auth, middleware, order, body-parser **Relevance:** HIGH **Block:** 01-02 Found: Auth middleware must run AFTER body parsing middleware. Impact: All protected routes affected if order is wrong. Action: Updated middleware order in src/middleware/index.ts. Evidence: - File: src/middleware/index.ts - Line: 23-45 - Commit: a1b2c3d --- ### [2] [2026-01-24T14:45:00Z] executor-001 | decision | database **Topic:** Database Schema **Tags:** database, prisma, soft-delete **Relevance:** MEDIUM **Block:** 01-02 Decision: Using soft deletes (deletedAt timestamp) instead of hard deletes. Rationale: Audit trail requirements, data recovery capability. Affects: All entity models in Prisma schema. Alternatives: Hard deletes with separate audit log table (rejected for complexity). --- ### [3] [2026-01-24T15:00:00Z] executor-001 | gotcha | auth **Topic:** JWT Token Handling **Tags:** auth, jwt, jose, refresh **Relevance:** HIGH **Block:** 01-02 Gotcha: The jose library's jwtVerify throws different error types for expired vs invalid tokens. First Hit: All token errors were returning 401 Unauthorized without distinguishing expired from invalid. Fix: Catch JWTExpired separately to return proper error code for token refresh flow. Evidence: - File: src/lib/auth/jwt.ts - Line: 45-60 --- ### [4] [2026-01-24T15:30:00Z] executor-002 | gotcha | api **Topic:** Next.js App Router **Tags:** api, next, request, body, json **Relevance:** HIGH **Block:** 01-03 Gotcha: Next.js App Router uses `req.json()` not `req.body` for JSON parsing. First Hit: POST /api/auth/login returned undefined body - spent 30 min debugging. Fix: Changed all route handlers to `const body = await req.json()`. Evidence: - File: src/app/api/auth/login/route.ts - Commit: d4e5f6g --- ### [5] [2026-01-24T15:45:00Z] executor-002 | pattern | api **Topic:** API Validation Pattern **Tags:** api, validation, zod, pattern **Relevance:** MEDIUM **Block:** 01-03 Pattern: All API routes use Zod for request validation with consistent error format. Where Found: src/app/api/auth/*, src/app/api/users/* Usage: ```typescript const schema = z.object({ email: z.string().email() }); const result = schema.safeParse(body); if (!result.success) { return NextResponse.json( { error: "Validation failed", details: result.error.flatten() }, { status: 400 } ); } ``` Examples: - src/app/api/auth/register/route.ts - src/app/api/users/route.ts --- ### [6] [2026-01-24T16:00:00Z] executor-002 | handoff | auth **Topic:** Auth Implementation Handoff **Tags:** auth, handoff, jwt, continuation **Relevance:** MEDIUM **Block:** 01-02 -> 01-03 Handoff: JWT authentication implementation complete. **Completed:** - Access token generation (15min expiry) - Refresh token rotation (7day expiry) - Token validation middleware - Login/register endpoints - Refresh endpoint **For Next Agent:** - Refresh endpoint at POST /api/auth/refresh - Use jose library for all JWT operations - HttpOnly cookies already configured for refresh tokens - See gotcha entry [3] for error handling pattern **Files to Review:** - src/lib/auth/jwt.ts - src/lib/auth/cookies.ts - src/middleware/auth.ts - src/app/api/auth/*/route.ts **Watch Out For:** - Middleware order (see entry [1]) - req.json() not req.body (see entry [4]) --- ### [7] [2026-01-24T16:30:00Z] executor-001 | heartbeat | progress **Topic:** Heartbeat **Tags:** heartbeat, progress, status **Relevance:** LOW **Block:** 01-03 **Status:** Working on thread 2 **Progress:** 45% **Current Action:** Implementing user CRUD endpoints **Files Touched:** src/app/api/users/route.ts **Duration:** 5 minutes since last heartbeat --- ### [8] [2026-01-24T16:45:00Z] executor-002 | heartbeat | progress **Topic:** Heartbeat **Tags:** heartbeat, progress, status **Relevance:** LOW **Block:** 01-03 **Status:** Working on thread 3 **Progress:** 60% **Current Action:** Implementing user profile endpoint **Files Touched:** src/app/api/users/[id]/route.ts, src/types/user.ts **Duration:** 5 minutes since last heartbeat --- ``` --- *End of Scratchpad Protocol. End of Line.*