# Research Infrastructure Setup

**Implementation Complete**
**Version:** 1.0
**Date:** 2026-01-23

---

## Overview

The research-first infrastructure is now functional and ready for MC integration. This document describes what was created and how to use it.

---

## Directory Structure Created

```
.grid/                              # Local state (gitignored)
├── research/                       # Fresh research outputs
│   ├── README.md                  # Documentation
│   ├── .gitignore                 # Ignore *.md except README
│   └── {topic}-{timestamp}.md     # Research outputs (created by Researchers)
│
├── research_cache/                 # Cached research (24h TTL)
│   ├── README.md                  # Documentation
│   ├── .gitignore                 # Ignore *.md except README/index
│   ├── index.json                 # Cache index with metadata
│   └── {topic-slug}.md            # Cached research (reusable)
│
└── scout/                          # Codebase reconnaissance
    ├── README.md                   # Documentation
    ├── .gitignore                  # Ignore RECON_*.md except README
    └── RECON_{timestamp}.md        # Scout reports (created by Scouts)
```

All directories are in `.grid/` which is gitignored for local state isolation.

---

## Files Created

### Documentation

| File | Purpose |
|------|---------|
| `.grid/research/README.md` | Documents research output directory |
| `.grid/research_cache/README.md` | Documents cache structure and policies |
| `.grid/scout/README.md` | Documents scout report directory |
| `docs/RESEARCH_CONFIG.md` | Configuration schema reference |
| `docs/MC_RESEARCH_INTEGRATION.md` | MC integration implementation guide |
| `docs/RESEARCH_INFRASTRUCTURE.md` | This file - setup summary |

### Gitignores

| File | Purpose |
|------|---------|
| `.grid/research/.gitignore` | Ignore research outputs, keep README |
| `.grid/research_cache/.gitignore` | Ignore cached files, keep README + index |
| `.grid/scout/.gitignore` | Ignore scout reports, keep README |

### State Files

| File | Purpose |
|------|---------|
| `.grid/research_cache/index.json` | Empty cache index (ready for entries) |

---

## Agent Files Verified

Both agent files are properly formatted for spawning:

### `/Users/jacweath/grid/agents/grid-researcher.md`

**Status:** ✅ Functional

**Contents:**
- Role definition
- Mission types (technology, patterns, similar projects, API docs)
- Research protocol (query generation, parallel search, context assembly)
- Caching protocol (cache structure, hit logic, invalidation)
- Search tool selection
- Output format
- Quality rules
- Integration with planning

**Ready to spawn:** Yes - MC can spawn with Task tool

### `/Users/jacweath/grid/agents/grid-scout.md`

**Status:** ✅ Functional

**Contents:**
- Role definition
- Reconnaissance protocol (4 phases in 2 minutes)
- Technology detection
- Pattern detection
- Constraint discovery
- Output format (scout report)
- Speed optimization
- Special modes (greenfield, monorepo)
- Integration with research

**Ready to spawn:** Yes - MC can spawn with Task tool

---

## Cache Index Schema

The cache index is initialized and ready:

```json
{
  "version": "1.0",
  "created": "2026-01-23T20:00:00Z",
  "updated": "2026-01-23T20:00:00Z",
  "entries": []
}
```

As Researchers cache results, entries will be added:

```json
{
  "entries": [
    {
      "key": "nextjs-14-app-router",
      "queries": ["Next.js 14 best practices 2024 2025", ...],
      "created": "2026-01-23T14:00:00Z",
      "expires": "2026-01-24T14:00:00Z",
      "file": "nextjs-14-app-router.md",
      "hit_count": 0,
      "size_kb": 15
    }
  ]
}
```

---

## How MC Spawns Researchers

### 1. Research Decision

MC checks if research should run:

```python
decision = should_run_research(user_request, config)
if not decision['run']:
    # Skip research, go direct to Planner
```

### 2. Scout Spawn (if existing codebase)

```python
scout_prompt = """
First, read ~/.claude/agents/grid-scout.md for your role.

<config>
project_root: /path/to/project
timeout_seconds: 120
max_files_analyzed: 1000
</config>

Scan the codebase. Output to .grid/scout/RECON_{timestamp}.md
"""

scout_result = Task(
    prompt=scout_prompt,
    subagent_type="general-purpose",
    description="Scout: Codebase reconnaissance"
)
```

### 3. Research Needs Extraction

```python
needs = extract_research_needs(user_request, scout_result)
# Returns: [
#   {'type': 'technology', 'topic': 'Next.js 14', 'queries': [...]},
#   {'type': 'pattern', 'topic': 'Server Components', 'queries': [...]},
# ]
```

### 4. Cache Check

```python
cached, needed = check_research_cache(needs, config)
# Returns:
# - cached: Research already in .grid/research_cache/
# - needed: Topics not cached or expired
```

### 5. Researcher Spawn (parallel)

```python
for need in needed:
    researcher_prompt = f"""
First, read ~/.claude/agents/grid-researcher.md for your role.

<research_request>
mission: {need['type']}
topic: {need['topic']}
queries: {need['queries']}
</research_request>

Research and output to .grid/research_cache/{topic}.md
"""

    Task(
        prompt=researcher_prompt,
        subagent_type="general-purpose",
        description=f"Research: {need['topic']}"
    )
```

All Researchers spawn in parallel (single MC message, multiple Task calls).

### 6. Context Assembly

```python
context = {
    'scout': scout_result,
    'cached_research': cached,
    'fresh_research': fresh_results
}
```

### 7. Planner Spawn

```python
planner_prompt = f"""
First, read ~/.claude/agents/grid-planner.md for your role.

<codebase_context>
{context['scout']}
</codebase_context>

<research_context>
{format_research(context['cached_research'])}
{format_research(context['fresh_research'])}
</research_context>

<user_request>
{user_request}
</user_request>

Create plan that respects codebase constraints and uses best practices.
"""

Task(
    prompt=planner_prompt,
    subagent_type="general-purpose",
    description="Planner: Research-informed plan"
)
```

---

## Configuration Schema Added

### Proposed `.grid/config.json`

```json
{
  "version": "1.0",
  "research": {
    "enabled": true,
    "scout": {
      "enabled": true,
      "timeout_seconds": 120,
      "max_files_analyzed": 1000,
      "depth_limit": 4
    },
    "researcher": {
      "enabled": true,
      "timeout_seconds": 300,
      "max_researchers_parallel": 3,
      "max_queries_per_topic": 10
    },
    "cache": {
      "enabled": true,
      "ttl_hours": 24,
      "max_entries": 50
    },
    "skip_conditions": {
      "quick_mode_eligible": true,
      "debug_mode": true,
      "all_tech_cached": true
    }
  }
}
```

Full schema documented in: `docs/RESEARCH_CONFIG.md`

---

## Integration Checklist

Research infrastructure is complete. Next steps for full integration:

### Phase 1: Validation (Complete)
- [x] Create `.grid/research/` directory
- [x] Create `.grid/research_cache/` directory
- [x] Create `.grid/scout/` directory
- [x] Write README files documenting structure
- [x] Initialize cache index JSON
- [x] Add .gitignore files
- [x] Verify agent files are spawn-ready
- [x] Document configuration schema
- [x] Document MC integration flow

### Phase 2: MC Integration (Ready to implement)
- [ ] Add config loading to MC initialization
- [ ] Implement `should_run_research()` logic
- [ ] Implement `spawn_scout()` function
- [ ] Implement `extract_research_needs()` function
- [ ] Implement `check_research_cache()` function
- [ ] Implement `spawn_researchers()` function
- [ ] Implement context assembly
- [ ] Update Planner spawn with research context

### Phase 3: User Experience (After MC integration)
- [ ] Add user commands ("skip research", "fresh research")
- [ ] Add progress output ("Research phase: Scout complete...")
- [ ] Document in `/grid:help`
- [ ] Add metrics logging

---

## File Locations Reference

### Agent Definitions
- `/Users/jacweath/grid/agents/grid-scout.md` - Scout agent
- `/Users/jacweath/grid/agents/grid-researcher.md` - Researcher agent
- `/Users/jacweath/grid/agents/grid-planner.md` - Planner agent (receives research)

### Documentation
- `/Users/jacweath/grid/docs/RESEARCH_FIRST.md` - Architecture design
- `/Users/jacweath/grid/docs/RESEARCH_CONFIG.md` - Configuration reference
- `/Users/jacweath/grid/docs/MC_RESEARCH_INTEGRATION.md` - MC integration guide
- `/Users/jacweath/grid/docs/RESEARCH_INFRASTRUCTURE.md` - This file

### State Directories (Local Only - Gitignored)
- `/Users/jacweath/grid/.grid/research/` - Fresh research outputs
- `/Users/jacweath/grid/.grid/research_cache/` - Cached research
- `/Users/jacweath/grid/.grid/scout/` - Scout reports

### Master Control
- `/Users/jacweath/grid/commands/grid/mc.md` - MC command definition

---

## Testing Research Infrastructure

### Manual Test: Scout Spawn

```python
# In MC
scout_prompt = """
First, read ~/.claude/agents/grid-scout.md for your role.

<config>
project_root: /Users/jacweath/grid
timeout_seconds: 120
</config>

Scan the Grid codebase. Write report to .grid/scout/RECON_test.md
"""

Task(prompt=scout_prompt, subagent_type="general-purpose", description="Test Scout")
```

Expected: Scout report in `.grid/scout/RECON_test.md`

### Manual Test: Researcher Spawn

```python
# In MC
researcher_prompt = """
First, read ~/.claude/agents/grid-researcher.md for your role.

<research_request>
mission: technology
topic: Next.js 14 App Router
queries:
  - Next.js 14 App Router best practices 2024 2025
  - Server Components production patterns
</research_request>

Research and write to .grid/research_cache/nextjs-14-app-router.md
Update cache index at .grid/research_cache/index.json
"""

Task(prompt=researcher_prompt, subagent_type="general-purpose", description="Test Researcher")
```

Expected:
- Research output in `.grid/research_cache/nextjs-14-app-router.md`
- Cache index updated with new entry

### Manual Test: Cache Hit

```python
# Second call with same topic should hit cache
cached, needed = check_research_cache([
    {'topic': 'Next.js 14 App Router', 'queries': ['Next.js 14 best practices']}
])

assert len(cached) == 1  # Hit!
assert len(needed) == 0  # Nothing to research
```

---

## Summary

Research-first infrastructure is **FUNCTIONAL**:

1. **Directories created** - `.grid/research/`, `.grid/research_cache/`, `.grid/scout/`
2. **Documentation complete** - READMEs, config schema, integration guide
3. **Agents validated** - Scout and Researcher are spawn-ready
4. **Cache initialized** - Empty index ready for entries
5. **Gitignores added** - State files properly excluded

**Next Step:** MC integration - implement research phase logic in Master Control.

**End of Line.**