# KERNL MCP - Persistent Intelligence Layer

**Transform AI assistants from stateless to persistent, project-aware intelligence systems**

[![License: MIT](https://img.shields.io/badge/License-MIT-blue.svg)](LICENSE)
[![Version](https://img.shields.io/badge/version-5.0.0-green.svg)](package.json)
[![V2.0](https://img.shields.io/badge/V2.0-COMPLETE-brightgreen.svg)](docs/V2_COMPLETE.md)
[![Tools](https://img.shields.io/badge/tools-75-blue.svg)](docs/TOOL_REFERENCE.md)

**Formerly "Project Mind" — Now KERNL: The authoritative system control layer for AI**

---

## 🎯 What is KERNL?

KERNL is an MCP (Model Context Protocol) server that solves three fundamental problems with AI assistants:

### The Problems
1. **The 8-Minute Death** 💀 - Operations >8 minutes cause crashes and total context loss
2. **The Bootstrap Tax** 🔄 - Starting fresh every session wastes time re-explaining context
3. **Isolated Learning** 🏝️ - Knowledge doesn't transfer between projects

### The Solution
**KERNL V2.0** - A persistent intelligence layer with 75 specialized tools:

✅ **Session Recovery** - Automatic checkpoints survive crashes  
✅ **Context Preservation** - Zero bootstrap tax with automatic resume  
✅ **Cross-Project Learning** - Patterns compound across projects  
✅ **Full System Control** - Complete desktop authority (100% Desktop Commander parity)  
✅ **Self-Documenting** - Introspection and auto-generated documentation  
✅ **Production Ready** - Comprehensive testing and validation  

---

## ⭐ V2.0 Revolutionary Features

### 1. 🛡️ Crash Recovery
**No more 8-minute death!**
```typescript
// Automatic checkpoints every 5-10 tool calls
auto_checkpoint({ project: "my-app", progress: 0.5 })

// Claude crashes...

// Next session automatically resumes
check_resume_needed({ project: "my-app" })
// → "You were refactoring auth (50% complete). Resume?"
```

### 2. 🧠 Semantic Search
**Find files by meaning, not keywords**
```typescript
search_semantic({
  project: "my-app",
  query: "authentication logic",
  limit: 10
})
// → Returns JWT validation, middleware, login routes
```

### 3. 🔗 Cross-Project Learning
**Solutions transfer automatically**
```typescript
suggest_patterns({
  currentProblem: "large markdown files slow to query"
})
// → "Hybrid Source of Truth pattern (85% confidence)"
```

### 4. 🎛️ Full System Control
**Complete desktop authority**
```typescript
// Interactive Python REPL
sys_start_process({ command: "python3 -i" })
sys_interact_with_process({ pid: 1234, input: "import pandas" })

// PDF generation
sys_write_pdf({ path: "report.pdf", content: "# Report..." })

// File operations anywhere
sys_copy_path({ source: "D:/file.txt", dest: "C:/backup/" })
```

### 5. ✅ Self-Documenting
**Auto-generated documentation**
```typescript
sys_generate_docs({
  format: 'markdown',
  output: 'D:/TOOLS.md'
})
// → 75 tools documented automatically
```

---

## 📊 What's Included (75 Tools)

### 🔄 Session Management (5 tools)
Survive crashes and preserve context
- `check_resume_needed` - Detect incomplete work
- `auto_checkpoint` - Automatic progress saves
- `mark_complete` - Clear resume state
- `get_session_state` - Manual state check
- `save_session_state` - Manual checkpoint

### 📁 Project Management (3 tools)
Multi-tenant project registry
- `pm_register_project` - Register projects
- `pm_list_projects` - List all projects
- `pm_get_project` - Get project details

### 📄 Filesystem Operations (6 tools)
Format-aware file operations
- `pm_read_file` - Read (Excel/PDF/Images/Text)
- `pm_write_file` - Write with format detection
- `pm_search_files` - Find files in projects
- `pm_batch_read` - Read multiple files
- `pm_list_files` - List directory contents
- `pm_get_file_info` - Get metadata + line counts

### 🧠 Intelligence Features (3 tools)
AI-powered semantic capabilities
- `search_semantic` - Find by meaning (ONNX embeddings)
- `suggest_patterns` - Cross-project solutions
- `pm_index_files` - Index for semantic search

### 📋 Backlog Management (4 tools)
Epic tracking and project status
- `query_backlog` - List EPICs with filtering
- `add_epic` - Create new EPICs
- `complete_epic` - Mark EPICs done
- `get_project_status` - Project overview

### 🔀 Git Integration (2 tools)
Smart commits with auto-generated messages
- `smart_commit` - Conventional commit generation
- `session_package` - Package session for commit

### 🔬 Research Capabilities (2 tools)
Progressive research with citations
- `research_progressive` - Streaming research
- `search_research` - Search history

### 💾 Conversation Export (3 tools)
Export Claude.ai conversation history
- `chrome_export_status` - Check Chrome readiness
- `chrome_export_setup` - Setup instructions
- `chrome_export_conversations` - Export conversations

### ⚙️ Process Control (7 tools)
Interactive REPL and process management
- `sys_start_process` - Start with smart detection
- `sys_interact_with_process` - Send input
- `sys_read_process_output` - Read output
- `sys_list_sessions` - List active sessions
- `sys_list_processes` - List system processes
- `sys_kill_process` - Terminate process
- `sys_force_terminate` - Force terminate session

### 🔍 Advanced Search (4 tools)
Background search with streaming
- `sys_start_search` - Start background search
- `sys_get_more_search_results` - Paginate results
- `sys_stop_search` - Stop search
- `sys_list_searches` - List active searches

### 📂 System Files (5 tools)
Low-level file system operations
- `sys_copy_path` - Copy files/directories
- `sys_delete_path` - Delete files/directories
- `sys_path_exists` - Fast existence check
- `sys_move_path` - Move/rename
- `sys_create_directory` - Create directories

### ⚙️ Configuration & Meta (4 tools)
System configuration and introspection
- `sys_get_config` - Get KERNL configuration
- `sys_set_config_value` - Update configuration
- `sys_get_usage_stats` - Tool usage analytics
- `sys_get_tool_info` - Tool discovery

### ✅ Testing Suite (4 tools)
Automated validation and monitoring
- `sys_run_tests` - Run test suites
- `sys_validate_tools` - Validate tool definitions
- `sys_check_health` - System health check
- `sys_benchmark` - Performance benchmarks

### 🔗 Integration Tools (3 tools)
Documentation and export
- `sys_export_tools` - Export tool registry (JSON)
- `sys_generate_docs` - Auto-generate docs
- `sys_get_version` - Version information

### 🎛️ System Control (3 tools)
Revolutionary desktop control
- `sys_edit_block` - Surgical file editing
- `sys_write_pdf` - PDF generation/modification
- `sys_copy_file_user_to_claude` - Bridge filesystems

### ⏱️ Background Jobs (14 tools)
Long-running operations with progress tracking
- Job creation, monitoring, and management
- Progress tracking and status updates
- Background indexing and research
- Session state persistence

**Total: 75 tools across 17 categories**

---

## 🚀 Quick Start

### Prerequisites
- Node.js 18+ (for MCP server)
- Claude Desktop app
- Windows, macOS, or Linux

### Installation

#### Option 1: NPM (Recommended)
```bash
npm install -g kernl-mcp
```

#### Option 2: From Source
```bash
git clone https://github.com/yourusername/kernl-mcp.git
cd kernl-mcp
npm install
npm run build
```

### Configuration

Add to your Claude Desktop configuration:

**Windows:** `%APPDATA%\Claude\claude_desktop_config.json`  
**macOS:** `~/Library/Application Support/Claude/claude_desktop_config.json`  
**Linux:** `~/.config/Claude/claude_desktop_config.json`

```json
{
  "mcpServers": {
    "kernl": {
      "command": "node",
      "args": [
        "C:/Users/YourName/AppData/Roaming/npm/node_modules/kernl-mcp/dist/index.js"
      ]
    }
  }
}
```

**For source installation**, use your local path:
```json
{
  "mcpServers": {
    "kernl": {
      "command": "node",
      "args": [
        "D:/Project Mind/kernl-mcp/dist/index.js"
      ]
    }
  }
}
```

### Verification

1. **Restart Claude Desktop**

2. **Test connection:**
```
Check if KERNL is connected
```

Claude should confirm connection and show available tools.

3. **Get version:**
```
Use sys_get_version to show KERNL version
```

Should return version 5.0.0 (V2.0).

---

## 📖 Core Workflows

### Basic Project Setup
```typescript
// 1. Register your project
pm_register_project({
  id: "my-app",
  name: "My Application",
  path: "C:/Projects/my-app"
})

// 2. Index files for semantic search
pm_index_files({ project: "my-app" })

// 3. Check project status
get_project_status({ project: "my-app" })
```

### Crash Recovery Workflow
```typescript
// At start of EVERY session
check_resume_needed({ project: "my-app" })
// → If resume needed, shows prompt

// During work - checkpoint every 5-10 tool calls
auto_checkpoint({
  project: "my-app",
  operation: "Building feature X",
  progress: 0.6,
  currentStep: "Implementing Y",
  decisions: ["Use approach Z"],
  nextSteps: ["Add tests", "Update docs"],
  activeFiles: ["src/feature.ts"]
})

// When task complete
mark_complete({
  project: "my-app",
  summary: "Feature X complete with tests"
})
```

### Semantic Search Workflow
```typescript
// Find files by meaning
search_semantic({
  project: "my-app",
  query: "database connection setup",
  topN: 5
})

// Get patterns from other projects
suggest_patterns({
  currentProblem: "need connection pooling"
})
```

### Interactive Development Workflow
```typescript
// Start Python REPL for data analysis
sys_start_process({ command: "python3 -i", timeout_ms: 30000 })

// Load data
sys_interact_with_process({
  pid: 1234,
  input: "import pandas as pd; df = pd.read_csv('data.csv')"
})

// Analyze
sys_interact_with_process({
  pid: 1234,
  input: "print(df.describe())"
})

// Read results
sys_read_process_output({ pid: 1234, timeout_ms: 5000 })
```

### Documentation Generation
```typescript
// Generate complete tool documentation
sys_generate_docs({
  format: 'markdown',
  output: 'D:/docs/TOOLS.md',
  includeExamples: true
})

// Export tool registry
sys_export_tools({
  format: 'full',
  output: 'D:/tools.json'
})

// Check system health
sys_check_health({ quick: false })
```

---

## 🎯 Use Cases

### 1. Long-Running Refactoring
**Problem:** Multi-hour refactoring exceeds Claude's timeout

**Solution:**
```typescript
// Claude automatically checkpoints every 5-10 operations
// If crash occurs, next session resumes exactly where left off
check_resume_needed({ project: "my-app" })
// → "Refactoring auth system (75% complete). Resume?"
```

### 2. Multi-Project Development
**Problem:** Need to find "how did I solve X in other project?"

**Solution:**
```typescript
suggest_patterns({
  currentProblem: "implementing rate limiting"
})
// → Returns rate limiting patterns from all projects
```

### 3. Data Analysis
**Problem:** Need to analyze CSV with pandas

**Solution:**
```typescript
// Start Python REPL
sys_start_process({ command: "python3 -i" })

// Interactive analysis
sys_interact_with_process({
  pid: 1234,
  input: "import pandas as pd; df = pd.read_csv('sales.csv')"
})
// → Full pandas workflow with persistent REPL
```

### 4. System Integration
**Problem:** Need to work with files across filesystems

**Solution:**
```typescript
// Copy from user's computer to Claude's computer
sys_copy_file_user_to_claude({
  path: "D:/user-data/large-file.xlsx"
})

// Process in Claude's environment
// Copy results back via sys_copy_path
```

### 5. Quality Assurance
**Problem:** Need to validate system health

**Solution:**
```typescript
// Run comprehensive tests
sys_run_tests({ category: 'all', verbose: true })

// Validate all tools
sys_validate_tools({ verbose: true })

// Check system health
sys_check_health({ quick: false })

// Run benchmarks
sys_benchmark({ operation: 'all', iterations: 100 })
```

---

## 🏗️ Architecture

### Three-Layer Design

#### Layer 1: Persistence
**Foundation for crash recovery**
- SQLite database for state
- Session management
- File indexing
- Pattern storage
- Research history

#### Layer 2: System Control
**Complete desktop authority**
- File operations
- Process management
- Search capabilities
- System integration

#### Layer 3: Intelligence
**AI-powered features**
- Semantic search (ONNX)
- Pattern recognition
- Smart commits
- Auto-documentation

### Key Components

**Database:** SQLite with WAL mode  
**Embeddings:** ONNX Runtime (local, no API calls)  
**Search:** Background execution with pagination  
**Processes:** Smart REPL detection and management  
**Testing:** Comprehensive validation suite  

---

## 📊 Performance

### Benchmarks (V2.0)
```
Database Write:    400 ops/sec
Database Read:     800 ops/sec
Semantic Search:   <500ms (10,000 files)
File Operations:   <100ms
Process Launch:    <50ms
Health Check:      <20ms (cached)
```

### Scalability
- **Projects:** 50+ simultaneous
- **Files:** 100,000+ per project
- **Patterns:** 1,000+ with clustering
- **Processes:** Unlimited concurrent
- **Search Results:** Streaming pagination

---

## 🧪 Testing

```bash
# Run all tests
npm test

# Run test suite via tool
sys_run_tests({ category: 'all', verbose: true })

# Validate tool definitions
sys_validate_tools({ verbose: true })

# Check system health
sys_check_health({ quick: false })

# Run performance benchmarks
sys_benchmark({ operation: 'all', iterations: 100 })
```

### Test Categories
- **smoke** - Quick validation (2 tests)
- **database** - SQLite operations
- **filesystem** - File operations
- **tools** - Tool handlers
- **config** - Configuration
- **integration** - End-to-end

---

## 🔧 Configuration

### Project Configuration
```typescript
{
  fileScanning: {
    enabled: true,
    mode: "auto",  // "manual" | "auto" | "watcher"
    excludes: ["**/node_modules/**", "**/.git/**"]
  },
  git: {
    enabled: true,
    autoCommit: false
  },
  features: {
    semanticSearch: true,
    crashRecovery: true,
    crossProjectLearning: true
  },
  limits: {
    maxFileSize: 10485760,  // 10MB
    maxSearchResults: 100
  }
}
```

### System Configuration
```typescript
// Get current config
sys_get_config()

// Update config
sys_set_config_value({
  key: 'features.semanticSearch',
  value: true
})

// Get usage statistics
sys_get_usage_stats()
```

---

## 📚 Documentation

### Complete References
- **[V2_COMPLETE.md](docs/V2_COMPLETE.md)** - Complete V2.0 overview (824 lines)
- **[TOOL_REFERENCE.md](docs/TOOL_REFERENCE.md)** - All 75 tools with examples
- **[CHANGELOG.md](CHANGELOG.md)** - Version history
- **[Phase Completion Docs](docs/v2-absorption/)** - Detailed phase documentation

### Auto-Generated Docs
Use `sys_generate_docs` to create:
- Markdown documentation
- HTML documentation
- JSON tool registry export

---

## 🐛 Troubleshooting

### MCP Connection Issues
**Problem:** Claude doesn't see KERNL tools

**Solutions:**
1. Verify Node.js: `node --version` (need 18+)
2. Check config path in claude_desktop_config.json
3. Rebuild: `npm run build`
4. Check logs: `%APPDATA%\Claude\logs\mcp.log`
5. Restart Claude Desktop completely

### Session Recovery Not Working
**Problem:** check_resume_needed returns false

**Solutions:**
1. Verify checkpoint was saved: Check database
2. Ensure mark_complete wasn't called early
3. Check database path: `sys_get_config()`

### Performance Issues
**Problem:** Slow semantic search

**Solutions:**
1. Check index status: `pm_index_status({ project })`
2. Reindex: `pm_index_files({ project, reindex: true })`
3. Run benchmark: `sys_benchmark({ operation: 'all' })`
4. Check health: `sys_check_health()`

---

## 🗺️ Roadmap

### ✅ V2.0 (COMPLETE - Jan 2026)
- ✅ 75 tools across 17 categories
- ✅ 100% Desktop Commander parity
- ✅ Self-documenting system
- ✅ Comprehensive testing
- ✅ Revolutionary features

### V2.1 (Q1 2026)
- [ ] Web UI for project management
- [ ] Real-time file watching
- [ ] Enhanced pattern visualization
- [ ] Additional format support

### V2.2 (Q2 2026)
- [ ] Multi-platform support (ChatGPT, Gemini)
- [ ] Cloud sync for session state
- [ ] Team collaboration features
- [ ] Advanced analytics

### V3.0 (Q3 2026)
- [ ] Distributed pattern learning
- [ ] Enterprise features (SSO, audit)
- [ ] Advanced security
- [ ] Plugin ecosystem

---

## 🤝 Contributing

We welcome contributions! Areas of interest:
- Additional file format support
- New intelligence features
- Performance optimizations
- Platform integrations
- Documentation improvements

See [CONTRIBUTING.md](CONTRIBUTING.md) for guidelines.

---

## 📄 License

MIT License - see [LICENSE](LICENSE)

---

## 🙏 Acknowledgments

Built with:
- [Model Context Protocol](https://modelcontextprotocol.io/) - AI integration framework
- [TypeScript](https://www.typescriptlang.org/) - Type-safe development
- [SQLite](https://www.sqlite.org/) - Embedded database
- [ONNX Runtime](https://onnxruntime.ai/) - Local AI inference

Inspired by:
- Desktop Commander - System control patterns
- The need for truly persistent AI intelligence
- Developers who believe AI should remember

---

## 📞 Support

- **Documentation:** [docs/](docs/)
- **Issues:** [GitHub Issues](https://github.com/yourusername/kernl-mcp/issues)
- **Discussions:** [GitHub Discussions](https://github.com/yourusername/kernl-mcp/discussions)
- **Tool Reference:** [TOOL_REFERENCE.md](docs/TOOL_REFERENCE.md)

---

## 🌟 Star the Project

If KERNL helps you, consider starring the repo! ⭐

---

**KERNL V2.0** - Transforming AI from stateless to persistent intelligence

**Made with ❤️ by developers who refuse to accept the 8-minute death**

```
From: Stateless AI Assistant
To: Persistent Intelligence Layer
Status: COMPLETE ✅
```