# How Probe Works

Probe combines efficient text search with code intelligence to find relevant code in your codebase.

## System Architecture

Probe operates in six stages:

1. **Scan**: File identification using ripgrep
2. **Parse**: Code structure analysis via Abstract Syntax Trees
3. **Process**: Query enhancement with NLP techniques
4. **Rank**: Result prioritization
5. **Extract**: Code block isolation
6. **Format**: Output generation

## Search Workflow

Here's how a search for "error handling" works:

1. **Query Processing**:
   - Tokenize → [error, handling]
   - Stem → [error, handl]
   - Generate patterns → `\berror\b`, `\bhandl\w*\b`
   - Parse query syntax → AND(error, handling)

2. **File Scanning**:
   - Identify potential matches using ripgrep
   - Filter based on .gitignore and custom patterns
   - Exclude test files (unless specified otherwise)

3. **Code Analysis**:
   - Parse matching files into ASTs
   - Identify complete code blocks containing matches
   - Extract metadata (function names, parameters, etc.)

4. **Result Ranking**:
   - Calculate relevance scores
   - Apply position weights for terms in identifiers
   - Sort by combined relevance metrics
   - Filter based on session cache (if enabled)

5. **Block Extraction**:
   - Extract complete functions/methods with matches
   - Merge related code blocks when appropriate
   - Apply context lines if requested

6. **Result Delivery**:
   - Format with syntax highlighting
   - Apply token/size constraints
   - Generate structured output

## Rapid Scanning

The foundation of Probe's speed:

- **Ripgrep Engine**: Fast line scanning at core
- **Parallel Processing**: Utilizes all CPU cores
- **Smart Filtering**: Respects .gitignore patterns
- **Stream Processing**: Minimal memory footprint
- **Incremental Matching**: Stops scanning when limits are reached

## Code Structure Parsing

Where Probe becomes more than just text search:

- **Tree-sitter**: Industry-standard parsing technology
- **AST Generation**: Builds complete code structure map
- **Language-specific**: Understands each language's unique patterns
- **Robust Handling**: Works with partial or imperfect code
- **Symbol Resolution**: Maps identifiers to declarations

## Session-based Caching

Avoiding duplicate results across multiple searches:

- **Unique Identifiers**: Cache keys based on file path and line numbers
- **Result Tracking**: Remembers which blocks have been shown
- **Session Management**: Generates and maintains session IDs
- **Cache Invalidation**: Clears cache when appropriate

## Output Strategies

Delivering results in the most useful format:

- **Markdown/Syntax**: Rich, readable code presentation
- **JSON**: Structured for programmatic use
- **Token Limiting**: Fits within AI context windows
- **Priority Handling**: Most relevant results survive limits
- **Streaming**: Real-time output for interactive use

## Integration Architecture

How Probe connects with other tools:

### MCP Server

- **STDIO Transport**: Communicates via standard input/output
- **JSON Protocol**: Structured message format
- **Tool Definitions**: Exposes search, query, and extract capabilities

### Node.js SDK

- **JS Bindings**: JavaScript interface to core functionality
- **Vercel AI SDK**: Integration for streaming AI responses
- **Promise-based**: Async/await compatible API
- **Type Definitions**: TypeScript support

### Web Interface

- **Express Backend**: Node.js server for API endpoints
- **Vanilla JS Frontend**: No framework dependencies
- **Streaming Responses**: Real-time AI output
- **Markdown Rendering**: Rich text and code formatting

For detailed information on specific features, see:
- [Search Functionality](search-functionality.md)
- [Code Extraction](code-extraction.md)
- [Feature Overview](features.md)