# Code Sage

A high-performance **MCP (Model Context Protocol) server** for semantic code search, written in Rust.

## Features

- **Hybrid Search**: Combines BM25 (keyword-based) + Vector embeddings (semantic) with RRF reranking
- **AST-Based Chunking**: Uses tree-sitter to intelligently split code into semantic units (functions, classes, methods)
  - **Character-based fallback**: For files where AST parsing fails or isn't available
  - **Comprehensive language support**: 60+ file extensions supported out of the box
- **Smart File Filtering**: 
  - Automatic .gitignore support (respects .gitignore, .ignore, .git/info/exclude)
  - Custom file extensions support for project-specific file types
  - No configuration needed - works out of the box
- **Embedded Storage**: Zero external dependencies - all data stored locally
  - USearch for vector similarity search
  - Tantivy for BM25 full-text search
  - Sled for metadata storage
- **Multiple Embedding Providers**:
  - **Builtin (Default)** - Integrated llama.cpp with nomic-embed-text-v1.5, zero external dependencies
  - OpenAI (text-embedding-3-small, text-embedding-3-large)
  - Ollama (local embeddings)
- **MCP Compatible**: Works with Claude Desktop, Cursor, and other MCP clients
- **Multi-Language Support**:
  - **Programming Languages (AST)**: Rust, Python, JavaScript/TypeScript, Java, C/C++, Go, C#, Swift, Kotlin, Ruby, Elixir, Objective-C, PHP, Scala
  - **Config/Markup (AST)**: JSON, YAML, XML, HTML, CSS, SCSS, TOML, Markdown
  - **iOS/macOS**: .xib, .storyboard, .plist (via XML parser), .xcconfig (via TOML parser)
  - **Android/Java**: .xml (layouts, manifests), .gradle, .properties
  - **Build Systems**: .cmake, .sbt, .make, Makefile, CMakeLists.txt
  - **Shell Scripts**: .sh, .bash, .zsh, .fish
  - **Character-based fallback**: .ini, .txt, .rst, and any extension added via `custom_extensions`

## Architecture

See [ARCHITECTURE.md](./ARCHITECTURE.md) for detailed architecture documentation.

**Key Design Decisions:**

1. **Hybrid Search over Pure Semantic**: Combines keyword and semantic search for better results
2. **Embedded over Client-Server**: Everything runs locally, no vector DB server needed
3. **AST-First with Fallback**: Semantic chunking when possible, character-based when needed
4. **Rust for Performance**: Efficient memory usage and fast processing

## Installation

### Prerequisites

- Rust 1.70+ (edition 2021)
- No external dependencies required for builtin provider

### Build from source

```bash
git clone https://github.com/faxioman/code-sage.git
cd code-sage
cargo build --release
```

The binary will be in `target/release/code-sage`

### Platform-Specific Builds

Code Sage automatically detects the best GPU acceleration for your platform:

**macOS (Apple Silicon):**
```bash
cargo build --release --features metal
```

**Linux/Windows with NVIDIA GPUs:**
```bash
cargo build --release --features cuda
```

**CPU-only (universal compatibility):**
```bash
cargo build --release --no-default-features
```

**Default build:**
The default `gpu-acceleration` feature enables platform detection but doesn't include any specific GPU backend. Use the platform-specific features above for optimal performance.

## Usage

### Zero-Setup Configuration

Add to your MCP client configuration (e.g., Claude Desktop):

```json
{
  "mcpServers": {
    "code-sage": {
      "command": "/path/to/code-sage"
    }
  }
}
```

That's it! Code Sage will:
- Use the built-in nomic-embed-text-v1.5 model automatically
- Store data in `~/.code-sage/` (created automatically)
- Download the model on first use (79MB)
- Work immediately with GPU acceleration when available

### Advanced Configuration

#### Custom Data Directory

To override the default `~/.code-sage/` location:

```json
{
  "mcpServers": {
    "code-sage": {
      "command": "/path/to/code-sage",
      "env": {
        "DATA_DIR": "/custom/path/to/data"
      }
    }
  }
}
```

#### OpenAI (Cloud-based)

If you prefer cloud embeddings instead of the built-in local model:

```json
{
  "mcpServers": {
    "code-sage": {
      "command": "/path/to/code-sage",
      "env": {
        "EMBEDDING_PROVIDER": "openai",
        "OPENAI_API_KEY": "your-openai-api-key",
        "EMBEDDING_MODEL": "text-embedding-3-small"
      }
    }
  }
}
```

#### Ollama (Local)

If you already have Ollama running:

```json
{
  "mcpServers": {
    "code-sage": {
      "command": "/path/to/code-sage",
      "env": {
        "EMBEDDING_PROVIDER": "ollama",
        "EMBEDDING_MODEL": "nomic-embed-text",
        "EMBEDDING_BASE_URL": "http://localhost:11434"
      }
    }
  }
}
```

### Advanced Parameters

Optional parameters can be added to the `env` section:

```json
{
  "mcpServers": {
    "code-sage": {
      "command": "/path/to/code-sage",
      "env": {
        "EMBEDDING_PROVIDER": "openai",
        "OPENAI_API_KEY": "sk-your-key-here",
        "EMBEDDING_MODEL": "text-embedding-3-small",
        "DATA_DIR": "./data",
        "DEFAULT_TOP_K": "10",
        "MIN_SCORE": "0.3",
        "RRF_K": "100",
        "CHUNK_SIZE": "2500",
        "CHUNK_OVERLAP": "300",
        "BATCH_SIZE": "100",
        "MAX_CHUNKS": "450000"
      }
    }
  }
}
```

### Available MCP Tools

#### 1. `analyze_code`

Create a searchable index of your code by analyzing functions, classes, and methods:

```json
{
  "path": "/absolute/path/to/codebase",
  "force": false,
  "splitter": "ast",
  "custom_extensions": [".proto", ".sql"],
  "ignore_patterns": ["*.test.ts", "tmp/*"]
}
```

**Parameters**:
- `path` (required): Absolute path to codebase directory
- `force` (optional): Force re-analysis if already analyzed (default: false)
- `splitter` (optional): Chunking strategy - "ast" or "langchain" (default: "ast")
- `custom_extensions` (optional): Additional file extensions to analyze beyond the 60+ defaults (e.g., [".proto", ".graphql"])
- `ignore_patterns` (optional): Additional patterns to ignore (complements .gitignore)

**How File Selection Works**:
1. **Extension Filtering**: Only files with supported extensions are analyzed (60+ defaults)
2. **Gitignore Respecting**: Automatically respects `.gitignore`, `.ignore`, and `.git/info/exclude`
3. **Custom Extensions**: Use `custom_extensions` to add project-specific file types not in defaults
4. **Hidden Files**: Skipped by default

**Supported Extensions by Default** (60+ total):
- **Core Languages**: .rs, .py, .js, .jsx, .ts, .tsx, .java, .c, .h, .cpp, .hpp, .go, .cs, .swift, .kt, .rb, .ex, .exs, .m, .mm, .php, .scala
- **JS/TS Variants**: .mjs, .cjs
- **Config Formats**: .json, .yaml, .yml, .toml, .xml, .ini
- **iOS/macOS**: .xib, .storyboard, .plist, .xcconfig
- **Android/Java**: .gradle, .properties
- **Build Systems**: .cmake, .sbt, .make
- **Web/Styling**: .html, .htm, .css, .scss, .sass, .less
- **Shell Scripts**: .sh, .bash, .zsh, .fish
- **.NET**: .csproj, .sln, .config, .props, .targets
- **Ruby**: .gemspec, .rake
- **Elixir**: .ex, .exs
- **Docs**: .md, .markdown, .txt, .rst
- **Notebooks**: .ipynb

**Example - Adding Custom Extensions**:
```json
{
  "path": "/path/to/project",
  "custom_extensions": [".proto", ".graphql", ".vue", ".svelte"]
}
```

**Returns**: JSON with success/error message

#### 2. `find_code`

Find code using natural language questions:

```json
{
  "path": "/absolute/path/to/codebase",
  "query": "authentication logic",
  "limit": 10,
  "extension_filter": [".ts", ".js"]
}
```

**Returns**: JSON with search results and formatted code snippets

#### 3. `delete_index`

Delete the search index for a codebase:

```json
{
  "path": "/absolute/path/to/codebase"
}
```

**Returns**: JSON with confirmation message

#### 4. `check_status`

Check if code analysis is complete, in progress, or failed:

```json
{
  "path": "/absolute/path/to/codebase"
}
```

**Returns**: JSON with status (analyzed, analyzing with %, failed, or not found)


## How It Works

### 1. Indexing Pipeline

```
Code Files
    ↓
AST Parsing (tree-sitter)
    ↓
Semantic Chunks (functions, classes)
    ↓
Embeddings (Builtin/OpenAI/Ollama)
    ↓
Storage (USearch + Tantivy + Sled)
```

### 2. Hybrid Search

```
Query
    ↓
    ├─→ Vector Search (USearch) → Top 50 results
    │
    └─→ BM25 Search (Tantivy) → Top 50 results
    
    ↓
RRF Reranking (merge with k=100)
    ↓
Final Results (Top K)
```

**RRF (Reciprocal Rank Fusion)**:
The hybrid search uses RRF reranking to balance results from vector and BM25 searches based on their ranking positions rather than raw scores. This creates a fair and balanced final ranking that combines semantic relevance with keyword matching. [Learn more about RRF](https://milvus.io/docs/it/rrf-ranker.md)

The RRF formula combines rankings using: `score = 1/(k + rank)` where `k` is a smoothing parameter (default: 100, configurable via `RRF_K` environment variable).

### Development Setup

```bash
# Install dependencies
cargo build

# Run tests
cargo test

# Run with logging
RUST_LOG=debug cargo run

# Format code
cargo fmt

# Lint
cargo clippy
```

## Inspiration & Credits

This project is inspired by:
- [claude-context](https://github.com/zilliztech/claude-context) - Original TypeScript implementation
- Design decisions around hybrid search and AST chunking
- MCP protocol implementation patterns

**Key Differences:**
- Written in Rust for performance
- Embedded storage (no Milvus/Qdrant server needed)
- Simplified architecture
- Native binary (easier deployment)
- Simpler handler responses (JSON strings)

## License

MIT License - see [LICENSE](./LICENSE)

## Known Issues & Limitations

- **File Size Limit**: 1MB per file
- **Extension Filtering**: Files must have a supported extension or be added via `custom_extensions` to be analyzed
- **Storage**: No compression yet (working on it)
- **Switching Providers**: When changing embedding providers with different dimensions (e.g., from OpenAI 1536 to LM Studio 768), **delete the `data/` folder** before re-indexing to avoid dimension mismatch errors


## Support

- Issues: [GitHub Issues](https://github.com/faxioman/code-sage/issues)
- Discussions: [GitHub Discussions](https://github.com/faxioman/code-sage/discussions)

---

**Built with ❤️ in Rust** 🦀