A pgpm workspace for building agentic, database-driven AI applications for the City of San Francisco.
# @sf-ai/rag-indexes
RAG vector index helpers for HNSW and IVFFlat.
## Overview
This package provides SQL functions for creating and managing pgvector indexes on RAG embeddings.
## Functions
### `rag.create_hnsw_index(collection_id, model_name, m, ef_construction, distance_op)`
Create an HNSW index for fast approximate nearest neighbor search.
```sql
-- Create HNSW index with defaults (m=16, ef_construction=64, cosine distance)
SELECT rag.create_hnsw_index('collection-uuid');
-- Create with custom parameters
SELECT rag.create_hnsw_index(
'collection-uuid',
'text-embedding-3-small',
32, -- m: connections per layer
128, -- ef_construction: index build quality
'l2' -- distance: cosine, l2, or ip
);
```
**Parameters:**
- `m`: Max connections per layer (default: 16). Higher = better recall, more memory
- `ef_construction`: Build-time quality (default: 64). Higher = better recall, slower build
- `distance_op`: Distance function - `cosine`, `l2`/`euclidean`, or `ip`/`inner_product`
### `rag.create_ivfflat_index(collection_id, model_name, lists, distance_op)`
Create an IVFFlat index for fast indexing with good recall.
```sql
-- Create IVFFlat index with defaults (lists=100, cosine distance)
SELECT rag.create_ivfflat_index('collection-uuid');
-- Create with custom parameters
SELECT rag.create_ivfflat_index(
'collection-uuid',
'text-embedding-3-small',
200, -- lists: number of clusters
'cosine' -- distance: cosine, l2, or ip
);
```
**Note:** IVFFlat requires data to be present before creating the index.
### `rag.list_vector_indexes(collection_id)`
List all vector indexes, optionally filtered by collection.
```sql
SELECT * FROM rag.list_vector_indexes();
SELECT * FROM rag.list_vector_indexes('collection-uuid');
```
### `rag.drop_vector_index(index_name)`
Drop a vector index by name.
```sql
SELECT rag.drop_vector_index('idx_rag_embedding_hnsw_...');
```
## When to Use Which Index
| Index | Best For | Trade-offs |
|-------|----------|------------|
| HNSW | High recall, easy management | Slower inserts, more memory |
| IVFFlat | Fast indexing, large datasets | Requires pre-populated data |
## Dependencies
- `@sf-ai/rag-core`
## License
[SF License](https://github.com/city-of-san-francisco/license)