# @composio/embeddings

TurboPuffer schema definition for the Composio tool lookup system.

## Overview

This package defines the TurboPuffer schema for storing and searching tools using:

- **Vector embeddings**: 3072-dimensional vectors from OpenAI text-embedding-3-large
- **BM25 full-text search**: On formatted tool descriptions
- **Filterable attributes**: Toolkit names, tool names, display names

All types extend the official [@turbopuffer/turbopuffer](https://www.npmjs.com/package/@turbopuffer/turbopuffer) SDK types.

## Exports

- `TOOL_SCHEMA`: TurboPuffer schema configuration using `AttributeSchema` type
- `TOOL_DISTANCE_METRIC`: Distance metric for vector similarity (cosine_distance)
- `ToolDocumentSchema`: TypeScript type for tool documents
- `ToolRow`: Query result type extending TurboPuffer's `Row`

### Re-exported TurboPuffer Types

- `Row`, `Vector`, `DistanceMetric`
- `AttributeSchema`, `AttributeSchemaConfig`
- `NamespaceQueryParams`, `NamespaceQueryResponse`
- `NamespaceWriteParams`, `NamespaceWriteResponse`
- `Turbopuffer` client

## Tool Document Schema

`TOOL_SCHEMA` is the single source of truth. All TypeScript types are derived from it.

```typescript
import type { ToolDocumentSchema } from '@composio/embeddings'
import { TOOL_SCHEMA } from '@composio/embeddings'

// Schema definition (source of truth)
TOOL_SCHEMA = {
  text: { type: 'string', full_text_search: true },
  tool_name: { type: 'string', filterable: true },
  display_name: { type: 'string', filterable: true },
  description: { type: 'string' },
  toolkit_name: { type: 'string', filterable: true },
  created_at: { type: 'string' },
}

// TypeScript type (automatically derived from TOOL_SCHEMA)
type ToolDocumentSchema = {
  id: string // Random 10-char identifier
  vector: Vector // 3072-dimensional embedding
  text: string // Formatted text (BM25 searchable)
  tool_name: string // Tool slug (filterable)
  display_name: string // Human-readable name (filterable)
  description: string // Tool description
  toolkit_name: string // App/toolkit name (filterable)
  created_at: string // ISO timestamp
}
```

## Usage

```typescript
import {
  TOOL_DISTANCE_METRIC,
  TOOL_SCHEMA,
  Turbopuffer,
} from '@composio/embeddings'

// Use schema when writing to TurboPuffer
const tpuf = new Turbopuffer({ apiKey: process.env.TURBOPUFFER_API_TOKEN })
const namespace = tpuf.namespace('tool-lookup-v3')

await namespace.write({
  upsert_rows: [...tools],
  schema: TOOL_SCHEMA,
  distance_metric: TOOL_DISTANCE_METRIC,
})
```

## Design Context & Resources

### Slack Discussions
- [TurboPuffer implementation](https://composioworkspace.slack.com/archives/C09HRALMGJZ/p1766484691042469) - Jayesh's update on TurboPuffer and Weaviate vector store implementations
- [Weaviate integration](https://composioworkspace.slack.com/archives/C09HRALMGJZ/p1766498105126319) - Testing Weaviate implementation for vector store interface

### Notion Documentation
- [Embedding Model Comparison for Workflow Cache System](https://www.notion.so/Embedding-Model-Comparison-for-Workflow-Cache-System-295f261a6dfe817081edca039410a261) - Comparison of embedding models for the workflow cache
- [Tool embedding (non-cached path)](https://www.notion.so/Tool-embedding-non-cached-path-293f261a6dfe8065895bfc75d346cd6c) - Experiment on tool embedding non-cached search path (Rahul Tarak, P1)

### Related Documentation
- See `CLAUDE.md` in this directory for AI assistant context
- See `/apps/apollo` for tool search API implementation