import { join } from "node:path"; import type { IndexStore } from "../store/store"; import { embeddingInputText } from "../store/embeddings"; import { cacheHome, configHome } from "../xdg"; import { embedTexts } from "./client"; import { loadOrCreateConfig } from "./config"; import { ensureModelFile } from "./download"; import { computeFingerprint } from "./fingerprint"; import { withEmbeddingServer } from "./process"; import { rankBySimilarity } from "./rank"; import { ensureRuntimeBinary } from "./release"; export class EmbedderManager { static fromEnvironment(): EmbedderManager { return new EmbedderManager( join(configHome(), "tool-query", "embedder.jsonc"), join(configHome(), "tool-query", "embedder.json"), join(cacheHome(), "tool-query", "models"), join(cacheHome(), "tool-query", "runtime"), ); } constructor( private readonly configPath: string, private readonly legacyConfigPath: string, private readonly modelCacheDir: string, private readonly runtimeCacheDir: string, ) {} async prepare(): Promise { const config = await loadOrCreateConfig(this.configPath, this.legacyConfigPath); const modelFile = await ensureModelFile(this.modelCacheDir, config); const runtimeBinary = await ensureRuntimeBinary(this.runtimeCacheDir, config); return { fingerprint: await computeFingerprint(config, runtimeBinary, modelFile), modelFile, runtimeBinary, vulkanDevice: config.vulkanDevice, }; } } export function syncPreparedEmbedder( store: IndexStore, embedder: PreparedEmbedder, ): void { if (store.metaValue("embedder_fingerprint") === embedder.fingerprint) { return; } // Any runtime or model change invalidates future embedding rows because the // vector space and serving binary are part of the effective backend. store.setMetaValue("embedder_fingerprint", embedder.fingerprint); store.setMetaValue("embedder_model_file", embedder.modelFile); store.setMetaValue("embedder_runtime_file", embedder.runtimeBinary); store.setMetaValue("embeddings_state", "stale"); } export async function refreshEmbeddings( store: IndexStore, embedder: PreparedEmbedder, ): Promise { const staleCommands = store.commandsNeedingEmbeddings(embedder.fingerprint); if (staleCommands.length === 0) { store.setMetaValue("embeddings_state", "ready"); return; } await withEmbeddingServer({ runtimeBinary: embedder.runtimeBinary, modelFile: embedder.modelFile, vulkanDevice: embedder.vulkanDevice, run: async (baseUrl): Promise => { const vectors = await embedTexts( baseUrl, staleCommands.map((command) => embeddingInputText(command.name, command.description)), ); staleCommands.forEach((command, index) => { const vector = vectors[index]; if (!vector) { throw new Error(`missing embedding vector for ${command.name}`); } store.upsertEmbedding( command.name, embedder.fingerprint, embeddingInputText(command.name, command.description), vector, ); }); }, }); store.setMetaValue("embeddings_state", "ready"); } export async function searchByEmbedding( store: IndexStore, embedder: PreparedEmbedder, query: string, ): Promise { const commands = store.storedEmbeddings(embedder.fingerprint); if (commands.length === 0) { throw new Error("no stored command embeddings are available"); } return withEmbeddingServer({ runtimeBinary: embedder.runtimeBinary, modelFile: embedder.modelFile, vulkanDevice: embedder.vulkanDevice, run: async (baseUrl): Promise => { const [queryVector] = await embedTexts(baseUrl, [query]); if (!queryVector) { throw new Error("missing query embedding vector"); } return rankBySimilarity(query, queryVector, commands); }, }); } export type PreparedEmbedder = { fingerprint: string; modelFile: string; runtimeBinary: string; vulkanDevice?: string; };