# fabric-harness Data Connector Spec This document is the contract for building a fabric-harness data connector. A data connector exposes content from an external system (Notion, Postgres, Confluence, S3, Google Drive, your internal CMS) so a fabric-harness agent can mount it as filesystem-shaped context (`FilesystemSource`) or use it as a custom data adapter. If you are an AI coding agent reading this to build a connector for a user, follow this document literally and produce a single TypeScript file that exports a factory function returning a `FilesystemSource`. --- ## When to use which shape - **`FilesystemSource`** (preferred) — the source maps cleanly to a tree of files (Notion pages, Confluence pages, GitHub markdown, S3 keys). Files become mountable into the agent's context with the existing `withFilesystemSources` helper. - **Custom data adapter** — the source is row-oriented (Postgres, BigQuery) or graph-shaped (Linear issue graph). Build a tool-shaped connector instead and follow the MCP spec. This document covers the `FilesystemSource` shape. For row/graph data, return tools and follow `mcp.md`. --- ## High-Level Shape ```ts // .fabricharness/connectors/.ts import type { FilesystemSource, FilesystemEntry, MountedSource } from '@fabric-harness/sdk'; export interface ProviderDataOptions { apiKey: string; /** Restrict to a subset of the provider tree (e.g. Notion database id). */ rootId?: string; } export function provider(options: ProviderDataOptions): FilesystemSource { return { name: 'provider', async list(prefix?: string): Promise { // Return entries with absolute virtual paths (e.g. `/notion/page-1.md`) // and `kind: 'file' | 'directory'`. }, async read(path: string): Promise { // Fetch the file content; return UTF-8 text. }, }; } ``` Mount it on the agent: ```ts import { withFilesystemSources } from '@fabric-harness/sdk'; import { provider } from './connectors/provider.js'; const fabric = await init(); await withFilesystemSources(fabric, [provider({ apiKey: process.env.PROVIDER_API_KEY! })]); ``` --- ## Imports You Will Use All from `@fabric-harness/sdk`: - `FilesystemSource` — the interface you implement. - `FilesystemEntry` — `{ path: string; kind: 'file' | 'directory'; size?: number; mtimeMs?: number }`. - `MountedSource` — what the host gets after mounting (for diagnostics). - `withFilesystemSources(fabric, sources)` — host-side helper for mounting. --- ## Required Methods | Method | Signature | Notes | |---|---|---| | `list` | `(prefix?: string) => Promise` | Return entries under `prefix` (or root when undefined). Paths are absolute virtual paths starting with the source name (`/notion/...`). | | `read` | `(path: string) => Promise` | UTF-8 text of the file. Throw on missing/binary. | ### Optional Methods - `name` (string) — display name for `fh inspect`. Defaults to the factory function name. - `stat(path)` — return `{ size, mtimeMs }` without fetching content. Used for change detection. - `watch(callback)` — push notifications when content changes. If absent, fabric polls on a schedule. --- ## Pagination and Caching External systems often paginate. Inside `list()`: - Fetch all pages internally if the source is bounded (< ~1000 entries). - Throw with a clear "source too large; restrict via options.rootId" if the call would download more than ~10 MB. - Cache results in memory for the lifetime of the connector (one factory call = one cache). `read()` should be idempotent and cheap to retry. --- ## Auth - API keys, OAuth tokens, DSN strings — all flow through `options`. No global env reads inside the connector. - Sensitive data must NEVER appear in `FilesystemEntry.path` (paths land in session history). If a path includes a secret-bearing id, hash or truncate it. --- ## Worked Example (Notion) ```ts // .fabricharness/connectors/notion.ts import type { FilesystemSource, FilesystemEntry } from '@fabric-harness/sdk'; import { Client } from '@notionhq/client'; export interface NotionSourceOptions { apiKey: string; databaseId: string; } export function notion(options: NotionSourceOptions): FilesystemSource { const client = new Client({ auth: options.apiKey }); return { name: 'notion', async list(prefix?: string): Promise { const pages = await client.databases.query({ database_id: options.databaseId }); return pages.results.map((page) => ({ path: `/notion/${page.id}.md`, kind: 'file' as const, mtimeMs: new Date((page as { last_edited_time?: string }).last_edited_time ?? 0).getTime(), })); }, async read(path: string): Promise { const id = path.replace(/^\/notion\//, '').replace(/\.md$/, ''); const blocks = await client.blocks.children.list({ block_id: id }); return blocks.results.map(blockToMarkdown).join('\n\n'); }, }; } function blockToMarkdown(block: unknown): string { // Provider-specific formatting... return ''; } ``` --- ## Checklist Before Submitting - [ ] Single TypeScript file exporting the factory function. - [ ] Returns a `FilesystemSource` (not a class). - [ ] All paths returned by `list()` are absolute and start with `//`. - [ ] No secrets in returned paths. - [ ] `read()` returns UTF-8 text (not binary). - [ ] Pagination is handled inside the connector. - [ ] No hardcoded API keys. - [ ] Imports only from `@fabric-harness/sdk` and the provider's official SDK. If the provider's content is truly binary (PDFs, images), document this in a top-of-file comment and direct the user toward the artifact-store pattern (`packages/connectors/src/s3.ts`) instead.