# `@x12i/catalox`

**Catalox** is platform infrastructure for **governed catalogs**: reusable item inventories that apps discover, render, search, reference, and validate — without rebuilding catalog plumbing in every product.

It is **not** a hosted SaaS by itself. It is a **library + CLI + optional HTTP service** you embed or deploy against **MongoDB** (primary persistence), with optional **GCS** for record history payloads and a one-time **Firestore → Mongo** import for migrating estates.

| | |
|---|---|
| **Node** | `>=20` |
| **Current version** | `5.9.10` (monorepo; npm root package `@x12i/catalox`) |
| **License** | MIT |

---

## What Catalox does

- **Catalog discovery** — list catalogs for an `appId`, bootstrap descriptors, app-agnostic search
- **Native catalogs** — items stored in MongoDB (`catalogData-{catalogId}-items`)
- **Mapped catalogs** — items normalized from external MongoDB or HTTP APIs
- **Descriptors** — capabilities, query metadata, identity rules, field metadata (generic UI/BFF consumption)
- **Bindings & authorization** — app↔catalog access, scope lenses, structured outcomes (`ok` / `denied` / `misconfigured`)
- **Operator tooling** — seed manifests, item CRUD CLI, record history, catalog lifecycle, `mongo import-from-firestore`
- **HTTP API + client SDK** (optional) — private internal service (5.9.x) or public Authix-gated API (6.0.0 line)

Catalox **does not** own UI, workflow orchestration, secret storage, or artifact blobs.

**Rule of thumb:** Catalox stores the **definition or selectable item**; your host stores the **transaction, run, or event** that uses it.

→ **Product:** [Overview](./docs/overview.md) · [Apps using Catalox as content backend](./docs/app-content-backend.md)  
→ **Technical:** [Use cases](./docs/USE-CASES.md) · [FAQ](./docs/FAQ.md) · [Deployment](./docs/DEPLOYMENT.md)

---

## Monorepo architecture

```text
                         ┌─────────────────────────────────────┐
                         │  Your app / BFF / catalox-ui        │
                         └──────────────┬──────────────────────┘
                                        │
          ┌─────────────────────────────┼─────────────────────────────┐
          │ embed in-process            │ HTTP                        │ CLI / CI
          ▼                             ▼                             ▼
   @x12i/catalox-engine          catalox-service              catalox (bin)
   createCatalox()               + @x12i/catalox-api           seed, items, mongo…
          │                             │
          │                    @x12i/catalox-client
          │                    @x12i/catalox-authix-bridge (6.0.0)
          ▼                             ▼
   @x12i/catalox-contracts ◄── shared types & outcome helpers
          │
          ▼
   MongoDB · GCS · (optional) aifunctions-js
```

### Domain folders

| Folder | Open together when… | Members |
|--------|---------------------|---------|
| [`packages/core/`](./packages/core/README.md) | Building catalog runtime, stores, seed/operator CLI | `contracts`, `engine`, `cli` |
| [`packages/http/`](./packages/http/README.md) | HTTP routes, OpenAPI, SDK, Authix hooks, running the API | `api`, `client`, `openapi`, `authix-bridge`, `catalox-service` |
| [`packages/management/`](./packages/management/README.md) | Admin UI + management server | `catalox-management` |

### Packages

| Package | npm name | Role |
|---------|----------|------|
| [packages/core/engine](./packages/core/engine/README.md) | `@x12i/catalox-engine` | Catalog engine, Firebase/Mongo stores, CLI binary |
| [packages/core/contracts](./packages/core/contracts/README.md) | `@x12i/catalox-contracts` | Shared TypeScript contracts |
| [packages/http/api](./packages/http/api/README.md) | `@x12i/catalox-api` | Fastify HTTP route layer |
| [packages/http/client](./packages/http/client/README.md) | `@x12i/catalox-client` | HTTP client SDK |
| [packages/http/authix-bridge](./packages/http/authix-bridge/README.md) | `@x12i/catalox-authix-bridge` | Authix token → `CataloxContext` |
| [packages/http/openapi](./packages/http/openapi/README.md) | `@x12i/catalox-openapi` | OpenAPI spec (JSON/YAML) |
| [packages/core/cli](./packages/core/cli/README.md) | `@x12i/catalox-cli` | Thin bin wrapper over engine |
| [packages/http/catalox-service](./packages/http/catalox-service/README.md) | `catalox-service` (private) | Deployable HTTP API |
| [packages/management/catalox-management](./packages/management/catalox-management/README.md) | `catalox-management` (private) | Operator admin console |

The **published root** `@x12i/catalox` re-exports the engine and wires the `catalox` CLI bin for npm consumers.

### Import surfaces (engine)

| Import | Use when |
|--------|----------|
| `@x12i/catalox` | Full surface (embedder + operator) |
| `@x12i/catalox/embedder` | Runtime only: `createCatalox`, `createCataloxFromEnv` (Mongo), `Catalox` |
| `@x12i/catalox/operator` | Markdown, GCS helpers, bindings, validation helpers |
| `@x12i/catalox/mongo` | Mongo primary persistence bootstrap |
| `@x12i/catalox/firebase` | Legacy Firestore Admin helpers for one-time `mongo import-from-firestore` |
| `@x12i/catalox/mapping` | Mapping spec validation and execution |

---

## Choose your integration path

### 1. Library embedder (most common)

Embed the engine in your Node backend or BFF:

```ts
import { createCataloxFromEnv } from "@x12i/catalox/mongo";
// or: import { createCataloxFromEnv } from "@x12i/catalox/embedder";

const { catalox } = await createCataloxFromEnv();
const scoped = catalox.withContext({ appId: "myApp" });
const list = await scoped.listCatalogItems("signals", { limit: 50 });
```

Credentials: `MONGO_URI` (required), `MONGO_DB_NAME` (default **`catalox`**). See [Environment](./docs/environment.md) and [Mongo persistence](./docs/mongo-persistence.md).

### 2. Operator CLI

Provision, diagnose, and operate catalogs without writing code:

```bash
npm run build
npm run cli -- mongo probe
npm run cli -- seed validate --file ./presets/native-map-v1.json
npm run cli -- seed apply --app myApp --preset builtin:native-map-v1 --god
npm run cli -- toolbox check-access --app myApp --catalog signals
```

Or install globally: `npm i -g @x12i/catalox` → `catalox …`

→ [Onboarding happy path](./docs/onboarding-happy-path.md) · [CLI items](./docs/cli-items.md) · [CLI toolbox](./docs/cli-toolbox.md)

### 3. HTTP API service

Run `catalox-service` on a private network (dev/staging) or publicly with Authix (6.0.0):

```bash
export CATALOX_LOCAL_STORE_PATH=./packages/core/engine/test/fixtures/catalox-catalog.local.v1.json
export CATALOX_AUTH_MODE=open
export PORT=3200
npm run dev -w catalox-service
```

→ [Deployment guide](./docs/DEPLOYMENT.md) · [catalox-service README](./packages/http/catalox-service/README.md) · [v6 public API](./docs/catalox-v6/6.0.0/README.md)

### 4. HTTP client SDK

```ts
import { createCataloxClient } from "@x12i/catalox-client";

const client = createCataloxClient({
  baseUrl: "http://localhost:3200",
  getToken: async () => "…", // Authix mode
});
const bound = client.withToken(token);
await bound.listCatalogItems("skills", { limit: 50 });
```

---

## Core concepts (60-second model)

```text
apps/{appId}  ──catalogBindings──►  catalogs/{catalogId}
                                           │
                                           ├── catalogDescriptors/{catalogId}  (capabilities, fields, identity)
                                           ├── catalogDefinitions/{catalogId}  (native vs mapped)
                                           └── catalogData-{catalogId}-items/{itemId}  (native rows)
```

**Minimum for generic consumption:** catalog record + **descriptor** + **binding** (app can see the catalog).

**Generic client flow:**

1. `listAppCatalogs(appId)` — what catalogs exist + access flags
2. `getAppCatalogBootstrap(appId)` — descriptors (capabilities, queryable fields, identity)
3. `listCatalogItems(catalogId, filter)` — `{ listOutcome, items }`
4. `getCatalogItem(catalogId, itemId)` — `{ outcome: found | not_found | mapping_blocked }`

→ [Firestore data model](./docs/firestore-data-model.md) · [Catalog format](./docs/catalog-format.md) · [Authorization](./docs/authorization.md) · [Outcomes](./docs/outcomes.md) · [Identity model](./docs/identity-model.md)

---

## Documentation index

### Start here

| Document | Description |
|----------|-------------|
| [**overview.md**](./docs/overview.md) | **Product:** why Catalox exists, value, maturity, decision guide |
| [**app-content-backend.md**](./docs/app-content-backend.md) | **Recommended pattern:** web apps & agents use Catalox as content/metadata API |
| [**USE-CASES.md**](./docs/USE-CASES.md) | Who uses Catalox, example flows, fit vs misfit |
| [**FAQ.md**](./docs/FAQ.md) | Common questions and troubleshooting |
| [**DEPLOYMENT.md**](./docs/DEPLOYMENT.md) | Library, CLI, Docker, production, Authix |
| [onboarding-happy-path.md](./docs/onboarding-happy-path.md) | Zero → Firestore probe → seed → validate |
| [environment.md](./docs/environment.md) | All env vars (`CATALOX_*`, Firebase, GCS, tests) |

### Data & storage

| Document | Description |
|----------|-------------|
| [firestore-data-model.md](./docs/firestore-data-model.md) | Collections, ids, layout |
| [native-catalog-storage-and-api.md](./docs/native-catalog-storage-and-api.md) | Native items, filters, `indexed` |
| [catalog-format.md](./docs/catalog-format.md) | Public catalog/item shapes |
| [mongo-persistence.md](./docs/mongo-persistence.md) | Mongo as primary backend |
| [local-file-store.md](./docs/local-file-store.md) | Offline read-only snapshots |
| [smart-properties.md](./docs/smart-properties.md) | Descriptor-driven foreign keys |
| [record-history.md](./docs/record-history.md) | Per-item GCS + Firestore history |
| [backup.md](./docs/backup.md) | Firestore/Mongo/GCS backup |
| [firestore-gcs-export.md](./docs/firestore-gcs-export.md) | NDJSON export/import/compare |

### CLI & operations

| Document | Description |
|----------|-------------|
| [cli-items.md](./docs/cli-items.md) | `items validate/get/list/upsert/export…` |
| [cli-toolbox.md](./docs/cli-toolbox.md) | Binding diagnostics and repair |
| [catalog-crud.md](./docs/catalog-crud.md) | Catalog delete/restore/rename |
| [restore-firestore-backup.md](./docs/restore-firestore-backup.md) | Restore with undo |
| [migration-native-catalog-data.md](./docs/migration-native-catalog-data.md) | Legacy → flat native layout |

### UI & presentation (host-owned rendering)

| Document | Description |
|----------|-------------|
| [catalog-custom-ui.md](./docs/catalog-custom-ui.md) | Custom renderers overview |
| [catalog-list-render-map.md](./docs/catalog-list-render-map.md) | List/grid render maps |
| [catalog-item-render-map.md](./docs/catalog-item-render-map.md) | Item render maps |
| [design-objects.md](./docs/design-objects.md) | Scoped design JSON |
| [catalox-ui-contract.md](./docs/catalox-ui-contract.md) | BFF contract for catalox-ui |

### API product line (v6)

| Document | Description |
|----------|-------------|
| [catalox-v6/README.md](./docs/catalox-v6/README.md) | v6 initiative overview |
| [catalox-v6/5.9.5/README.md](./docs/catalox-v6/5.9.5/README.md) | Private internal API |
| [catalox-v6/6.0.0/README.md](./docs/catalox-v6/6.0.0/README.md) | Public Authix-gated API |

### Host integration

| Document | Description |
|----------|-------------|
| [host-canonical-source.md](./docs/host-canonical-source.md) | Design vs runtime catalog sources |
| [schema-packs.md](./docs/schema-packs.md) | Versioned `item.data` shapes |
| [cli-validate-payload.md](./docs/cli-validate-payload.md) | Payload validation in CI |

---

## Install & develop

```bash
git clone <repo>
cd catalox
npm ci
npm run build
npm test
```

**npm consumers:**

```bash
npm install @x12i/catalox
# or workspace packages: @x12i/catalox-engine, @x12i/catalox-contracts, …
```

**Integration tests** (live Firestore — dedicated test project only):

```bash
export FIRESTORE_LIVE_TESTS=1
export FIREBASE_PROJECT_ID=your-test-project
export GOOGLE_SERVICE_ACCOUNT_BASE64=…
npm run test:integration
```

→ [Environment](./docs/environment.md) · [CHANGELOG](./CHANGELOG.md)

---

## Quick API reference (embedder)

```ts
import { createCatalox } from "@x12i/catalox";
import { getFirestore } from "firebase-admin/firestore";

const catalox = createCatalox({ firestore: getFirestore() });
const ctx = { appId: "myApp" };

await catalox.listAppCatalogs(ctx, { appId: "myApp" });
await catalox.getCatalogDescriptor(ctx, "signals");
await catalox.listCatalogItems(ctx, "signals", { filter: { categoryId: "core" } });
await catalox.upsertNativeCatalogItem(ctx, "signals", { categoryId: "core", code: "S1", title: "Example" });

// Opt-in structured errors (never throws):
const bound = catalox.withContext(ctx);
const result = await bound.getCatalogItemResult("signals", "core:S1");
```

**Structured results:** `CataloxResult<T>`, `CataloxClientError` — exported from `@x12i/catalox` and `@x12i/catalox-contracts`.

**Lookup by indexed field:**

```ts
await bound.getCatalogItemByFieldResult("myCatalog", "data.code", "LOGICAL-1");
```

Full embedder reference remains in source JSDoc and [native-catalog-storage-and-api.md](./docs/native-catalog-storage-and-api.md).

---

## Boundaries

- **Secrets** — store refs (`credentialsRef`), not raw keys, in catalog data
- **Artifacts** — descriptor metadata + remote keys; blobs live in object storage
- **UI** — Catalox returns descriptors, render maps, and bindings; your presentation layer renders
- **Auth** — engine enforces bindings + scope; Authix (6.0.0) gates HTTP; SSO stays in your SaaS

---

## Changelog

See [CHANGELOG.md](./CHANGELOG.md). Recent highlights:

- **5.9.8** — `CataloxResult` helpers, `getCatalogItemByField`, monorepo HTTP API packages
- **4.0.x** — `seed validate`, `items` CLI, base64-first Firebase credentials
- **3.1.x** — Record history, catalog lifecycle CRUD