# Content index sync

cms-edit keeps a **local SQLite index** per space/environment for fast asset, media, and **content catalog** discovery. The default index is built from the **Content Preview API** (draft content) — matching cms-edit’s draft-only workflow.

## What gets indexed

| Table | Content |
|-------|---------|
| `assets` | All CDN assets (filename, title, MIME, dimensions) |
| `media` | Media wrapper entries linked to assets |
| `reference_entries` | Content catalog metadata (no content trees) |

Catalog and media pages use Contentful `select` + `include=0` so only index fields are fetched.
That keeps responses under Contentful’s ~7 MB limit on large spaces (e.g. many articles with rich text).

### Catalog content types (schema v3)

Typically fewer than ~1000 rows each: `page`, `article`, `pageVariant`, `template`, `articleType`, `tagType`, `tag`, `navigation`, `navigationItem`.

Each catalog row stores:

| Column | Purpose |
|--------|---------|
| `slug` | Normalized URL slug when the type has `slug` |
| `label` | List line: `title`, `name`, or template cmsLabel |
| `cmsLabel` | `cmsLabel` when present (page, article, pageVariant, tag, template) |
| `tagTypeSlug` | Parent tag type slug (tags only) |
| `articleTypeSlug` | Parent article type slug (articles only) |
| `date` | Publication date `YYYY-MM-DD` (articles only; schema v4) |

## When you need it

These commands **require** a built index:

- `cms-edit asset search` (title, `--filename`, `--filename-match`)
- `cms-edit asset audit`
- `cms-edit list --type media --asset-id <id>`
- `cms-edit list --type media --asset-filename …` / `--asset-filename-match …`
- `cms-edit asset upload --if-exists-by-filename` (lookup uses the index)
- `*AssetFilename` keys in `create from-json`

Catalog commands **prefer** the index when present (CMA fallback if a row is missing):

- `cms-edit list --type page|article|pageVariant|template|articleType|tagType|tag|navigation|navigationItem`
- `cms-edit open`, `cms-edit resolve`, `cms-edit nav open` — see [lookup.md](./lookup.md)
- `cms-edit create … --if-not-exists` for taxonomy
- `tagTypeSlug` / `tagTypeName` shorthands in JSON create flows

For `--type template`, `--slug` filters by **cmsLabel** (not URL slug).

Use `--force-cma` on `list` to bypass the catalog and paginate via CMA.

## Commands

```bash
# Full rebuild (draft/preview content — default)
cms-edit index sync

# Progress on stderr (per-page fetches, catalog types, SQLite write)
cms-edit index sync --verbose
# Same via env (also applies to automatic background sync):
LOG_CMS_INDEX=1 cms-edit index sync

# Check freshness and row counts
cms-edit index status

# Browse index contents in the terminal
cms-edit index dump
cms-edit index dump --all
cms-edit index dump --type image/ -n 20
cms-edit index dump --assets-only --skip 50
cms-edit index dump --reference-only
cms-edit index dump --reference-type page

# Published content only (separate database file suffix -published)
cms-edit index sync --published
cms-edit index status --published
```

Index path: `~/.contentful-cms/index/<spaceId>/<environment>/media.db` (add `-published` suffix for published-only mode).

## Automatic sync (simplicity)

Commands that depend on the index call **`ensureContentIndex`** internally. You do not need to run `index sync` first in normal agent workflows.

| Condition | Auto-sync behaviour |
|-----------|---------------------|
| Index missing | **Hydrate** from shared cache blob if valid, else **full** rebuild |
| Schema version outdated (e.g. v2 → v3) | **Full** rebuild |
| Committed write invalidated index (incremental) | **Incremental** upsert on next index command |
| Committed write invalidated index (full — slug change or catalog delete) | **Full** rebuild on next index command |
| Last sync older than **24 hours** | **Incremental** upsert (`sys.updatedAt[gte]` last `syncedAt`) |
| Index fresh | No sync |

Manual `cms-edit index sync` always performs a **full** rebuild.

**Invalidation (lazy, no thrashing):** `save`, `create page|article|…`, `asset upload`, and similar **committed** writes mark the index dirty (Redis on hosted MCP; `invalidation.json` locally). They do **not** sync immediately. The next `list`, `open`, `asset search`, etc. refreshes the index. In-session `set` / `rtf` edits do **not** invalidate.

**Shared cache (hosted MCP):** After sync, the SQLite file is gzipped into Redis (~sub-MB for typical spaces). Cold serverless instances restore from Redis instead of re-fetching all assets from the Preview API.

**Incremental limitations:** Only rows changed since the last sync are fetched. Entries deleted in Contentful may remain in the index until the next **full** invalidation/sync. Duplicate slug/label/cmsLabel rows in the index cause lookup commands to fail with an explicit error (fix in Contentful, then full sync).

**Single writer:** File lock beside `media.db` plus a shared Redis lock on hosted MCP. If another sync is in progress, the command **fails immediately** (exit code 1). Stale file locks (>30 min or dead pid) are removed automatically.

## Preview / delivery API token

The management token cannot call the CDN. For default `index sync`:

| Config field | Purpose |
|--------------|---------|
| `previewAccessToken` | Explicit Preview API token (`${CONTENTFUL_PREVIEW_ACCESS_TOKEN}`) |
| *(auto)* | Linked preview key from a space delivery key via CMA |

For `index sync --published`:

| Config field | Purpose |
|--------------|---------|
| `deliveryAccessToken` | Explicit CDA token (`${CONTENTFUL_ACCESS_TOKEN}`) |
| `deliveryApiKeyName` | Pick a space delivery key by name via CMA |
| *(auto)* | First delivery key matching the configured environment |
| `autoCreateDeliveryApiKey` | Create `cms-edit-index` key if none match (needs PAT permissions; warns when used) |

If CMA cannot read API keys, set `previewAccessToken` or `deliveryAccessToken` manually.

## Typical workflow before imports

```bash
cms-edit index sync
cms-edit asset search --filename-match 'istock.*'
cms-edit list --type page
cms-edit list --type article --article-type-name "Publications"
cms-edit resolve --tag-type-slug topic
cms-edit open --page-slug /about
```

See [lookup.md](./lookup.md) for all `open` / `resolve` flags. See [MEDIA_INDEX_PLAN.md](../MEDIA_INDEX_PLAN.md) for architecture details.