# Graph top-level label allowlist ontology single-source — Implementation Plan

> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.

**Goal:** Make the `/graph` filter-popover chip allowlist (and the two other server routes that read the same set) derive from each vertical's `## Top-level node types` schema table instead of the hardcoded `FILTER_TOP_LEVEL_LABELS`, so a top-level label added to a vertical schema surfaces as a chip with no TypeScript edit, while preserving today's exact chip visibility.

**Architecture:** A new server-only module `platform/ui/server/lib/top-level-labels.ts` reads every `platform/plugins/memory/references/schema-*.md`, extracts the `Neo4j Label` column of each file's `## Top-level node types` table, and unions those with an explicit `STATIC_TOP_LEVEL_LABELS` const (base-schema infra plus verticals lacking a top-level section). The three server routes (`graph-labels-in-graph`, `graph-subgraph`, `graph-default-view`) call `getTopLevelLabelAllowlist()` instead of importing `FILTER_TOP_LEVEL_LABELS`. Count-gating in the cypher is unchanged.

**Tech Stack:** TypeScript (ES modules), Node `fs` (`readdirSync`/`readFileSync`), Vitest, Hono routes.

## Global Constraints

- Derived union MUST deep-equal today's `FILTER_TOP_LEVEL_LABELS` (48 members) against the real references dir — the byte-identical acceptance baseline.
- No `schema:` namespace filter: every row of a `## Top-level node types` table contributes its `Neo4j Label`, so `WhatsAppGroup` (`cdm:Channel`) stays a chip.
- The `Neo4j Label` column is resolved by header name, never by index (construction table has 5 columns, estate-agent 4).
- The new module is server-only (reads files). `FILTER_TOP_LEVEL_LABELS` stays in `app/lib/graph-labels.ts` for the client `searchHelpers.ts`.
- `getTopLevelLabelAllowlist()` returns a `ReadonlySet<string>` so `.has()` and spread work at every call site.
- Observability line on computation: `op=top-level-labels source=ontology count=<n> files=<...> derived=<d> static=<s>`, written to stderr via `console.error`.
- No em-dashes in any shipped log string. No task numbers in operator-visible strings or log lines.
- Path resolution mirrors `server/index.ts`: `process.env.MAXY_PLATFORM_ROOT ?? join(__dirname, '..')`, then `plugins/memory/references`.

---

### Task 1: The derivation module and its regression test

**Files:**
- Create: `platform/ui/server/lib/top-level-labels.ts`
- Test: `platform/ui/server/lib/top-level-labels.test.ts`

**Interfaces:**
- Produces: `getTopLevelLabelAllowlist(opts?: { referencesDir?: string }): ReadonlySet<string>` and `STATIC_TOP_LEVEL_LABELS: ReadonlySet<string>`.
- Consumes: nothing from earlier tasks. Reads `FILTER_TOP_LEVEL_LABELS` from `../../app/lib/graph-labels` in the test only.

- [ ] **Step 1: Write the failing test**

```ts
// platform/ui/server/lib/top-level-labels.test.ts
import { describe, it, expect } from 'vitest'
import { resolve } from 'node:path'
import { mkdtempSync, writeFileSync, mkdirSync } from 'node:fs'
import { tmpdir } from 'node:os'
import { getTopLevelLabelAllowlist } from './top-level-labels'
import { FILTER_TOP_LEVEL_LABELS } from '../../app/lib/graph-labels'

// The real shipped references directory (four hops up from this test file:
// server/lib -> server -> ui -> platform -> plugins/memory/references).
const REAL_REFERENCES = resolve(
  import.meta.dirname,
  '../../../plugins/memory/references',
)

describe('getTopLevelLabelAllowlist', () => {
  it('deep-equals FILTER_TOP_LEVEL_LABELS against the real references dir', () => {
    const derived = getTopLevelLabelAllowlist({ referencesDir: REAL_REFERENCES })
    expect([...derived].sort()).toEqual([...FILTER_TOP_LEVEL_LABELS].sort())
  })

  it('surfaces a new top-level row with no TypeScript edit', () => {
    const dir = mkdtempSync(resolve(tmpdir(), 'tll-'))
    writeFileSync(
      resolve(dir, 'schema-widgets.md'),
      [
        '# Widgets',
        '',
        '## Top-level node types (operator-entry, natural key)',
        '',
        '| Entity | Neo4j Label | Schema.org Type | Required Properties |',
        '|--------|-------------|-----------------|---------------------|',
        '| Widget | `Widget` | `schema:Product` | `accountId`, `sku` |',
        '',
      ].join('\n'),
    )
    const derived = getTopLevelLabelAllowlist({ referencesDir: dir })
    expect(derived.has('Widget')).toBe(true)
  })

  it('includes a cdm:-namespaced top-level row (no namespace filter)', () => {
    const dir = mkdtempSync(resolve(tmpdir(), 'tll-'))
    writeFileSync(
      resolve(dir, 'schema-transport.md'),
      [
        '## Top-level node types',
        '',
        '| Entity | Neo4j Label | Schema.org Type | Required Properties |',
        '|--------|-------------|-----------------|---------------------|',
        '| Channel | `SomeChannel` | `cdm:Channel` | `accountId`, `id` |',
        '',
      ].join('\n'),
    )
    const derived = getTopLevelLabelAllowlist({ referencesDir: dir })
    expect(derived.has('SomeChannel')).toBe(true)
  })

  it('ignores tables that are not under a Top-level node types heading', () => {
    const dir = mkdtempSync(resolve(tmpdir(), 'tll-'))
    writeFileSync(
      resolve(dir, 'schema-child.md'),
      [
        '## Child node types (reached via parent neighbourhood)',
        '',
        '| Entity | Neo4j Label | Schema.org Type | Required Properties |',
        '|--------|-------------|-----------------|---------------------|',
        '| Line | `QuoteLine` | `schema:Thing` | `accountId`, `id` |',
        '',
      ].join('\n'),
    )
    const derived = getTopLevelLabelAllowlist({ referencesDir: dir })
    // Only the STATIC base labels survive; QuoteLine is a child, not top-level.
    expect(derived.has('QuoteLine')).toBe(false)
    expect(derived.has('Person')).toBe(true)
  })
})
```

- [ ] **Step 2: Run test to verify it fails**

Run: `cd platform/ui && npx vitest run server/lib/top-level-labels.test.ts`
Expected: FAIL — `getTopLevelLabelAllowlist` is not exported / module not found.

- [ ] **Step 3: Write minimal implementation**

```ts
// platform/ui/server/lib/top-level-labels.ts
/**
 * Server-side derivation of the /graph top-level chip allowlist from the
 * per-vertical ontology.
 *
 * The allowlist is the union of two sources:
 *   1. The `Neo4j Label` column of every `## Top-level node types` table in
 *      `plugins/memory/references/schema-*.md`. Today only schema-construction
 *      and schema-estate-agent carry that section; more arrive as verticals
 *      gain one.
 *   2. `STATIC_TOP_LEVEL_LABELS` — base-schema infra (schema-base.md has no
 *      top-level section; its Node Types table mixes in child types) plus any
 *      vertical whose schema has no top-level section yet.
 *
 * No namespace filter: every row contributes its label, so transport labels
 * like WhatsAppGroup (cdm:Channel) stay chips. This differs from the account
 * file-schema generator, which excludes non-schema: rows because a channel
 * has no file bucket. A channel is still a legitimate operator-entry graph node.
 */
import { readdirSync, readFileSync } from 'node:fs'
import { join, resolve } from 'node:path'

/**
 * Top-level labels not (yet) sourced from a vertical `## Top-level node types`
 * section. This set shrinks as verticals gain a top-level section; each such
 * change moves its labels from here into the derived union.
 */
export const STATIC_TOP_LEVEL_LABELS: ReadonlySet<string> = Object.freeze(
  new Set([
    // Base-schema infra (schema-base.md has no top-level section).
    'LocalBusiness',
    'Service',
    'PriceSpecification',
    'OpeningHoursSpecification',
    'Organization',
    'Person',
    'UserProfile',
    'Preference',
    'AdminUser',
    'AccessGrant',
    'KnowledgeDocument',
    'DigitalDocument',
    'CreativeWork',
    'Question',
    'FAQPage',
    'DefinedTerm',
    'Review',
    'ImageObject',
    'Invoice',
    'Task',
    'Project',
    'Event',
    'Workflow',
    'Email',
    'EmailAccount',
    'Agent',
    // Knowledge-work vertical: schema-knowledge-work.md has no top-level
    // section yet.
    'Objective',
    'KeyResult',
    'Decision',
    'Risk',
    'Source',
    'Finding',
    'Hypothesis',
  ]),
) as ReadonlySet<string>

/**
 * Split a pipe-delimited markdown row into trimmed cells, honouring backtick
 * code-spans so a `|` inside `` `A|B` `` does not split the row. Strips the
 * optional leading/trailing pipes.
 */
function splitPipeRow(row: string): string[] {
  let trimmed = row.trim()
  if (trimmed.startsWith('|')) trimmed = trimmed.slice(1)
  if (trimmed.endsWith('|')) trimmed = trimmed.slice(0, -1)
  const cells: string[] = []
  let inBacktick = false
  let start = 0
  for (let i = 0; i < trimmed.length; i++) {
    const ch = trimmed[i]
    if (ch === '`') inBacktick = !inBacktick
    else if (ch === '|' && !inBacktick) {
      cells.push(trimmed.slice(start, i).trim())
      start = i + 1
    }
  }
  cells.push(trimmed.slice(start).trim())
  return cells
}

const stripBackticks = (s: string): string => s.replace(/`/g, '').trim()

/**
 * Extract the `Neo4j Label` column of the first pipe-table beneath a
 * `## Top-level node types` heading. Returns [] when the file has no such
 * section. The heading is matched by case-insensitive prefix so the
 * `(operator-entry, natural key)` suffix is tolerated; parsing stops at the
 * next `## ` heading.
 */
function parseTopLevelLabels(content: string): string[] {
  const lines = content.split(/\r?\n/)
  let i = 0
  // Find the heading.
  while (i < lines.length) {
    const l = lines[i]
    if (l.startsWith('## ') && l.toLowerCase().includes('top-level node types')) break
    i++
  }
  if (i >= lines.length) return []
  i++
  // Find the header row of the first table before the next `## `.
  for (; i < lines.length - 1; i++) {
    const line = lines[i]
    if (line.startsWith('## ')) return []
    if (!line.includes('|')) continue
    const sep = lines[i + 1]
    if (!/^\s*\|?[\s:|-]+\|?\s*$/.test(sep) || !sep.includes('-')) continue
    const headers = splitPipeRow(line)
    const labelCol = headers.findIndex((h) => h.toLowerCase() === 'neo4j label')
    if (labelCol === -1) return []
    const out: string[] = []
    let j = i + 2
    for (; j < lines.length; j++) {
      const row = lines[j]
      if (!row.trim() || !row.includes('|')) break
      const cells = splitPipeRow(row)
      const label = stripBackticks(cells[labelCol] ?? '')
      if (label) out.push(label)
    }
    return out
  }
  return []
}

function resolveReferencesDir(): string {
  // Production: MAXY_PLATFORM_ROOT is `<install>/platform`, and the schema
  // references live at `<platform>/plugins/memory/references` (same root
  // schema-loader.ts resolves brand.json against).
  if (process.env.MAXY_PLATFORM_ROOT) {
    return join(process.env.MAXY_PLATFORM_ROOT, 'plugins', 'memory', 'references')
  }
  // Dev fallback (env unset, unbundled): three hops up from this source file,
  // server/lib -> server -> ui -> platform, then plugins/memory/references.
  return resolve(import.meta.dirname, '../../../plugins/memory/references')
}

let cached: ReadonlySet<string> | null = null

/**
 * The /graph top-level chip allowlist, derived from the ontology. Cached at
 * module scope because schema files are static per install. An injected
 * `referencesDir` (tests) bypasses the cache and does not populate it.
 */
export function getTopLevelLabelAllowlist(
  opts: { referencesDir?: string } = {},
): ReadonlySet<string> {
  if (!opts.referencesDir && cached) return cached

  const referencesDir = opts.referencesDir ?? resolveReferencesDir()
  const fileNames = readdirSync(referencesDir)
    .filter((f) => f.startsWith('schema-') && f.endsWith('.md'))
    .sort()

  const derived = new Set<string>()
  const contributingFiles: string[] = []
  for (const fileName of fileNames) {
    const content = readFileSync(join(referencesDir, fileName), 'utf-8')
    const labels = parseTopLevelLabels(content)
    if (labels.length > 0) contributingFiles.push(fileName)
    for (const label of labels) derived.add(label)
  }

  const derivedCount = derived.size
  for (const label of STATIC_TOP_LEVEL_LABELS) derived.add(label)

  const result = Object.freeze(derived) as ReadonlySet<string>
  process.stderr.write(
    `[graph-page] op=top-level-labels source=ontology count=${derived.size} ` +
      `files=${contributingFiles.join('|') || 'none'} derived=${derivedCount} ` +
      `static=${STATIC_TOP_LEVEL_LABELS.size}\n`,
  )

  if (!opts.referencesDir) cached = result
  return result
}
```

- [ ] **Step 4: Run test to verify it passes**

Run: `cd platform/ui && npx vitest run server/lib/top-level-labels.test.ts`
Expected: PASS — all four tests green. If the deep-equal test fails, the diff names the drifted label(s); reconcile `STATIC_TOP_LEVEL_LABELS` against `FILTER_TOP_LEVEL_LABELS` (do not weaken the assertion).

- [ ] **Step 5: Commit**

```bash
git add platform/ui/server/lib/top-level-labels.ts platform/ui/server/lib/top-level-labels.test.ts
git commit -m "feat: derive /graph top-level chip allowlist from vertical ontology (1624)"
```

---

### Task 2: Point the three server routes at the derived allowlist

**Files:**
- Modify: `platform/ui/server/routes/admin/graph-labels-in-graph.ts`
- Modify: `platform/ui/server/routes/admin/graph-subgraph.ts`
- Modify: `platform/ui/server/routes/admin/graph-default-view.ts`

**Interfaces:**
- Consumes: `getTopLevelLabelAllowlist(): ReadonlySet<string>` from Task 1.
- Produces: nothing new; behaviour of the three routes is unchanged because the derived set deep-equals `FILTER_TOP_LEVEL_LABELS`.

- [ ] **Step 1: Confirm the existing route tests pass before the swap (baseline)**

Run: `cd platform/ui && npx vitest run app/graph/__tests__/labels-in-graph.test.ts server/routes/__tests__/graph-default-topLevel.test.ts`
Expected: PASS. This is the regression baseline the swap must preserve.

- [ ] **Step 2: Swap the popover route**

In `graph-labels-in-graph.ts`, change the import block

```ts
import {
  FILTER_TOP_LEVEL_LABELS,
  AGENT_ACTION_LABELS,
} from '../../../app/lib/graph-labels'
```

to

```ts
import { AGENT_ACTION_LABELS } from '../../../app/lib/graph-labels'
import { getTopLevelLabelAllowlist } from '../../lib/top-level-labels'
```

and change the allowlist construction (currently `new Set<string>(FILTER_TOP_LEVEL_LABELS)`)

```ts
    const allowedSet = new Set<string>(getTopLevelLabelAllowlist())
    if (includeAgentActions) {
      for (const label of AGENT_ACTION_LABELS) allowedSet.add(label)
    }
```

- [ ] **Step 3: Swap the default-view fetch route**

In `graph-subgraph.ts`, remove `FILTER_TOP_LEVEL_LABELS` from the `../../../app/lib/graph-labels` import (keep `HIDDEN_BY_DEFAULT_LABELS` and any others it imports), add `import { getTopLevelLabelAllowlist } from '../../lib/top-level-labels'`, and change line ~280

```ts
  const labels = [...getTopLevelLabelAllowlist()].filter((l) => !HIDDEN_BY_DEFAULT_LABELS.has(l))
```

- [ ] **Step 4: Swap the eligibility route**

In `graph-default-view.ts`, replace

```ts
import { FILTER_TOP_LEVEL_LABELS } from '../../../app/lib/graph-labels'
```

with

```ts
import { getTopLevelLabelAllowlist } from '../../lib/top-level-labels'
```

and change the eligibility check (line ~158)

```ts
    if (!getTopLevelLabelAllowlist().has(lbl)) {
```

Update the adjacent `reason` string to drop the parenthetical `(FILTER_TOP_LEVEL_LABELS)` naming so no internal set name leaks; keep the operator-facing text: `label "${lbl}" is not eligible for default view — outside the chip allowlist`.

- [ ] **Step 5: Run the route tests to verify no behaviour change**

Run: `cd platform/ui && npx vitest run app/graph/__tests__/labels-in-graph.test.ts server/routes/__tests__/graph-default-topLevel.test.ts`
Expected: PASS — the derived set equals `FILTER_TOP_LEVEL_LABELS`, so every assertion still holds.

- [ ] **Step 6: Typecheck the UI package**

Run: `cd platform/ui && npx tsc --noEmit`
Expected: no errors (no unused-import errors for the removed `FILTER_TOP_LEVEL_LABELS`).

- [ ] **Step 7: Commit**

```bash
git add platform/ui/server/routes/admin/graph-labels-in-graph.ts platform/ui/server/routes/admin/graph-subgraph.ts platform/ui/server/routes/admin/graph-default-view.ts
git commit -m "refactor: three /graph server routes read the ontology-derived allowlist (1624)"
```

---

### Task 3: Update the graph-labels doc comment and file the client follow-up

**Files:**
- Modify: `platform/ui/app/lib/graph-labels.ts` (comment only — the set stays for the client)
- Create: `.tasks/NNNN-graph-searchhelpers-derive-top-level-from-ontology.md` (number assigned at land time from LANES)

**Interfaces:**
- Consumes: nothing. Documentation and task-tracking only.

- [ ] **Step 1: Annotate `FILTER_TOP_LEVEL_LABELS` as the client-only mirror**

Add a sentence to the `FILTER_TOP_LEVEL_LABELS` doc block in `graph-labels.ts` recording that the server routes now derive this set from the vertical ontology (`server/lib/top-level-labels.ts`), and that this constant remains only for the client `searchHelpers.ts`, which cannot read files. Do not change the set members.

- [ ] **Step 2: Write the follow-up task file**

Create the task file describing: plumb the ontology-derived allowlist to the client `app/data/searchHelpers.ts` so its `/data` search chip pre-filter stops reading the static `FILTER_TOP_LEVEL_LABELS`, closing the last second source. In scope: deliver the derived set to the client (via the `graph-labels-in-graph` response the page already fetches, or an equivalent). Out of scope: the server routes (done here). Include Testing and Observability sections per the `.tasks` format.

- [ ] **Step 3: Commit**

```bash
git add platform/ui/app/lib/graph-labels.ts .tasks/NNNN-graph-searchhelpers-derive-top-level-from-ontology.md
git commit -m "docs: mark FILTER_TOP_LEVEL_LABELS client-only; file searchHelpers follow-up (1624)"
```

---

## Self-Review

**Spec coverage:**
- New module + derivation → Task 1. ✅
- Three server routes switch → Task 2. ✅
- Byte-identical baseline → Task 1 Step 1 deep-equal test + Task 2 route tests. ✅
- No namespace filter (WhatsAppGroup) → Task 1 Step 1 cdm test + the real-dir deep-equal (which contains WhatsAppGroup). ✅
- Base list explicit → `STATIC_TOP_LEVEL_LABELS` in Task 1. ✅
- Observability line → Task 1 Step 3 `process.stderr.write`. ✅
- Client follow-up filed → Task 3. ✅
- Count-gating unchanged → cypher untouched in Task 2; asserted by the pre-existing route tests. ✅

**Placeholder scan:** The only deferred value is the follow-up task number (`NNNN`), assigned from LANES at archive time per sprint protocol — not a code placeholder.

**Type consistency:** `getTopLevelLabelAllowlist(opts?: { referencesDir?: string }): ReadonlySet<string>` is used identically in Task 1 (definition), Task 1 test, and all three Task 2 call sites. `STATIC_TOP_LEVEL_LABELS` is a `ReadonlySet<string>`. Call sites use `.has()` and spread, both valid on `ReadonlySet`.