# Task 1887 — Reconcile `allowed-top-level` to the ontology Implementation Plan

> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.

**Goal:** Make `merge()` in `account-schema-owned-dirs.py` rebuild the account's `allowed-top-level` list each run from `fixed-seed ∪ plugin-owned ∪ ontology-buckets`, pruning a stale entry only when its on-disk dir is empty or absent and keeping-and-logging it when it holds files.

**Architecture:** `merge()` currently appends owned+domain dirs into the fenced `allowed-top-level` block and never removes. Replace the append with a reconcile: keep every current entry that is still in the target set, keep a stale entry that holds files (logged as `keptPopulated`), drop a stale entry that is empty/absent (logged as `removed`), then append newly-projected owned/domain dirs as before. A hardcoded fixed-seed set shields base template dirs from pruning.

**Tech Stack:** Python 3 (stdlib only), bash test harness (`__tests__/*.test.sh`).

## Global Constraints

- Fixed seed set (prune-protection allowlist, verbatim from `platform/docs/superpowers/plans/2026-06-23-account-filesystem-schema.md:65-89`): `projects contacts documents url-get output generated extracted uploads agents specialists sites public cache secrets state logs tmp SCHEMA.md account.json AGENTS.md .claude .git .quarantine`. This is 23 entries and includes `url-get` (the task body's inline list dropped it; the operator confirmed the plan-doc list governs).
- The seed set is prune-protection only. `merge()` never *adds* a missing seed entry; seed membership only prevents an entry already in the list from being flagged stale.
- "Holds files" = at least one regular file exists anywhere under `account_dir/<entry>` (recursive), or `<entry>` is a non-empty regular file. Absent path, empty dir, or dir whose whole subtree contains no files → prunable. Strict reading: never silently drop a path that carries any file content.
- Summary line format (single line, extends the existing one): `brand=<b> added=<…|none> domainBuckets=<…|none> removed=<…|none> keptPopulated=<…|none>`.
- Idempotency is mandatory: two consecutive `merge()` runs produce a byte-identical `SCHEMA.md`.
- Out of scope: `account-dir-schema-reconcile.ts` (1877), `fs-schema-guard.sh`, `schema-exposed-dirs.mjs`, and disposition of existing populated strays. Do not touch them.

---

### Task 1: Reconcile in `merge()` + extended log

**Files:**
- Modify: `platform/scripts/lib/account-schema-owned-dirs.py` (add `FIXED_SEED`, add `_dir_holds_files`, rewrite the block-mutation part of `merge()`, extend the summary `print`)
- Test: `platform/scripts/__tests__/account-schema-owned-dirs.test.sh` (append new cases)

**Interfaces:**
- Consumes: existing `resolve(project_dir) -> [{dir, description, plugin}]`, `resolve_domain_buckets(project_dir) -> [{dir, label}]`, `_allowed_block(text) -> (entries, start, end)`, `_read_brand`, `_strip_region`.
- Produces: `FIXED_SEED` (frozenset of 23 str), `_dir_holds_files(account_dir, name) -> bool`, and a `merge()` whose summary line carries `removed=` and `keptPopulated=`.

- [ ] **Step 1: Write the failing tests**

Append to `platform/scripts/__tests__/account-schema-owned-dirs.test.sh`, before the final `echo "PASS=$PASS FAIL=$FAIL"` line. Helper to inject a stray line inside the fenced block:

```bash
# insert_stray <schema_path> <name>: add <name> as the first entry inside the
# allowed-top-level fence (simulates an account whose list carries a bucket no
# longer in the ontology).
insert_stray() {
  awk -v n="$2" '{print} /^```allowed-top-level$/{print n}' "$1" > "$1.tmp" && mv "$1.tmp" "$1"
}

# --- prune: stale bucket, absent on disk -> removed from list + logged ---
A_PRUNE="$(make_vertical_tree '["sitedesk"]' 'schema-construction')"
A_P="$(make_account "$A_PRUNE")"; export PROJECT_DIR="$A_PRUNE/platform"
insert_stray "$A_P/SCHEMA.md" "inbound-invoices"
OUT_P="$(merge_owned_dirs_into_schema "$A_P")"
ALLOWED_P="$(awk '/^```allowed-top-level$/{f=1;next} /^```$/{f=0} f' "$A_P/SCHEMA.md")"
assert_nogrep "inbound-invoices" "$ALLOWED_P" "prune-empty-removed-from-list"
assert_grep "removed=inbound-invoices" "$OUT_P" "prune-empty-logged"

# --- keep: stale bucket that holds files -> retained + keptPopulated ---
A_KEEP="$(make_vertical_tree '["sitedesk"]' 'schema-construction')"
A_K="$(make_account "$A_KEEP")"; export PROJECT_DIR="$A_KEEP/platform"
insert_stray "$A_K/SCHEMA.md" "inbound-invoices"
mkdir -p "$A_K/inbound-invoices"; echo "stamp" > "$A_K/inbound-invoices/stamp_invoice.py"
OUT_K="$(merge_owned_dirs_into_schema "$A_K")"
ALLOWED_K="$(awk '/^```allowed-top-level$/{f=1;next} /^```$/{f=0} f' "$A_K/SCHEMA.md")"
assert_grep "inbound-invoices" "$ALLOWED_K" "keep-populated-retained"
assert_grep "keptPopulated=inbound-invoices" "$OUT_K" "keep-populated-logged"
assert_nogrep "removed=inbound-invoices" "$OUT_K" "keep-populated-not-in-removed"

# --- fixed-seed survival: empty base dir is never pruned ---
A_SEED="$(make_tree '[]')"; A_S="$(make_account "$A_SEED")"; export PROJECT_DIR="$A_SEED/platform"
OUT_S="$(merge_owned_dirs_into_schema "$A_S")"
ALLOWED_S="$(awk '/^```allowed-top-level$/{f=1;next} /^```$/{f=0} f' "$A_S/SCHEMA.md")"
for d in documents url-get .git output; do
  assert_grep "$d" "$ALLOWED_S" "seed-survives-$d"
done
assert_grep "removed=none" "$OUT_S" "seed-nothing-removed"

# --- idempotency after a prune: second run byte-identical ---
B_P="$(shasum "$A_P/SCHEMA.md" | awk '{print $1}')"
merge_owned_dirs_into_schema "$A_P" >/dev/null
assert_eq "$(shasum "$A_P/SCHEMA.md" | awk '{print $1}')" "$B_P" "prune-idempotent"
```

- [ ] **Step 2: Run the suite to verify the new cases fail**

Run: `bash platform/scripts/__tests__/account-schema-owned-dirs.test.sh`
Expected: FAIL — `removed=`/`keptPopulated=` absent from output; `inbound-invoices` still in `ALLOWED_P`.

- [ ] **Step 3: Add `FIXED_SEED` and `_dir_holds_files`**

Insert after the `ONT_END` constant (line 25) region:

```python
FIXED_SEED = frozenset({
    "projects", "contacts", "documents", "url-get", "output", "generated",
    "extracted", "uploads", "agents", "specialists", "sites", "public",
    "cache", "secrets", "state", "logs", "tmp", "SCHEMA.md", "account.json",
    "AGENTS.md", ".claude", ".git", ".quarantine",
})


def _dir_holds_files(account_dir, name):
    """True iff account_dir/<name> carries file content: a non-empty regular
    file, or a directory whose subtree contains at least one regular file.
    Absent path, empty dir, or a dir tree with no files -> False (prunable)."""
    path = os.path.join(account_dir, name)
    if not os.path.exists(path):
        return False
    if os.path.isfile(path):
        return os.path.getsize(path) > 0
    for _root, _dirs, files in os.walk(path):
        if files:
            return True
    return False
```

- [ ] **Step 4: Rewrite the block-mutation section of `merge()`**

Replace the current block (lines 256-270, the `if entries is not None:` … `text = "\n".join(lines)` section) with:

```python
    removed = []
    kept_populated = []
    if entries is not None:
        present = set(entries)
        owned_dirs = {e["dir"] for e in owned}
        domain_dirs = {b["dir"] for b in domain}
        target = FIXED_SEED | owned_dirs | domain_dirs
        # Reconcile existing entries: keep in-target; keep populated strays
        # (logged); drop empty/absent strays (logged).
        new_entries = []
        for entry in entries:
            if entry in target:
                new_entries.append(entry)
            elif _dir_holds_files(account_dir, entry):
                new_entries.append(entry)
                kept_populated.append(entry)
            else:
                removed.append(entry)
        # Append newly-projected owned then domain dirs not already present.
        insert = []
        for e in owned:
            if e["dir"] not in present and e["dir"] not in insert:
                insert.append(e["dir"])
                added.append(e["dir"])
        for b in domain:
            if b["dir"] not in present and b["dir"] not in insert:
                insert.append(b["dir"])
                added_domain.append(b["dir"])
        new_entries.extend(insert)
        lines = text.split("\n")
        lines[start + 1:end] = new_entries
        text = "\n".join(lines)
```

- [ ] **Step 5: Extend the summary `print`**

Replace the final `print(...)` of `merge()` (lines 299-300) with:

```python
    print(f"brand={brand} added={','.join(added) if added else 'none'} "
          f"domainBuckets={','.join(added_domain) if added_domain else 'none'} "
          f"removed={','.join(removed) if removed else 'none'} "
          f"keptPopulated={','.join(kept_populated) if kept_populated else 'none'}")
```

Also update the early-return `print` (line 249, the no-SCHEMA.md branch) to keep the log shape uniform:

```python
        print(f"brand={brand} added=none domainBuckets=none removed=none keptPopulated=none")
```

- [ ] **Step 6: Run the full suite to verify all pass**

Run: `bash platform/scripts/__tests__/account-schema-owned-dirs.test.sh`
Expected: `PASS=<n> FAIL=0` (n = prior 38 + new assertions).

- [ ] **Step 7: Commit**

```bash
git add platform/scripts/lib/account-schema-owned-dirs.py platform/scripts/__tests__/account-schema-owned-dirs.test.sh
git commit -m "feat(admin): reconcile account allowed-top-level to ontology, empty-only prune (Task 1887)"
```

---

### Task 2: Document the projection + empty-only prune rule

**Files:**
- Modify: `.docs/data-portal-folder-index.md` (add a short section stating the reconcile rule)

- [ ] **Step 1: Add the rule to the living reference**

Append a section after the "Three details that fail silently" block explaining: `allowed-top-level` is reconciled each `merge()` to `fixed-seed ∪ plugin-owned ∪ ontology-buckets`; a stale entry (in the list, not in that union) is pruned only when its on-disk dir is empty or absent, and kept-and-logged (`keptPopulated=`) when it holds files, so the write-guard list tracks the ontology without ever orphaning live data into the 1877 quarantine reconcile.

- [ ] **Step 2: Commit**

```bash
git add .docs/data-portal-folder-index.md
git commit -m "docs: account allowed-top-level projection + empty-only prune rule (Task 1887)"
```

---

## Self-Review

- **Spec coverage:** `merge()` reconcile (Task 1 Step 4), empty-only prune (Step 3 `_dir_holds_files` + Step 4), fixed-seed survival (Step 3 `FIXED_SEED` in `target`), logging `removed`/`keptPopulated` (Step 5), tests for prune-empty / keep-populated / seed-survival / idempotency (Step 1), `.docs` update (Task 2). New-ontology-bucket-add and collision cases stay covered by the existing untouched assertions.
- **Placeholder scan:** none — every step carries the literal code or command.
- **Type consistency:** `FIXED_SEED` frozenset of str; `_dir_holds_files(account_dir, name)->bool`; `removed`/`kept_populated` lists of str consumed only by the summary `print`. `owned`/`domain` element shapes match `resolve`/`resolve_domain_buckets` (`e["dir"]`, `b["dir"]`).
- **Regression boundary:** `lines[start+1:end] = new_entries` replaces only the fence body; the wholesale regeneration of the two descriptive regions (lines 272-296) is unchanged, so the em-dash/hyphen separator contract and portal exposure are untouched.