--- name: document-ingest description: Universal document ingestion — maps any unstructured document (PDF, text, transcript, web page) OR chat archive (WhatsApp `_chat.txt`) to ontologically-grounded graph nodes. The dispatched specialist reads the cached extract and produces typed-section JSON in-turn from the loaded ontology, then calls memory-ingest (the separate server-side memory-classify step was removed). Triggers when the operator uploads or fetches a document or chat export for ingestion. One skill for every input shape — no per-doctype, no per-channel branching. --- # Document Ingest **Invoked from `specialists:librarian`** — the admin agent dispatches the librarian when the operator hands over a document or chat archive, and the librarian loads this skill. The admin agent never calls this skill directly. **Default parent.** Every `:KnowledgeDocument` (whether the document or the conversation-document writer wrote it) is anchored to a `:Project` when the dispatch brief names one, or to the account's `:LocalBusiness` when the document is account-scoped rather than project-scoped. The admin agent's brief carries the parent elementId; `memory-ingest` refuses the write if the parent is missing or does not exist. Brief-named entity wiring (Persons, Organizations, Tasks) creates cross-hierarchy edges from the document — never substitutes for the containment parent. Ingests any unstructured input — documents (PDF, text, transcript, web page) and chat archives (WhatsApp `_chat.txt`) — into the graph. Every classified section becomes one `:Section` node under a `:KnowledgeDocument` parent. **One parent label, two writer paths chosen by identity property:** | Input shape | Parent label | Identity property (dispatcher signal) | Section secondary label | mode | |---|---|---|---|---| | PDF / text / web (default) | `:KnowledgeDocument` | `attachmentId` | `:Section:` from closed enumeration (`Position`, `Chapter`, `Parties`, …) | `document` | | Chat archive (`_chat.txt`, X DMs, Telegram, …) | `:KnowledgeDocument` | `conversationIdentity` | `:Section` (no secondary label — source distinguishes at the parent) | `chat` | | X (Twitter) tweet-stream transcript (rendered by `x-import`) | `:KnowledgeDocument` (`source='x'`) | `attachmentId` | `:Section:` from closed enumeration | `document` | **You** (the dispatched specialist) decide which section kinds each section maps to (document mode) or chunk the archive into topic-bounded `:Section` nodes (chat mode). The skill drives the pipeline; you read the loaded ontology directly and produce the typed JSON in your turn; the writer enforces the validator. **Classification failure is terminal — the ingest aborts entirely; nothing is written. Loud failures, never silent landfill.** ## Routing — chat vs document (mandatory first decision) Before anchor confirmation, decide which identity property to pass to `memory-ingest`: - **Chat archive** — input filename ends in `_chat.txt`, the dispatch brief names the input as a WhatsApp chat / messaging-channel export, or the operator labels it as such. Set `mode='chat'` and pass `conversationIdentity` (omit `attachmentId`). Skip anchor confirmation; run participant confirmation instead (see § Participant confirmation). The classifier produces `:Section` chunks; no anchor edges, no related entities. - **Document** — everything else. Set `mode='document'` and pass `attachmentId` (omit `conversationIdentity`). Run the anchor confirmation flow below. Both branches go through the same two tools (`memory-ingest-extract` → `memory-ingest`) with you producing typed-section JSON in between; only the parameters differ. ## Anchor confirmation (mandatory first step) Every document has a subject — the node every identity-kind section attaches to. Anchors are parameter input, never inferred from "who uploaded the file". The dispatch brief from the admin agent names the anchor explicitly. If the brief is ambiguous, ask the operator before any read. The four common shapes: | Document | Anchor | Anchor label | |---|---|---| | Owner CV / personal-profile content | `:UserProfile` (or `:Person {role: 'admin-personal'}`) | `UserProfile` (or `Person`) | | Business pricing guide / policy / brochure | `:LocalBusiness` for this account | `LocalBusiness` | | Contract or document about a third party | `:Person` or `:Organization` (named in brief) | `Person` or `Organization` | | Reference document with no natural subject (industry overview, generic FAQ) | `:LocalBusiness` (default) | `LocalBusiness` | The confirmation flow: 1. Read the dispatch brief. Extract the document path and the named subject. 2. If the subject is not named, or is ambiguous, ask the operator in one sentence: "I'm about to ingest ``. Anchor it to the account owner's UserProfile, the business (``), or a third party — which?". Wait for the answer. 3. Run a one-shot graph read to resolve the anchor's element ID. For UserProfile: `MATCH (u:UserProfile {accountId: $accountId}) RETURN elementId(u) AS anchorId, 'UserProfile' AS anchorLabel`. For LocalBusiness: `MATCH (b:LocalBusiness {accountId: $accountId}) RETURN elementId(b) AS anchorId, 'LocalBusiness' AS anchorLabel`. For a third party: search by name via `memory-search` and pick the matching node. 4. Persist `$anchorNodeId` and `$anchorLabel` for the rest of the run. They flow into your in-turn classification (as the anchorDescription you reason against) and into `memory-ingest` (as the `anchorNodeId` + `anchorLabel` parameters). ## Participant confirmation (chat mode only) Chat archives are multi-party — no single subject anchor. Instead, every distinct sender name in the archive must resolve to an existing `:AdminUser` or `:Person` elementId before any classify or ingest call. No auto-creation; missing participants are blockers, not silent skips. The confirmation flow: 1. Read the dispatch brief. Extract the archive path and any operator-stated participant identities. 2. Read a small sample of the archive (head ~50 lines) via the `Read` tool to discover the distinct sender names that appear at line starts after the bracketed-timestamp prefix. 3. For every distinct senderName, search the graph via `memory-search` (or a one-shot `MATCH (n) WHERE (n:Person OR n:AdminUser) AND n.accountId = $accountId AND (n.name = $name OR (n.givenName + ' ' + n.familyName) = $name) RETURN elementId(n)`). 4. **Resolved fully** — every senderName mapped to exactly one elementId. Capture the owner's elementId (the operator who exported the archive — usually the `:AdminUser` for this account; ask if ambiguous) and the comma-separated list of remaining participant elementIds. 5. **Unresolved** — at least one senderName has no matching node. Surface to the operator: *"Archive `` mentions sender `` but no `:Person` / `:AdminUser` matches. Create the contact first or correct the name, then re-dispatch."* Do NOT proceed. 6. **Ambiguous** — a senderName matches multiple nodes. Ask the operator which one. Do NOT proceed until disambiguated. Persist `$ownerElementId` and `$participantElementIds` (array, owner excluded) for the run. These flow into `memory-ingest` as the `participantElementIds` parameter (owner + others, deduped). Compute archive metadata before classify: - `archiveSha256` — `bash sha256sum "" | cut -d' ' -f1`. Stamped on the parent + every chunk. - `archiveSourceFile` — the basename (e.g. `_chat.txt`). - `conversationIdentity` — pass as the `conversationIdentity` parameter to `memory-ingest` (in place of `attachmentId`; the two are mutually exclusive). Format: `chat:` where `sortedParticipantElementIds` is the sorted-then-comma-joined list of `[owner, ...participants]`. Same conversation across re-exports → same identity → idempotent MERGE on the conversation-document `:KnowledgeDocument`. ## Pipeline Four steps in order. Step 1 is a deterministic tool call. Step 2 is **your in-turn work**: you read the cached text plus the loaded ontology and produce typed-section JSON. Step 3 hands that JSON to `memory-ingest`. Step 4 is agent-driven graph writes against the existing graph, gated by the dispatch brief's named entity list. Ontology validation stays server-side in the `memory-ingest` writer (the writer rejects sections whose `kind` is outside the loaded label set). ``` EXTRACT --> CLASSIFY (in-turn) --> INGEST --> WIRE-BRIEF-ENTITIES │ │ │ │ v v v v text sections one :Section KD-level edges from cached + KD-level per semantic the new KnowledgeDocument by id edges boundary + to each entity the + orphans NEXT chain + dispatch brief named (you produce anchor edges (Person / Organization / this JSON + related Service / Task / Event / yourself, from entities + KnowledgeDocument / the ontology + typed-edge BrandingData) — plus an the cached text) KD-level orphan list for brief- edges named entities not found (PARTY / in the graph. PARTICIPANT / FROM/TO/CC / SPEAKER) ``` ### 1. `memory-ingest-extract` Pulls text from the file and caches it under the `attachmentId`. Inputs: `storagePath`, `filename`, `mimeType`, `attachmentId` (all from the attachment metadata block in your brief). Returns metadata + a 240-char preview. The full text lives in the in-process cache. After extract returns, emit one line in chat: "Ingesting `` (`` chars)." Documents > ~525K chars need chunked processing in step 2 — flag this in the line so the operator knows the turn will run long. ### 2. Classify in-turn (your work) You produce typed-section JSON yourself from the cached text and the loaded ontology. There is no classifier tool — it was deleted. Read the relevant ontology references first (`platform/plugins/memory/references/schema-base.md` and, when `brand.json#vertical` is set, the matching `schema-.md`) so every `kind` you emit is a label the writer will accept. **Document mode (default).** Build the JSON with this top-level shape: ``` { "documentSummary": "<1-3 sentences>", "documentKeywords": ["<3-10 lowercase topic keywords>"], "sections": [ { "kind": "", "title": "", "summary": "<≤500 chars>", "sourceStart": , "sourceEnd": , "properties": { ... }, "anchorEdge": { "type": "", "direction": "from-anchor", "properties": { ... } }, "related": [ ... ], "classifierReason": "" } ], "documentEdges": [ ... ], "orphanCandidates": [ ... ] } ``` Use the natural-edge map below to pick `anchorEdge` per kind. The writer derives `:Section.body` from `cachedText.slice(sourceStart, sourceEnd)` — you emit offsets, not body text. Any `body` field you supply is logged and ignored. `summary` ≤ 500 chars. ``` Section kind | Anchor edge (direction = from-anchor unless noted) Position (identity) | (anchor:UserProfile)-[:HAS_POSITION]->(:Section:Position); section -[:AT]-> related:Organization (merge:true) Education (identity) | (anchor:UserProfile)-[:ATTENDED {degree, startDate, endDate}]->(:Section:Education); section -[:ATTENDED]-> related:Organization {organizationCategory:'educational'} (merge:true) Credential (identity) | (anchor:UserProfile)-[:HOLDS]->(:Section:Credential) Skill (identity) | (anchor:UserProfile)-[:HAS_SKILL]->(:Section:Skill) Biography (identity) | (anchor:UserProfile)-[:DESCRIBED_BY]->(:Section:Biography) Project (standalone, not :Section) | (anchor:UserProfile)-[:CREATED]->(:Project); optional (:Project)-[:UNDER]->related:Organization (merge:true) Preface, Abstract, Introduction, TableOfContents, Chapter, Conclusion, Appendix, Bibliography, Glossary, Acknowledgments (structural) | anchorEdge: null — anchored only via (:KnowledgeDocument)-[:HAS_SECTION]->(:Section:) + (:Section)-[:NEXT]->(:Section) Parties, Recitals, Definitions, Scope, Term, Payment, Confidentiality, IntellectualProperty, Warranties, Indemnification, Liability, Termination, GoverningLaw, ForceMajeure, Notices, EntireAgreement, Amendment, Assignment, Severability, Signatures (contract-clause) | anchorEdge: null — anchored via HAS_SECTION + NEXT; for Parties section, additionally emit top-level documentEdges PARTY entries; for Definitions, emit related DefinedTerm nodes with edge type DEFINES (direction outgoing from the Definitions section). Other (label fallback) | anchorEdge: null — section-kind closure miss; you MUST also include 'classifierReason' (one-line about-what) ``` `documentEdges` covers party-like links the document itself names: `PARTY` for contract parties, `PARTICIPANT` for meeting/call attendees, `FROM`/`TO`/`CC` for email-header parties, `SPEAKER` for transcript voices. The `memory-ingest` tool definition spells out the exact field shape. **Typed-entity edges target real entities, never `:Section`.** Edge types governed by the typed-edge allowlist — `FOUNDED`, `WORKS_AT`, `INVESTED_IN`, `ADVISES`, `MENTIONS`, `AUTHORED`, `ATTACHED_TO`, `REFERENCES`, `ATTENDED` — connect real entities (`Person`, `Organization`, etc.) per the (source, target) shapes declared in `platform/plugins/memory/mcp/src/lib/typed-edge-schema.ts`. They MUST NOT land on a `:Section` node as either endpoint. The writer rejects loud with the offending `(:Source)-[:EDGE]->(:Target)` triple if a classifier emission violates this. When a section names entities related to the **document anchor** (e.g. a Team section on a company-page document whose anchor is the `:Organization`), set `edge.targetMode = "anchor"` on the related entry. The writer connects the related entity to the anchor instead of the section node: ``` // Company website, anchor = :Organization "BioSymm Technologies Ltd" { kind: "Other", title: "Team", anchorEdge: null, related: [ { kind: "Person", properties: { name: "Ricky Du Plessis" }, merge: true, edge: { type: "FOUNDED", direction: "incoming", targetMode: "anchor" } }, { kind: "Person", properties: { name: "Clarence Bissessar" }, merge: true, edge: { type: "FOUNDED", direction: "incoming", targetMode: "anchor" } } ]} // Writer creates: // (:Person {name:"Ricky Du Plessis"})-[:FOUNDED]->(:Organization {anchor}) // (:Person {name:"Clarence Bissessar"})-[:FOUNDED]->(:Organization {anchor}) ``` The default `targetMode` is `"section"` (omit when the edge legitimately lives on the section, e.g. `:Section:Definitions -[:DEFINES]-> :DefinedTerm`). Anchor-routing is only meaningful when the document anchor is the natural endpoint per the typed-edge allowlist — emitting `targetMode: "anchor"` on a CV whose anchor is `:UserProfile` for a `:FOUNDED` edge is still a rejected shape (`:UserProfile` is not a valid FOUNDED target). `orphanCandidates` is the list of entities you wanted to emit but could not natural-edge. Surface them so the ontology can grow. **Chat mode.** Same top-level shape with every `sections[].kind = "Conversation"`. `summary` is the per-section top-level field (already required by the schema) — the writer reads it from there and stamps `:Section.summary`. The `properties` bag is for non-authoritative per-chunk metadata only: `keywords`, `firstMessageAt`, `lastMessageAt`, `participantNames`, `messageCount`. The writer strips reserved writer-owned keys (`summary`, `title`, `body`, `bodyPreview`, `embedding`, provenance fields, etc.) from `properties` and logs the attempt — so duplicating an authoritative field into `properties` is a noisy no-op. Cover every message in chronological order with no gaps. Drop `anchorEdge` (always null) and `related`. Use the participant elementIds from § Participant confirmation; do not invent new ones. **Chunking — long inputs.** For inputs > ~525K chars, work through the cached text in deterministic ~525K-char windows with ~17.5K-char overlap, emit sections per window, and merge same-kind boundary straddlers yourself before calling `memory-ingest`. Inputs > 682K chars in chat mode are too large to keep within one turn — abort with `cause:"input-too-large"` and ask the operator to sessionize the export before re-dispatch. **Failure is terminal.** If you cannot produce the JSON (the ontology you need isn't loaded, the extract is empty, the document is opaque), **do not call `memory-ingest`. Do not write anything.** Return to the dispatching agent immediately with a structured blocker: `Classifier unable to map document — . Document not ingested. Operator must .` There is no "fallback-to-UNMAPPED-section" write path — that fallback was removed by design. After you've assembled the JSON, emit a chat message before step 3 naming what you produced: **"Classified into N sections, covering: ."** This is step 2 of the four-step narrative the operator sees per ingest. ### 3. `memory-ingest` Writes the classified document or chat archive. **Document mode (default).** Inputs: `attachmentId`, `documentSummary`, `anchorNodeId`, `anchorLabel`, `sections`, `documentEdges` (pass through if present), `orphanCandidates` (pass through if present), `scope` (from the brief — confirm with the operator if absent), optional `documentKeywords`, `userKeywords`, `sourceUrl`, `sourceType`. **Chat mode.** Inputs: `conversationIdentity` (passed instead of `attachmentId` — the dispatcher signal), `documentSummary`, `sections` (the chunks from chat-mode classify), `scope`, plus the chat-archive metadata: `archiveSha256` (cleanup discriminator), `archiveSourceFile` (audit), `participantElementIds` (owner + others, for `:PARTICIPANT_IN` edges). Pass `anchorNodeId` and `anchorLabel` as any non-empty placeholder (e.g. the owner's elementId + `'AdminUser'`) — they are unused on the chat path but the parameter is non-optional. The writer MERGEs `:KnowledgeDocument { conversationIdentity }`, drops any chunks stamped with this `archiveSha256` (idempotent re-ingest), CREATEs new `:Section` chunks chained by `:NEXT`, and MERGEs `:PARTICIPANT_IN` edges from each participant. Returns: - `documentNodeId`, `sectionCount` - `kindBreakdown` — per-kind count, e.g. `{"Position": 4, "Chapter": 12, "Other": 1}` - `edgeBreakdown` — per-edge-type count, e.g. `{"HAS_SECTION": 17, "NEXT": 16, "HAS_POSITION": 4, "ATTENDED": 1, "AT": 4}` - `relatedCount` — Organizations/Persons/DefinedTerms created or merged - `standaloneCount` — `:Project` nodes created - `orphanCandidates` — passed through from the classifier so the agent can surface them in the post-write narrative - `documentSummary`, `keywords` After step 3 succeeds, emit the third chat message naming everything the writer created or connected (see "Four-step operator narrative" below). Continue to step 4 (`wire-brief-entities`) before the final fourth chat message. Re-ingesting the same `attachmentId` is safe — the writer drops prior `:Section` children (any secondary label), prior `:Project` standalone nodes, and prior KD-level edges (every edge stamped `sourceDocumentId=` plus the legacy `PARTY` edge type by name — covers classifier-emitted PARTICIPANT/FROM/TO/CC/SPEAKER and agent-emitted brief-wired edges) before re-creating. Shared related entities (Organizations, Persons MERGEd by identifying property) are spared. ### 4. `wire-brief-entities` **Skipped in chat mode** — conversation-document parents do not carry KD-level brief-wired edges; participants are already attached via `:PARTICIPANT_IN` and message bodies stay verbatim inside chunk text (mention extraction is deferred to a separate insight-derivation task). Document mode only. After `memory-ingest` returns the new KnowledgeDocument's `documentNodeId`, this step iterates the entities the dispatch brief named and connects each to the new document with the natural KD-level edge. **Entity sources.** The dispatch brief's "key entities to connect" list. Brief shape: prose names of Persons, Organizations, Services, Tasks, Events, KnowledgeDocuments, BrandingData that the document describes or references. Example: *"Person nodes for Joel Smalley, Adam Mackay, Dan McLeod; LocalBusiness / Organization nodes for Real Agent / Real Agency; Any existing Task nodes related to Real Agent Lettings."* Extract every named entity from the brief before any `memory-write`. **Resolution.** For each named entity: 2. If the entity resolves to exactly one node, write the edge from KD to that node (see edge-type table below). 3. If the entity resolves to zero nodes, append it to the orphan list (do NOT create a placeholder Person/Organization — that's classifier-emitted `documentEdges` territory, not brief-wiring). 4. If the entity resolves to multiple nodes (ambiguity — two Persons named "Joel"), pick the one whose context best fits the brief and document (e.g. last-mentioned-recently, or matching account). Surface the choice in the operator narrative so it can be corrected. **Edge-type table.** Edge depends on entity kind and document shape: | Entity kind | Document shape | Edge | Direction | |---|---|---|---| | Person | meeting / call (notes or transcript) | `PARTICIPANT` | KD → Person | | Person | email | `FROM` (sender) / `TO` (direct recipient) / `CC` (carbon copy) | KD → Person | | Person | voice-note / single-speaker transcript | `SPEAKER` | KD → Person | | Person | contract / agreement | `PARTY` | KD → Person | | Person | other (report, brief, generic notes) | `MENTIONS` | KD → Person | | Organization | meeting / call | `PARTICIPANT` | KD → Organization | | Organization | email | `TO` / `CC` | KD → Organization | | Organization | contract | `PARTY` | KD → Organization | | Organization | other | `MENTIONS` | KD → Organization | | Task | any | `REFERENCES` | KD → Task | | Event | any | `REFERENCES` | KD → Event | | Service | any | `REFERENCES` | KD → Service | | KnowledgeDocument (cross-link) | any | `REFERENCES` | KD → KnowledgeDocument | | BrandingData | any | `REFERENCES` | KD → BrandingData | When the classifier already emitted a `documentEdges` entry for the same (KD, target) pair (because the document body itself names the party — common for emails and contracts), the brief-wiring step is redundant for that party — `memory-search` for `elementId(n)` will find the just-written party and the second `memory-write` would create a duplicate edge. Skip the brief entry whose target was already wired by the classifier. **Provenance discipline.** Every brief-wired edge carries `sourceDocumentId=` and `createdByAgent='document-ingest'` in its edge properties. The `memory-ingest` writer's re-ingest cleanup matches edges by `sourceDocumentId`, so a re-ingest of the same `attachmentId` drops them deterministically — without these stamps, brief-wired edges would accumulate on every re-ingest. This is load-bearing: forgetting the stamp leaks edges silently. Pass them on every `memory-write` `relationships` entry's properties block. **Operator narrative line.** Emit one chat line after the loop completes: ``` [document-ingest] wire-brief-entities resolved= orphans= edges= — Persons via PARTICIPANT/FROM/TO/CC/SPEAKER/PARTY/MENTIONS, Organizations via PARTICIPANT/TO/CC/PARTY/MENTIONS, Tasks via REFERENCES, Events via REFERENCES, Services via REFERENCES. Not found in graph: . ``` The leading `[document-ingest] wire-brief-entities` token is grep-able from the agent's stream log so `edges=0` while the brief listed entities is the regression metric. If the brief lists no entities (single-author personal CV, generic FAQ, etc.), this step is a no-op: emit `[document-ingest] wire-brief-entities resolved=0 orphans=0 edges=0` and continue. The four-step narrative reverts to three lines for downstream chat (no fourth narrative line is emitted when this step does nothing). ## Four-step operator narrative Each ingest produces up to four chat messages, each at the moment it happens. Failures replace the relevant step's message with a blocker. Step 4 collapses to nothing visible when the brief named no entities (the agent still emits the `[document-ingest] wire-brief-entities resolved=0 orphans=0 edges=0` log line for grep-ability, but no fourth chat message). | Step | Emitted by | Trigger | Content | |---|---|---|---| | 1 | Admin agent | On dispatch to librarian | "Sending `` to librarian for ingest." | | 2 | Librarian (this skill) | After you finish producing the typed-section JSON in-turn | "Classified `` into `` sections, covering: `, , …`." | | 3 | Librarian (this skill) | After `memory-ingest` returns | "Created/connected: `` for each kind; `` for any orphans; ``." | | 4 | Librarian (this skill) | After `wire-brief-entities` step completes AND the brief named at least one entity | "Wired `` brief entities: `` Persons via ``, `` Organizations via ``, `` Tasks via `REFERENCES`. `

` entities not found in graph: ``." | **Step 2 failure form:** "Classifier unavailable — ``. `` not ingested. ``." Do not call step 3. **Step 3 failure form:** "`` sections classified but write failed at `` — ``. `` not in graph." Do not call step 4. **Step 4 partial form:** "Wired `` brief entities; `

` not found: ``." If every brief-named entity resolved, drop the "not found" clause. If zero brief entities were named, suppress the chat message entirely (the log line still fires). The operator always sees what was attempted and where it stopped. ## Web-page ingestion For URL-based ingestion, run `memory-ingest-web` first with the URL and the WebFetch result. It writes the content to a temp file and routes through `memory-ingest-extract`, returning a fresh `attachmentId`. The rest of the pipeline (classify + ingest) is identical to a file upload. Pass `sourceUrl` and `sourceType: 'web'` to `memory-ingest`. **Page title is mandatory input — parse it before the call.** `KnowledgeDocument.name` is what shows up everywhere the document is referenced (`pickShortLabel` / `pickDisplayName` check `name` first), so it must be an editorial title, never body prose. Before calling `memory-ingest-web`, scan the WebFetch result for the page's title and pass it as the `title` parameter: - The readability output often starts with the page's `` or `og:title` as a literal H1 (e.g. `# I am no longer the Official Representative for Reform UK`). Use that line verbatim — strip the leading `# ` and any trailing site name suffix (e.g. ` - Substack`, ` | Medium`). - When the readability output strips the title entirely (rare; some publishers prefix the body with marketing chrome), look at the URL's final path segment — `…/i-am-no-longer-the-official-representative` is the slug the publisher chose for the article. - Never invent a title from the article's opening paragraph. Body prose is exactly what this parameter exists to avoid. The server's fallback chain when `title` is absent: markdown H1 in the cached text → URL-derived slug → the URL itself. The librarian's job is to make the `caller` branch the common case; the server-side fallbacks are last-resort, not the default path. The response includes `titleSource` (`caller` / `h1` / `url-slug` / `url`) so the operator can see which path fired. **WebFetch limitation — subscriber-gated and JS-rendered pages (Substack, Medium-locked, paywalled news, gated PDFs, single-page apps).** `WebFetch` retrieves only the public, server-rendered surface. For gated articles the return is the meta-description / preview blurb / third-person summary that the publisher exposes to scrapers — not the author's original prose. Verbatim text is unrecoverable through this path. **Detect this before ingesting:** if the WebFetch result looks like a third-person synopsis of an article whose author would have written in first person, or carries publisher-summary framing (`# <Title> - Full Article Text` headers, marketing blurbs, "Subscribe to read"), abort the ingest and surface the gap to the dispatching agent in one line: *"WebFetch for `<url>` returned a publisher summary, not the article body. Ask the operator for a different input shape — for a Substack URL on the operator's own publication, use the `substack-import` plugin's export-archive flow (Substack admin → 'Export your data' → drop the ZIP); otherwise paste the article text into chat or attach a PDF / .txt export."* The dispatching agent relays this to the operator and re-dispatches with the pasted text, uploaded file, or Substack archive as the input. Do NOT proceed to `memory-ingest-web` with the summary — `:Section.body` will faithfully reproduce that summary downstream, defeating the verbatim-prose contract. ## Attachment metadata format Your task description contains attachment metadata in this format: ``` 1. report.pdf application/pdf 1,234,567 bytes ID: 04caba6b-... Path: /home/admin/.maxy/data/uploads/... ``` Extract: - **filename** = the text BEFORE the MIME type on the FIRST line. Always the human-readable name; never a UUID. - **mimeType** = the MIME type on the first line. - **attachmentId** = the UUID after `ID:` on the SECOND line. - **storagePath** = the absolute path after `Path:` on the second line. Process each text-based attachment (PDF, plain text, markdown). Skip images — they have no text to ingest. The `storagePath` you pass to `memory-ingest-extract` is threaded through to the writer automatically (via the extract cache) and persisted on the `:KnowledgeDocument` as `sourcePath`, so the sidebar download resolver can serve the real file later. You do not pass it to `memory-ingest` — the cache carries it. Web ingests are excepted: `memory-ingest-web`'s temp file is unlinked after extraction, so no `sourcePath` is persisted and the doc stays non-downloadable by design. ## Visibility scope `memory-ingest` requires `scope`. The brief specifies it; if not, ask the operator before ingesting. - **public** — visible to public agents and admin. For business knowledge customers should access (products, services, FAQs, policies). - **shared** — visible to all account agents but not public visitors. Internal operational knowledge. - **admin** — admin agent only. Sensitive content (contracts, credentials, internal processes). CV / personal-profile content defaults to `admin`. Pricing guides, services, FAQs default to `public`. When uncertain, ask in one sentence. ## Discipline (load-bearing) 1. **Anchor confirmed first.** No extract / classify / ingest until `$anchorNodeId` and `$anchorLabel` are persisted and echo-confirmed. 2. **Generic skill — no document-type branching.** The classifier maps sections to the closed enumeration. The writer applies the classifier's edges; never invents its own. 3. **Schema-base property names.** The classifier emits `givenName` / `familyName` / `telephone` / `dateSent` — not `firstName` / `phone` / `sentAt`. The `memory-write` validator enforces this via the synonym map. 4. **Closed enumeration of section kinds.** Identity, structural, contract-clause, plus standalone `Project` and label-fallback `Other`. The classifier never invents a new kind. `Other` collapses unrecognised content with a `classifierReason`; the operator runs `MATCH (s:Section:Other) WHERE s.accountId = $acc RETURN s.title, s.classifierReason, count(*) AS occurrences ORDER BY occurrences DESC` to surface ontology-growth candidates. 5. **Provenance stamp.** Every section, standalone, and edge gets `createdByAgent='document-ingest'`, `createdBySession=<sessionId>`, `source='document'`, `sourceDocumentId=<attachmentId>`. Stamped automatically by `memory-ingest` for everything it writes; **also stamped by every `memory-write` call in step 4** (`wire-brief-entities`) on the edge properties — without these stamps, brief-wired edges leak on re-ingest. 6. **Idempotent re-ingest.** `memory-ingest` is safe to re-run with the same `attachmentId`. Prior `:Section` children (any secondary label) and `:Project` standalones are deleted; prior KD-level edges are deleted by **provenance match** — every edge stamped `sourceDocumentId=<this attachmentId>` is dropped, regardless of edge type. The legacy `PARTY` edge type is also matched by name as a belt-and-braces for pre-stamping data. Any KD-level edge that this pipeline writes (classifier-emitted PARTY/PARTICIPANT/FROM/TO/CC/SPEAKER and agent-emitted brief-wired MENTIONS/REFERENCES/etc.) MUST carry the stamp — without it, the edge persists across re-ingests as a duplicate. MERGEd shared entities (Organizations, Persons referenced by other docs) are spared. 7. **Loud failure, never silent landfill.** Classifier failure aborts the ingest; nothing is written. Edge orphans are surfaced via `orphanCandidates`; the writer never synthesises edges to "fix" them. 8. **Brief is a deliverable, not a hint.** Step 4 is mandatory whenever the dispatch brief named entities. Returning *"document ingested as a KnowledgeDocument anchored to <X>"* without listing wired/unresolved brief entities is a silent failure — the document becomes an island anchored to one node while the named Persons/Organizations/Tasks stay disconnected, defeating adjacency-ranked `memory-search`. The `[document-ingest] wire-brief-entities` log line is the verification surface; `edges=0` while the brief listed entities is a regression. 9. **Typed-entity edges never target a `:Section`.** Edge types declared in `TYPED_EDGE_ALLOWLIST` (`FOUNDED`, `WORKS_AT`, `INVESTED_IN`, `ADVISES`, `MENTIONS`, `AUTHORED`, `ATTACHED_TO`, `REFERENCES`, `ATTENDED`) must honour the (source, target) label shapes declared there. When the natural target is the document anchor (e.g. founders in a company-page Team section), set `edge.targetMode = "anchor"` on the related entry — the writer routes the edge to the anchor instead of the section. The writer rejects loud with the offending `(:Source)-[:EDGE]->(:Target)` triple on any classifier emission that violates the allowlist. ## Output contract Return to the admin agent: - **What you did** — filename, anchor (`UserProfile` / `LocalBusiness` / etc.), scope. - **Outcome** — `documentNodeId`, `sectionCount`, `kindBreakdown`, `edgeBreakdown`, `relatedCount`, `standaloneCount`. Surface the document summary and extracted keywords so the admin can echo them back to the operator. - **Brief-entity wiring** — list of brief-named entities resolved (with elementId + edge type written) and not-found (orphans). Empty when the brief named no entities. Never elide; absence is silently broken adjacency. - **Orphan candidates** — for each entry the classifier flagged: kind, label, reason. The admin agent reads these to decide whether the orphan is intentional (rare, valid) or a classifier miss (bug). Never silently skip. - **Ontology gaps** — list any `:Section:Other` titles + classifierReasons. The admin agent reads these to decide whether to extend the ontology in a follow-up task. - **Blockers** — anchor unconfirmed, classifier failure (Haiku unavailable, malformed JSON, etc.), schema validator rejection, write failure mid-ingest.