---
description: Document Management
alwaysApply: false
---

# Document Management

Rules for ingesting, parsing, classifying, and storing documents within the knowledge base.

## Ingestion Pipeline

1. **Intake** — Accept document (PDF, web clip, note, annotation export)
2. **Parsing** — Extract text, metadata, sections, figures, references
3. **Classification** — Assign type, topics, domain tags, quality score
4. **Enrichment** — Resolve citations, extract keywords, run NER, cross-reference
5. **Storage** — Index full text, store metadata, generate vector embeddings, archive file

## Metadata Requirements

Every document must have: unique ID (UUID), title, author(s), date, document type, at least one topic tag, relevance score (1-5), ingestion date, original file hash for deduplication.

## Classification Standards

| Type | Priority | Retention |
|------|----------|-----------|
| Academic Paper | High | Permanent |
| Book Chapter | High | Permanent |
| Personal Note | High | Permanent |
| Technical Report | Medium | Review annually |
| Blog Post / Article | Medium | Review bi-annually |
| Lecture / Talk | Medium | Permanent |
| Email / Thread | Low | Archive after extraction |
| Social Media | Low | Extract insights only |

## Duplicate Detection

Apply in order, stop at first match:
1. Exact file hash (SHA-256)
2. DOI or ISBN match
3. Title + Author fuzzy match (Levenshtein < 3)
4. Embedding cosine similarity > 0.95

Merge metadata on duplicate; discard newer copy. Never silently create duplicates.

## Processing State Tracking

Track per document: `parsed`, `summarized` (level 0-3), `linked` (to graph), `reviewed` (last human review date). Flag unprocessed inbox items older than 7 days.

## Quality Gates

Before fully ingested: metadata complete, no unresolved duplicates, topic tag assigned, full text indexed, vector embedding generated, processing state updated.