# Process Brief: Advanced Reference

Edge-case tools and techniques. **Load only when the main `process.md` doesn't cover the task.**

```
Edge case
  ├─ No text extracted from PDF    → §OCR Fallback
  ├─ PDF is encrypted/locked       → §Encrypted PDFs
  ├─ PDF is corrupted/damaged      → §Corrupted PDFs
  ├─ Need precise text coordinates → §pdfplumber Advanced
  ├─ Need fast rendering           → §pypdfium2
  ├─ Advanced page extraction      → §poppler-utils Advanced
  ├─ Complex page ranges / merge   → §qpdf Page Manipulation
  ├─ Optimize file size            → §qpdf Optimization
  ├─ Batch process many PDFs       → §Batch Processing
  └─ Memory issues with large PDF  → §Performance Optimization
```

For basic operations (extract, merge, split, fill forms, convert), go back to `process.md`.

---

## §pypdfium2 (Apache/BSD License)

A Python binding for PDFium (Chromium's PDF library). Excellent for fast rendering and text extraction — serves as a PyMuPDF replacement.

#### Render PDF to Images
```python
import pypdfium2 as pdfium

pdf = pdfium.PdfDocument("document.pdf")

# Render single page
page = pdf[0]
bitmap = page.render(scale=2.0, rotation=0)
img = bitmap.to_pil()
img.save("page_1.png", "PNG")

# Batch render all pages
for i, page in enumerate(pdf):
    bitmap = page.render(scale=1.5)
    img = bitmap.to_pil()
    img.save(f"page_{i+1}.jpg", "JPEG", quality=90)
```

#### Extract Text with pypdfium2
```python
import pypdfium2 as pdfium

pdf = pdfium.PdfDocument("document.pdf")
for i, page in enumerate(pdf):
    text = page.get_text()
    print(f"Page {i+1}: {len(text)} chars")
```

## §pdfplumber Advanced Features

#### Extract Text with Precise Coordinates
```python
import pdfplumber

with pdfplumber.open("document.pdf") as pdf:
    page = pdf.pages[0]

    # All characters with coordinates
    for char in page.chars[:10]:
        print(f"'{char['text']}' at x:{char['x0']:.1f} y:{char['y0']:.1f}")

    # Extract text within a specific bounding box (left, top, right, bottom)
    bbox_text = page.within_bbox((100, 100, 400, 200)).extract_text()
```

#### Advanced Table Extraction with Custom Settings
```python
import pdfplumber

with pdfplumber.open("complex_table.pdf") as pdf:
    page = pdf.pages[0]

    table_settings = {
        "vertical_strategy": "lines",
        "horizontal_strategy": "lines",
        "snap_tolerance": 3,
        "intersection_tolerance": 15
    }
    tables = page.extract_tables(table_settings)

    # Visual debugging
    img = page.to_image(resolution=150)
    img.save("debug_layout.png")
```

---

## §poppler-utils Advanced Features

### Extract Text with Bounding Box Coordinates
```bash
pdftotext -bbox-layout document.pdf output.xml
```

### Advanced Image Conversion
```bash
# High-resolution PNG
pdftoppm -png -r 300 document.pdf output_prefix

# Specific page range
pdftoppm -png -r 600 -f 1 -l 3 document.pdf high_res_pages

# JPEG with quality setting
pdftoppm -jpeg -jpegopt quality=85 -r 200 document.pdf jpeg_output
```

### Extract Embedded Images
```bash
pdfimages -all document.pdf images/img
pdfimages -list document.pdf
```

---

## §qpdf Advanced Features

### Complex Page Manipulation
```bash
# Split into groups of N pages
qpdf --split-pages=3 input.pdf output_group_%02d.pdf

# Extract complex page ranges
qpdf input.pdf --pages input.pdf 1,3-5,8,10-end -- extracted.pdf

# Merge specific pages from multiple PDFs
qpdf --empty --pages doc1.pdf 1-3 doc2.pdf 5-7 doc3.pdf 2,4 -- combined.pdf
```

### PDF Optimization and Repair
```bash
qpdf --linearize input.pdf optimized.pdf
qpdf --optimize-level=all input.pdf compressed.pdf
qpdf --check input.pdf
qpdf --fix-qdf damaged.pdf repaired.pdf
```

### Encryption and Decryption
```bash
qpdf --encrypt user_pass owner_pass 256 --print=none --modify=none -- input.pdf encrypted.pdf
qpdf --show-encryption encrypted.pdf
qpdf --password=secret123 --decrypt encrypted.pdf decrypted.pdf
```

---

## §Encrypted PDFs

```python
from pypdf import PdfReader

try:
    reader = PdfReader("encrypted.pdf")
    if reader.is_encrypted:
        reader.decrypt("password")
except Exception as e:
    print(f"Failed to decrypt: {e}")
```

Or via qpdf:
```bash
qpdf --password=secret123 --decrypt encrypted.pdf decrypted.pdf
```

---

## §Corrupted PDFs

```bash
qpdf --check corrupted.pdf
qpdf --replace-input corrupted.pdf
```

---

## §OCR Fallback for Scanned PDFs

When `pdfplumber` extracts 0 characters, the PDF is likely a scanned image:

```python
import pytesseract
from pdf2image import convert_from_path

def extract_text_with_ocr(pdf_path):
    images = convert_from_path(pdf_path)
    text = ""
    for i, image in enumerate(images):
        text += pytesseract.image_to_string(image)
    return text
```

Prerequisites: `pip install pdf2image pytesseract` + install Tesseract OCR and poppler.

---

## §Batch Processing with Error Handling

```python
import os, glob
from pypdf import PdfReader, PdfWriter
import logging

logging.basicConfig(level=logging.INFO)
logger = logging.getLogger(__name__)

def batch_process_pdfs(input_dir, operation='merge'):
    pdf_files = glob.glob(os.path.join(input_dir, "*.pdf"))

    if operation == 'merge':
        writer = PdfWriter()
        for pdf_file in pdf_files:
            try:
                reader = PdfReader(pdf_file)
                for page in reader.pages:
                    writer.add_page(page)
                logger.info(f"Processed: {pdf_file}")
            except Exception as e:
                logger.error(f"Failed: {pdf_file}: {e}")
                continue
        with open("batch_merged.pdf", "wb") as output:
            writer.write(output)

    elif operation == 'extract_text':
        for pdf_file in pdf_files:
            try:
                reader = PdfReader(pdf_file)
                text = "".join(page.extract_text() for page in reader.pages)
                output_file = pdf_file.replace('.pdf', '.txt')
                with open(output_file, 'w', encoding='utf-8') as f:
                    f.write(text)
                logger.info(f"Extracted: {pdf_file}")
            except Exception as e:
                logger.error(f"Failed: {pdf_file}: {e}")
```

---

## §Performance Optimization

### Text Extraction
- `pdftotext -bbox-layout` is fastest for plain text
- Use pdfplumber for structured data and tables
- Avoid `pypdf.extract_text()` for very large documents

### Image Extraction
- `pdfimages` is much faster than rendering entire pages
- Use low resolution for previews, high resolution for final output

### Memory Management for Large PDFs
```python
def process_large_pdf(pdf_path, chunk_size=10):
    reader = PdfReader(pdf_path)
    total_pages = len(reader.pages)

    for start_idx in range(0, total_pages, chunk_size):
        end_idx = min(start_idx + chunk_size, total_pages)
        writer = PdfWriter()
        for i in range(start_idx, end_idx):
            writer.add_page(reader.pages[i])
        with open(f"chunk_{start_idx//chunk_size}.pdf", "wb") as output:
            writer.write(output)
```

---

## Extended Tooling Inventory

| Library / Tool | Role | Licence |
|----------------|------|---------|
| pikepdf | Low-level PDF manipulation (forms, pages, metadata) | MPL-2.0 |
| pdfplumber | Content extraction (text, tables) | MIT |
| pypdfium2 | Fast rendering, text extraction (PyMuPDF alternative) | Apache/BSD |
| pypdf | Merge, split, crop, metadata, encryption | BSD |
| poppler-utils | CLI text/image extraction, rendering | GPL-2 |
| qpdf | Page manipulation, optimization, encryption, repair | Apache |
| pytesseract | OCR for scanned PDFs | Apache |
| pdf2image | PDF-to-image conversion via poppler | MIT |
| LibreOffice | Office format conversion engine | MPL-2.0 |