---
name: job-eval
description: Parallel batch processing — evaluate 10-100+ job offers simultaneously via claude -p workers with state tracking, retry logic, and resumability.
---

# Batch Job Evaluation

Process multiple job offers in parallel. Each worker runs a full A-G evaluation independently. The orchestrator tracks state, handles retries, and merges results into the application tracker.

Each worker produces a report + tracker entry. The orchestrator manages the coordination.

## Setup

**Requirements**
- Python 3.10+; Bash for the batch runner
- Install the Notion integration package:
  ```bash
  python -m pip install sarvesh-ai-notion-interface
  ```
- Notion access via environment variables or a `.env` file in your working directory:
  - `NOTION_TOKEN` — Notion integration token (required)
  - Database IDs as needed: `NOTION_DB_APPLICATIONS`, `NOTION_DB_COMPANIES`, `NOTION_DB_CONNECTIONS`
  - Page IDs as needed: `NOTION_PAGE_PARENT`, `NOTION_PAGE_RESUME`, `NOTION_PAGE_PROJECTS`

The Notion integration package (`sarvesh-ai-notion-interface`) is published on PyPI and contains all database helpers for job tracking.

## When to Activate

- User has 3+ job offers to evaluate
- User wants to process their entire pipeline at once
- User says "batch evaluate", "process all pending", or "evaluate everything"
- After job-scan discovers many new offers

## Prerequisites

- **`claude` CLI** installed and authenticated
- **NOTION_TOKEN** environment variable set
- `Notion (fetch via `python3 -m sarvesh_ai_notion_interface.page_reader resume`)` and `Notion (fetch via `python3 -m sarvesh_ai_notion_interface.page_reader projects`)` populated
- URLs to evaluate (in `batch-data/batch-input.tsv` or use `--from-notion` to pull "Scanned" jobs)

## Non-Negotiables

1. **Lock file prevents double execution** — never run two batch-runners simultaneously
2. **Every worker is independent** — one failure doesn't affect others
3. **State is always persisted** — interrupted runs resume from where they left off
4. **Workers write to Notion directly** — no local merge needed
5. **Workers don't generate PDFs** — only reports + tracker entries. Run job-cv-tailor separately for high-scoring roles.
6. **Disqualification filters run before A-G evaluation** — non-US jobs and clearance-required jobs are rejected early with score 0

## Disqualification Filters

Workers apply two automatic disqualification filters **before** the expensive A-G evaluation. If either triggers, the job is immediately scored 0, marked "Discarded" in Notion, and the worker stops early.

### What triggers automatic disqualification

| Filter | Disqualify (score 0) | Pass |
|--------|---------------------|------|
| **Non-US Location** | Explicitly non-US locations (London UK, Berlin Germany, Toronto Canada, etc.), "Remote (EU only)", "Remote (EMEA)", "Remote (APAC)" | US cities/states, "Remote", "Remote (US)", "Hybrid" with US city, ambiguous/missing location |
| **Security Clearance / US Citizenship** | Active clearance required (TS, TS/SCI, Secret, Top Secret, DoD), ability to obtain clearance, "US citizenship required", "Must be a US citizen", "US Person" per ITAR/EAR | "Authorized to work in US" (F-1 OPT satisfies), "Sponsorship available", "US Person preferred" (soft) |

### How disqualification appears

- **State file:** status = `completed`, score = `0`
- **Notion:** status = `Discarded`, score = `0`
- **Worker JSON output:** includes `"disqualified": true` and `"disqualification_reason": "{reason}"`
- **Batch summary:** counted separately in the "Disqualified" counter
- **Report:** short mini-report (no A-G sections) explaining the disqualification reason

## Running the Batch

### From the repo root:

```bash
# Pull "Scanned" jobs from Notion and evaluate in parallel
bash batch-runner.sh --from-notion --parallel 4

# Preview what would be processed (no execution)
bash batch-runner.sh --from-notion --dry-run

# Evaluate all offers in batch-input.tsv (manual input)
bash batch-runner.sh

# Evaluate 4 offers in parallel
bash batch-runner.sh --parallel 4

# Retry only failed offers
bash batch-runner.sh --retry-failed --parallel 2

# Start from offer #10, max 3 retries each
bash batch-runner.sh --start-from 10 --max-retries 3

# Skip tracker entries for low-scoring offers
bash batch-runner.sh --min-score 3.5 --parallel 4
```

### CLI Arguments

| Argument | Default | Description |
|----------|---------|-------------|
| `--parallel N` | 1 | Number of concurrent workers |
| `--dry-run` | false | Preview without executing |
| `--from-notion` | false | Query Notion for "Scanned" jobs and generate batch-input.tsv |
| `--retry-failed` | false | Only retry previously failed offers |
| `--start-from N` | 0 | Skip offers with ID < N |
| `--max-retries N` | 2 | Max retry attempts per offer |
| `--min-score N` | 0 (off) | Skip tracker/report for scores below N |

## Input Format

### batch-input.tsv

Location: `skills/job-eval/batch-data/batch-input.tsv`

```tsv
id	url	source	notes	page_id
1	https://job-boards.greenhouse.io/anthropic/jobs/123	Greenhouse
2	https://jobs.ashbyhq.com/openai/abc	Ashby	high priority
3	https://jobs.lever.co/scaleai/xyz	Notion	Scale AI — ML Engineer	34598a5a-ae72-xxxx-xxxx-xxxxxxxxxxxx
```

**Columns:**
1. `id` — sequential integer (1, 2, 3...)
2. `url` — full job posting URL
3. `source` — where it came from (Greenhouse, Ashby, Lever, Notion, etc.)
4. `notes` — optional metadata
5. `page_id` — Notion page ID (optional; auto-populated by `--from-notion`)

**How to populate:**
- **From Notion (recommended):** use `--from-notion` flag — auto-generates from "Scanned" jobs
- Manually: add URLs you want to evaluate (leave `page_id` empty)

## How It Works

### 1. Lock Acquisition
- Creates `batch-data/batch-runner.pid` with process ID
- If another batch-runner is running → error exit
- Stale locks (dead PID) are auto-recovered

### 2. State Initialization
- Creates `batch-data/batch-state.tsv` if missing
- Reads existing state to determine what's pending/completed/failed

### 3. Worker Spawning (per offer)

For each pending offer:

```
Reserve report number (sequential, 3-digit zero-padded)
    ↓
Resolve placeholders in batch-prompt.md ({{URL}}, {{REPORT_NUM}}, {{DATE}}, {{ID}})
    ↓
Launch: claude -p --dangerously-skip-permissions --append-system-prompt-file resolved-prompt.md "Process offer..."
    ↓
Worker produces: report to Notion (via db_applications.py) + JSON stdout
    ↓
Orchestrator extracts score from JSON, updates state
```

### 4. Parallel Processing

With `--parallel N`:
- Maintains pool of N background workers
- When one finishes, next offer starts immediately
- State lock prevents race conditions on state file updates

### 5. Post-Processing

After all workers finish:
- Each worker already pushed its evaluation to Notion directly
- No local merge needed — Notion is the source of truth

### 6. Summary
```
job-eval — 2026-04-15
━━━━━━━━━━━━━━━━━━━━━━━━━━
Total:     50
Completed: 47
Failed:    2
Skipped:   1 (below min-score)
Average:   3.9/5

→ Run job-cv-tailor on high-scoring offers
```

## State Tracking

### batch-state.tsv

Location: `skills/job-eval/batch-data/batch-state.tsv`

```tsv
id	url	status	started_at	completed_at	report_num	score	error	retries
1	https://...	completed	2026-04-15T10:30:00Z	2026-04-15T10:45:00Z	001	4.3	-	0
2	https://...	failed	2026-04-15T10:46:00Z	2026-04-15T10:50:00Z	002	-	Timeout	1
3	https://...	pending	-	-	-	-	-	0
```

### State Transitions

```
pending → processing → completed (score > 0, full A-G evaluation)
                    → completed (score 0, disqualified — skipped A-G evaluation)
                    → failed (retry with --retry-failed)
                    → skipped (score below --min-score)
```

- `completed` → skipped on re-run (already done)
- `failed` + retries >= max-retries → skipped permanently
- `failed` + retries < max-retries → retried with `--retry-failed`
- `processing` (stale — PID dead) → auto-recovered as pending

## Resumability

If the batch is interrupted (Ctrl+C, crash, laptop dies):

1. Re-run the same command
2. Orchestrator reads `batch-state.tsv`
3. Skips all `completed` offers
4. Retries `failed` offers (if retries < max)
5. Continues with `pending` offers

**Nothing is lost.** Reports and tracker entries from completed workers are already on disk.

## Output

### Per worker:
- **Notion row:** Created with Company, Role, URL, Score, Status, Date
- **Notion report:** Full evaluation content pushed into the page body
- **Log:** `skills/job-eval/batch-data/logs/{report_num}-{id}.log`

### After all workers:
- All evaluations already in Notion — no local merge needed

## Error Handling

| Scenario | What happens |
|----------|-------------|
| URL inaccessible | Worker fails → marked `failed`, continue with others |
| JD behind login wall | Worker can't extract JD → fails → flag for manual paste |
| Worker crashes | Marked `failed` → retry with `--retry-failed` |
| Orchestrator dies | Re-run → skips completed, continues pending |
| Score below min | Marked `skipped` → no report/tracker entry |
| All workers fail | Summary shows 0 completed → investigate logs |

## File Structure

```
skills/job-eval/
├── SKILL.md                        # This file
├── batch-runner.sh                 # Orchestrator script
├── batch-prompt.md                 # Self-contained worker prompt
└── batch-data/                     # Runtime data
    ├── batch-input.tsv             # URLs to process (user populates)
    ├── batch-state.tsv             # State tracking (auto-created)
    └── logs/                       # Worker execution logs

skills/job-scan/
└── portals.yml                     # Scan config (companies + title filters)
```

## Related Skills

- **job-scan** — Discover new offers (writes to Notion as "Scanned")
- **job-eval** — A-G evaluation logic lives in `batch-prompt.md`
- **job-cv-tailor** — Generate tailored CV for high-scoring offers (run after batch)
- **job-apply** — Fill application forms for offers you decide to apply to
