---
description: Evaluate agent quality across three modes — without BK, BK grep-only, and BK full
argument-hint: "[query | --predefined | --predefined N]"
allowed-tools: ["mcp__bluera-knowledge__execute", "mcp__bluera-knowledge__search", "mcp__bluera-knowledge__get_full_context", "Read", "Grep", "Glob", "WebSearch", "Bash"]
context: fork
---

# Agent Quality Evaluation

Compare how well Claude answers library questions across three access levels:

- **Without BK** — web search + training knowledge only
- **BK Grep** — Grep/Read/Glob on cloned repos, no vector search
- **BK Full** — vector search + get_full_context + Grep/Read

## Arguments

Parse `$ARGUMENTS`:

- **No arguments**: Show usage help
- **Quoted string**: Run eval for that single question
- **`--predefined`**: Run all predefined queries
- **`--predefined N`**: Run predefined query #N only

## Workflow

1. **Prerequisites**: Call `execute` with `{ command: "stores" }` to list stores. Abort if none.
2. **Resolve queries**: Load from `$CLAUDE_PLUGIN_ROOT/evals/agent-quality/queries/predefined.yaml` or use arbitrary query.
3. **Load templates**: Read agent prompts + judge rubric from `$CLAUDE_PLUGIN_ROOT/evals/agent-quality/templates/`
4. **Spawn 3 agents in parallel** per query (replace `{{QUESTION}}`, `{{STORES}}`, `{{STORE_PATHS}}`)
5. **Judge**: Score all 4 criteria (1-5): Accuracy, Specificity, Completeness, Source Grounding

Detailed procedures: [references/procedures.md](references/procedures.md)

Output format: [references/output-format.md](references/output-format.md)