---
type: Rubric
title: "Eval Rubric: Agent-run capability baseline (kit seed — copy per initiative)"
timestamp: 2026-07-19
eval_type: capability
initiative_id: (set per initiative)
pass_threshold: 0.80
canonical_path: pmos/planning/evals/agent-run-eval-rubric.md
---

# Eval Rubric: Agent-run capability baseline (kit seed — copy per initiative)

> Kit-seeded generic rubric (D54). Two purposes: (1) it is the TEMPLATE to copy to
> `<initiative-id>-eval-rubric.md` and specialize per initiative (rewrite the dimensions to the
> initiative's own outcomes; keep weights summing to 1.0); (2) it keeps the `rubrics` CI check
> meaningful from day one. It scores a generic agent run outcome-only (D14) with weighted partial
> credit (D15).

## Gate assignment
- **Quality Gate (deterministic):** repo CI green; `pmos-gate` (anchored + rubrics) green.
- **Review Gate (adversarial):** the weighted dimensions below — evaluator skill, default-to-refute,
  outcome-only; verdict appended to `pmos/state/evals.jsonl`; ADVISORY until locally calibrated.
- **Acceptance Gate (PM):** the human PM merges and disposes — final and non-delegable (D12).

## Weighted dimensions (Review Gate)
Outcome-only (D14); partial credit (D15); anchors 0 / 0.5 / 1.0. **Evidence rule:** the verdict
must cite, per scored dimension, a short verbatim quote or precise pointer (file+line, command
output) from the evaluated artifact; a dimension without evidence is marked `unverified` and
scored conservatively (≤ 0.5) — never high on trust (evidence-anchored scoring,
llm-judge-calibration research).

| Dimension | Weight | Score | Wtd |
|---|---:|---:|---:|
| Contract satisfaction — every done_criterion of the pre-run contract is met, verified per its test_method | 0.40 | — | — |
| Correctness of the produced artifact — behaves/reads as specified; no defect a reviewer can demonstrate | 0.30 | — | — |
| Scope discipline — nothing out of contract scope changed; state files appended, never rewritten | 0.20 | — | — |
| Traceability — initiative anchored; run + eval + friction logged to pmos/state/ | 0.10 | — | — |
| **Total** | **1.00** | | **vs 0.80** |

### Anchors (0 / 0.5 / 1)
- **Contract:** 0 = a criterion unmet; 0.5 = met but a test_method not actually exercised; 1 = all met and exercised.
- **Correctness:** 0 = demonstrable defect; 0.5 = works with a material caveat; 1 = defect-free under the rubric's probes.
- **Scope:** 0 = out-of-scope change or state rewrite; 0.5 = minor drift, flagged; 1 = clean.
- **Traceability:** 0 = unanchored or unlogged; 0.5 = partial logging; 1 = complete.

## Sprint contract (pre-run)
initiative_id: (set)
run_id: (set)
done_criteria:
  1. (numbered, testable — agreed BEFORE work starts)
test_method:
  1. (one per criterion; name the gate that checks it)
PM_approved: false  <!-- flip to true only on the PM's explicit word -->