#!/usr/bin/env npx tsx /** * HackAPrompt Benchmark Runner * * Measures ATR recall on the pinned 4,780-sample HackAPrompt set produced by * scripts/hackaprompt-to-corpus.py (dataset revision, seed and row indices in * data/hackaprompt/hackaprompt-corpus.manifest.json). One command: * * npm run eval:hackaprompt * * which regenerates the corpus at the pinned revision (HuggingFace access to * the gated dataset required) and then runs this file. The runner refuses a * corpus whose SHA-256 differs from the manifest. * * Every figure is per lane. The headline (metrics.recall) is the hunt lane, * as for every other source; breakdown.lanes carries hunt, alert and enforce * side by side and breakdown.enforce_lane carries enforce in the shared shape. * Because rules were written from this corpus, the runner also measures it * with those rules removed (breakdown.in_sample_ablation). That ablation is * the reference to cite beside the headline; it approximates an out-of-sample * figure but is not one (see data/hackaprompt/METHOD.md). * * HackAPrompt is an all-adversarial corpus: recall is defined, precision and * FP rate are not (they are recorded as the schema's convention values). * * @module agent-threat-rules/eval/run-hackaprompt-benchmark */ export {}; //# sourceMappingURL=run-hackaprompt-benchmark.d.ts.map