/** * Is a WEAKER model a valid lower bound for trigger-rate? * * THE QUESTION. `measureTriggerRate` floors at Sonnet, because a weaker model * under-selects: the same skill measured 0.50 on haiku and 0.90 on sonnet, so a * haiku number reads as "bad description" when the description is fine. The * standing counter-proposal is the ordinary worst-case argument — tune against * the weakest supported model and anything stronger is covered, the way you test * against the oldest supported runtime. It is a good argument, and it is valid * only if one thing is true: * * CONTAINMENT: fires on the weak model => fires on the strong one * * If containment holds, the weak model is a genuine floor: cheaper, faster, and * it buys margin. If it does not, the weak model is not a floor at all — it is a * DIFFERENT router, and a prompt it fires on may be one the strong model * declines, so tuning against it optimises a configuration nobody runs. * * Selection is routing, not raw capability: a stronger model can legitimately * route ELSEWHERE — doing the work itself, or picking a more specific sibling — * which is why containment was genuinely open rather than obviously true. * * MEASURED 2026-08-11, and it DOES NOT HOLD. 21 skills x 4 prompts, one trial * each, whole-harness against 37 competing skills, haiku against sonnet: of 84 * shared prompts, 3 fired on haiku and NOT on sonnet — and one skill scored * haiku 1.00 against sonnet 0.75, i.e. the weak model is not seeing less, it is * routing differently. Read that as "containment is NOT established" (which is * all the worst-case argument needs to fail — it requires containment to hold, * not to usually hold) rather than "it breaks often": at one trial per prompt * three breaks in 84 could still be dispersion. The model floor stays. * * Pure: give it two reports (or anything carrying `perPrompt`) and it classifies * every prompt the two have in common. * * 🔴 HARNESS-AGNOSTIC BY CONSTRUCTION, BUT NOT EQUALLY TRUSTWORTHY EVERYWHERE. * This compares whatever two runs report, so it runs on Codex reports unchanged — * and should NOT be trusted there yet. Codex emits no skill-selection event, so * `fired` is INFERRED (from a `SKILL.md` read), and `measureTriggerRate` flags the * whole tier experimental for that reason. The inference error lands exactly where * this function looks: a prompt wrongly marked fired on one model and not the * other manufactures a `weakOnly` entry, i.e. a counterexample made of noise. * Read a containment verdict as strong as its weakest input signal — on Claude * Code, where firing is an observed event, that is a real comparison. */ /** The minimum a report must expose to be compared — structural, no import cycle. */ export interface ContainmentInput { /** Identifies the surface these prompts belong to (skill name, usually). */ readonly subject: string; readonly perPrompt: readonly { readonly prompt: string; readonly rate: number; }[]; } export interface ContainmentVerdict { /** Prompts present in BOTH inputs — the only ones a comparison can speak about. */ readonly compared: number; /** Fired on both. */ readonly both: readonly string[]; /** Fired on neither. */ readonly neither: readonly string[]; /** * Fired on the WEAK model but not the strong one. Each is a counterexample to * "the weak model is a floor" — the whole point of the comparison. */ readonly weakOnly: readonly string[]; /** * Fired on the STRONG model but not the weak one. EXPECTED, and not a defect: * it is the weak model under-selecting, which is the known reason for the * model floor. Reported separately so the two are never conflated. */ readonly strongOnly: readonly string[]; /** `weakOnly.length === 0` — the weak model never fired where the strong did not. */ readonly holds: boolean; } /** * Classify every prompt the two runs share. * * ⚠️ At one trial per prompt every cell is a SINGLE observation, so one * `weakOnly` may be noise rather than a counterexample. Read a handful as * evidence and one as a reason to re-run at higher trials. */ export declare function compareContainment(weak: readonly ContainmentInput[], strong: readonly ContainmentInput[]): ContainmentVerdict; /** Human-readable verdict — the counterexamples in full, since they are the result. */ export declare function formatContainment(v: ContainmentVerdict): string; //# sourceMappingURL=trigger-containment.d.ts.map