/** * MastraCode Outcome Scorer (Code-based, Agent type, always-on) * * Grades the objective outcome of a coding session by checking hard facts: * - Did the build/typecheck pass? * - Did tests pass? * - What was the tool error rate? * - Did the agent get stuck in a loop? * - Was there a regression (build passed earlier, failed later)? * - Did the agent use ask_user unnecessarily? (autonomy) * * This scorer is always-on (no sampling) because it's cheap — it only * reads exit codes and error states from tool results already in the messages. * * Scoring philosophy (from Anthropic's eval guidance): * "Grade outcomes, not paths" — we check what happened, not how. * Partial credit: a session that passes build but fails tests scores * higher than one that fails build entirely. * * Dimensions that don't apply to a session (e.g. build for a docs-only task) * are excluded from the weighted average, so sessions aren't penalized for * irrelevant criteria. */ type DimensionResult = { score: number; detail: string; applicable: boolean; }; export declare function createOutcomeScorer(): import("@mastra/core/evals").MastraScorer<"mastracode-outcome", import("@mastra/core/evals").ScorerRunInputForAgent, import("@mastra/core/evals").ScorerRunOutputForAgent, Record<"preprocessStepResult", { empty: true; build: DimensionResult; tests: DimensionResult; toolErrors: DimensionResult; loops: DimensionResult; regression: DimensionResult; autonomy: DimensionResult; totalCalls: number; } | { empty: false; build: DimensionResult; tests: DimensionResult; toolErrors: DimensionResult; loops: DimensionResult; regression: DimensionResult; autonomy: DimensionResult; totalCalls: number; }> & Record<"generateScoreStepResult", number> & Record<"generateReasonStepResult", string>>; export {}; //# sourceMappingURL=outcome.d.ts.map