/** * Saturation curve analysis — answers "have I run enough samples?" * * Why this exists * --------------- * The most common omk-user question after "is the difference significant?" * is "is N=30 enough, or should I keep running?". Without a principled * answer, users either over-pay (running 200 when 50 was enough) or * under-pay (calling skill effects null at N=30 when N=80 would have * shown clearly significant — Bootstrap CI just hasn't * converged yet). * * Saturation analysis fits the right tool to the question: track a * convergence-of-evidence metric across cumulative sample counts; when * the metric stops moving meaningfully, you've saturated. Three metric * choices are exposed because each has a failure mode: * * - **slope** (simplest): mean's rate of change between consecutive * checkpoints. Easy to explain. Fragile to outlier samples. * - **bootstrap-ci-width** (default): CI shrinks as O(1/√N); when its * decay rate flattens, more samples buy little. Statistically * grounded; pairs naturally with the Bootstrap CI module. * - **plateau-height**: range of mean across the last K checkpoints. * Conservative — slow to declare saturation, hard to fool. * * The function never says "saturated" off a single observation: a 3- * window run-of-success is required. This guards against random dips * that look like convergence but aren't. */ export type SaturationMethod = 'slope' | 'bootstrap-ci-width' | 'plateau-height'; /** Default consecutive-window run length required before declaring saturation. */ export declare const DEFAULT_SATURATION_WINDOW_SIZE = 3; /** * Default `bootstrap-ci-width` cutoff: declare saturation when the relative * CI-width shrink per checkpoint stays under 5%. Single source of truth for * the documented threshold; guarded by `doc-constants-drift.test.ts`. */ export declare const DEFAULT_CI_WIDTH_SHRINK_THRESHOLD = 0.05; /** * One observation in a saturation curve. `n` is the cumulative sample * count at this checkpoint; `mean` and (optional) `ciWidth` come from * applying the chosen metric to all samples up to and including `n`. */ export interface SaturationCheckpoint { n: number; mean: number; ciWidth?: number; } export interface SaturationResult { /** True only after threshold satisfied for `windowSize` consecutive transitions. */ saturated: boolean; /** Sample count at which saturation was first declared. null if not saturated. */ atN: number | null; /** 'high' for ≥ 50 cumulative samples; 'medium' for [20, 50); 'low' below 20. */ confidence: 'high' | 'medium' | 'low'; /** Method that produced the verdict. */ method: SaturationMethod; /** Threshold used. */ threshold: number; /** Per-checkpoint metric trace, useful for plotting. */ trace: Array<{ n: number; metric: number; }>; /** Human-readable explanation of why we said yes / no. */ reason: string; } /** * Compute saturation from a sequence of cumulative score arrays. * * `cumulativeScores[i]` is the array of all scores observed by checkpoint i * (the i-th element is the LATEST cumulative slice; arrays grow). Each slice * is fed to bootstrapMeanCI to get a (mean, CI) pair. The metric trajectory * is then evaluated by the chosen method. * * @param cumulativeScores Cumulative score arrays in chronological order. * @param method Detection algorithm (default 'bootstrap-ci-width'). * @param threshold Method-specific cutoff (defaults match the method). * @param windowSize Number of consecutive transitions that must all * satisfy the threshold (default 3). * @param bootstrapSamples Used only by 'bootstrap-ci-width' (default 1000). * @param seed Optional seed for reproducible CIs. */ export declare function findSaturationPoint(cumulativeScores: number[][], method?: SaturationMethod, threshold?: number, windowSize?: number, bootstrapSamples?: number, seed?: number): SaturationResult; /** * Build cumulative score arrays from a per-run flat list. `runs[i]` is the * array of per-sample scores from the i-th repeat. Returns an array where * the j-th entry contains all scores from runs 0..j (concatenated). * * Use this when feeding `runMultiple` results to `findSaturationPoint`. */ export declare function buildCumulativeScores(runs: number[][]): number[][];