/** * ScrapeThrottle — per-engine scrape rate window + proxy-USD ledger, * SEPARATE from the governor USD ledger, SAME mutex + atomic-write + * fail-closed discipline as `SeoQuotaGovernor` (spec-20260718-in-house- * ai-visibility, Sprint 9; arch ScrapeThrottle §208-220 / ADR-3). * * The scrape arm's `estCostUsdPerPrompt` is booked as $0 to the * `SeoQuotaGovernor` — its real proxy cost is tracked HERE, in an * independent, independently-capped filesystem ledger keyed by engine * string at `.bober/seo/scrape-throttle-ledger.json` (a DISTINCT path * from `.bober/seo/quota-ledger.json`). The two ledgers are never * cross-reconciled (arch §320). * * `acquire(engine)` both DECIDES and CONSUMES a rate slot in one * read-modify-write under the shared per-path mutex (`withLedgerLock`, * reused from `quota-ledger.ts` — path-generic, so this ledger gets its * own serialization chain and cannot interfere with the governor's). * It writes ONLY on the granting branch — a refusal has no side effect, * mirroring `SeoQuotaGovernor.admit()`. * * `recordProxyCost(engine, usd)` is the governor's `record()` twin: * heals a corrupt ledger back to `{}`, accrues a NaN/negative-guarded * USD amount, and writes atomically (temp-file + rename) so a crash * mid-write never leaves a torn file. * * Fail-closed: a corrupt/unreadable ledger makes `acquire` treat proxy * spend as `+Infinity` (mirrors the governor's Infinity-on-corrupt * refuse) and deny with `reason: "proxy-budget"`. A MISSING ledger * (first run) is a fresh `{}` — allow. * * The clock is INJECTED (`() => ISO string`, mirrors * `DamcrawlerCrawlEngine.now`), never `Date.now()` — deterministic * rate-window tests drive it across the window boundary explicitly. * * ISOLATED sprint: no scrape-provider wiring (Sprint 10), no config- * schema field (caps come from the constructor), no egress coupling. */ /** Persisted at `.bober/seo/scrape-throttle-ledger.json` — keyed by engine string. */ export type ScrapeProxyLedger = { [engine: string]: { windowStart: number; count: number; proxyUsdSpent: number; }; }; /** `acquire()` decision. Note `proceed`, NOT `allowed` — per contract sc-9-1. */ export type ThrottleDecision = { proceed: true; } | { proceed: false; reason: "rate-window" | "proxy-budget"; }; /** Caps injected via constructor — no config-schema field this sprint (isolation). */ export type ScrapeThrottleLimits = { /** Max `acquire()` grants per rolling fixed window, per engine. */ maxPerWindow: number; /** Fixed-window length in milliseconds. */ windowMs: number; /** Max cumulative `proxyUsdSpent` per engine before `acquire` denies. */ maxProxyUsd: number; }; export declare class ScrapeThrottle { private readonly ledgerPath; private readonly limits; private readonly now; constructor(ledgerPath: string, limits: ScrapeThrottleLimits, now?: () => string); /** * Decide whether `engine` may scrape now, consuming a rate slot on grant. * Read-modify-write under the per-path mutex; writes ONLY on the granting * branch (a refusal has no side effect on the ledger, mirroring * `SeoQuotaGovernor.admit()`/`record()` no-write-on-refuse). */ acquire(engine: string): Promise; /** * Persist completed proxy spend for `engine`. Heals a corrupt ledger back * to `{}` (corruption blocks `acquire`, not `recordProxyCost` — mirrors * `SeoQuotaGovernor.record()`). Guards NaN/negative/zero cost so a bad * value can never corrupt or reduce the running total. Concurrent calls * sharing `ledgerPath` never lose an update (per-path mutex + read-fresh- * inside-lock + atomic write). */ recordProxyCost(engine: string, usd: number): Promise; } //# sourceMappingURL=scrape-throttle.d.ts.map