/** * Cross-block dedup: the same bytes, sent twice, charged twice. * * WHY THIS IS THE BIGGEST REMAINING WIN, and why the per-block engines cannot * find it. Every engine here is a pure function of ONE block, so it cannot see * that the file it is compressing is the same file the agent read nine turns * ago. An agentic coding session repeats itself constantly and for good * reasons: read a file, edit it, read it back; run the tests, fix, run them * again; grep, follow a hit, grep the same pattern from a different directory. * The second copy is pure cost -- it teaches the model nothing the first copy * did not already say. * * THE REFERENT IS INSIDE THE REQUEST, WHICH IS THE WHOLE POINT. * * HeadRoom dedups too -- a content hash naturally collapses repeats -- but * their marker points OUT of the payload, at a cache entry keyed by that hash. * When the entry is missing the model receives `<> [unresolved: * entry not found]` (their #2509), a dead token mid-context. A back-reference * here points at bytes that are still in the very request being sent. It cannot * miss, because there is no lookup: the content the model needs is above. * * That is why these elisions are `lossless: true` with no `recoverAt`. The rule * is not "we hope it is retrievable" -- it is that the output alone fully * determines what was removed, which is exactly what `lossless` means. * * TWO KINDS OF REPEAT, IN ORDER OF VALUE: * * verbatim the referent was never rewritten -- it sits before the cache * frontier, or nothing claimed it -- so the later copy can be * dropped whole and never compressed at all. This is the case * that pays best: a file re-read after an edit whose earlier copy * is already in the cached prefix costs nothing on the wire. * * compressed the referent was rewritten, so the later copy is compared * AFTER compression. Identical compressed forms mean the later * block would have contributed nothing new, and the reference is * still exact -- it names bytes the model can read above. * * WHAT IS DELIBERATELY NOT DONE: near-duplicate matching. A file read before * and after an edit is NOT the same file, and the difference is precisely what * the agent is looking at. Collapsing "almost the same" would hide the edit -- * a silent, confident wrong answer, which is worse than the tokens it saves. * Only exact equality dedups here. */ import type { Elision } from './types.js'; /** * Below this a reference costs more than the repeat. * * The marker runs about 130 characters once it quotes an opening line, so the * floor is set well above it rather than at break-even: a marginal saving is * not worth asking a model to follow a pointer. */ export declare const MIN_DEDUP_BYTES = 600; /** One block on its way through a strategy. */ export interface DedupBlock { /** The text as it stands now -- already compressed, if it was going to be. */ readonly text: string; /** The text before any engine touched it. */ readonly original: string; /** * May this block be rewritten at all? False for a signed message or content * behind the cache frontier, both of which must stay byte-identical. */ readonly touchable: boolean; } export interface DedupResult { /** One text per input block, in order. */ readonly texts: readonly string[]; readonly elisions: readonly Elision[]; } export declare function dedupBlocks(blocks: readonly DedupBlock[]): DedupResult; //# sourceMappingURL=dedup.d.ts.map