/** * Content-hash-keyed memoization of Pass-1 extraction facts (change: optimize-hash-keyed-analyze). * * Pass 1 — per-file tree-sitter parsing and fact extraction — is a PURE function of * `(language, content)`. Nothing it produces depends on another file, on the clock, or on * anything outside the file's own bytes. That purity is what makes memoizing it sound: if the * bytes and the extracting code are both unchanged, re-parsing can only reproduce the answer * already on disk. * * So each file's Pass-1 output is stored keyed by * * (file path, sha256(language + content), extractor stamp) * * and a later `analyze` reuses every row whose key still matches, re-extracting only the diff. * Global (cross-file) passes are untouched — they still run over the whole merged fact set. * * ## The stamp is the soundness boundary * * A memo is only as safe as its invalidation rule. The content hash covers the INPUT; the * stamp covers the FUNCTION — every input to extraction that is not the file's own bytes: * * 1. **OpenLore's own extraction code.** A digest over the source that implements Pass 1 * (see {@link STAMP_ROOTS}), so editing an extractor invalidates the cache with no * version bump to remember. Deliberately over-broad: a root covers whole directories, so * an unrelated edit inside one costs a re-extract. That error direction is free; the * other one silently corrupts the graph. `pass1-fact-cache.test.ts` walks the real static * import closure of the extraction entry point and fails if any module it reaches is * outside these roots, so the coverage cannot rot as the analyzer grows. * 2. **The installed grammar versions.** The grammars are OPTIONAL dependencies loaded at * runtime, so the same OpenLore build extracts differently depending on which are present * and at what version — including the "grammar absent → this language yields nothing" * case, which a cache must never freeze in. * * Anything the stamp cannot see is a stale-fact bug, which is why the stamp is computed from * evidence on disk rather than from a hand-maintained constant. * * ## What is deliberately not here * * No cross-machine cache sharing and no dependency graph of derived queries: facts are * memoized at exactly one boundary — per-file extraction — because that is where the cost is * and where the purity is already proven. `--force` (or `OPENLORE_NO_FACT_CACHE=1`) bypasses * the cache entirely and then repopulates it: the always-available escape, and the reference * output the reused lane is verified against. */ import type { FileExtractResult } from './call-graph-types.js'; /** One file's Pass-1 facts as they are stored. */ export interface Pass1FactRow { filePath: string; /** sha256 over `language + '\0' + content` — a language reclassification is a miss too. */ contentHash: string; /** The serialized {@link FileExtractResult}, or `null` for "no extractor for this language". */ facts: string; } /** The Pass-1 memo, as {@link CallGraphBuilder} sees it. Production wires the EdgeStore-backed one. */ export interface Pass1FactCache { /** Cached facts for this file's exact content, or `undefined` on a miss. */ lookup(file: ExtractionInput): { facts: FileExtractResult | undefined; } | undefined; /** Record what a fresh extraction produced for this file's exact content. */ record(file: ExtractionInput, facts: FileExtractResult | undefined): void; /** Why no row could be reused at all, when that is known up front. */ readonly noReuseReason?: NoReuseReason; } /** The Pass-1 input record (kept structural so this module stays dependency-light). */ export interface ExtractionInput { path: string; content: string; language: string; } /** * Why a build reused nothing. Modeled on the extraction lane's `serialReason` for the same * reason it exists there: "reused 0" alone cannot tell an operator whether they asked for a * full re-extraction or the memo is quietly broken. */ export type NoReuseReason = 'requested' | 'no-index-yet' | 'memo-absent' | 'index-not-ready' | 'store-unreadable' | 'no-stamp'; /** How many files the last build reused vs. re-extracted — always disclosed, never silent. */ export interface Pass1CacheDisclosure { reused: number; /** Files handed to the extraction lane. */ extracted: number; /** * Files that were extracted but could NOT be memoized: the extraction threw, or it produced * no facts at all (the shape an unloadable grammar returns, which must never be persisted). * These re-extract on every later run, so `reused` will never reach the full file count * while they exist. * * Reported because the population is language-skewed and would otherwise be invisible. A * cleanly-parsed file is distinguishable from a failed parse only by the evidence it leaves * behind, and style counters — the usual evidence — are tallied only for TypeScript, * JavaScript, Python and Go. In a C-, Java- or Ruby-heavy repository, a header, an * interface-only file, or a constants file can legitimately yield nothing and so is * permanently re-parsed. That is bounded by today's cost, but an operator should be able to * tell "14 files cannot be cached" from "the memo is broken". */ uncacheable: number; /** Present only when the memo was bypassed or unavailable wholesale. */ noReuseReason?: NoReuseReason; } /** * The content key for one file. The LANGUAGE is hashed with the content because the same * bytes extract differently under a different classification (a `.h` resolved as C vs C++, * an HTML file whose inline scripts are blanked into JavaScript), and the caller's content is * the post-transform text, so the hash covers the transform too. */ export declare function factKey(file: ExtractionInput): string; /** * Serialize one file's extractor output. `undefined` (a language with no extractor) is stored * as the JSON literal `null` — a real, reusable answer, not an absence, so a repo full of * unsupported files does not re-dispatch them on every run. * * Every field is plain data: the same values already cross a `structuredClone` boundary to * reach the extraction workers, and the CFG overlay is documented to retain no AST nodes. */ export declare function serializeFacts(facts: FileExtractResult | undefined): string; /** * Rebuild one file's extractor output from a stored row. Returns a miss (`undefined` outer) * for anything unreadable or written under a different payload format — a cache is never * allowed to be the reason a build produces different facts, so every doubt re-extracts. */ export declare function deserializeFacts(raw: string): { facts: FileExtractResult | undefined; } | undefined; /** * The installed version of one package, or `undefined` if it is genuinely not installed. * * Two lookups, because one is not enough. `require.resolve('/package.json')` is the * direct route, but it obeys the package's `exports` map — and a package that does not export * `./package.json` (`web-tree-sitter` is one) throws `ERR_PACKAGE_PATH_NOT_EXPORTED` while * being perfectly well installed. Reading the manifest beside the resolved ENTRY point * recovers exactly those. A package that fails both is the one that is really absent. */ export declare function resolvePackageVersion(require: NodeRequire, name: string): string | undefined; /** * Digest the code under `roots` (resolved relative to `baseDir`). * * Deliberately position-independent: a file contributes its path RELATIVE to `baseDir` plus * its content, so the same code installed at two different absolute locations digests * identically — otherwise every user's cache would key on their own home directory. * * Never throws. A root that does not exist contributes nothing (a layout that does not ship * it), and a file that cannot be read contributes an explicit unreadable marker rather than * vanishing — a file silently dropping out of the digest would make the stamp LESS specific, * which is the one direction a cache key must never move. */ export declare function digestStampRoots(baseDir: string, roots: readonly string[]): string; /** * The extractor version stamp for this process: a digest over the extraction code and the * installed grammars. Computed once — it cannot change while the process runs, and every * analyze in a long-lived daemon would otherwise re-read the same few hundred files. * * Never throws: an unreadable root or package simply contributes nothing to the digest. The * failure mode of a partial stamp is over-reuse, so anything that makes the stamp LESS * specific must be impossible — hence the roots are read whole, and a read error on an * individual file is folded in as an explicit marker rather than skipped silently. */ export declare function computeExtractorStamp(): string; /** Test-only: recompute the stamp on the next call. */ export declare function __resetExtractorStampForTests(): void; /** The stamp roots, for the import-closure coverage test. */ export declare const __STAMP_ROOTS_FOR_TESTS: string[]; /** The grammar packages folded into the stamp, for the coverage test. */ export declare function __grammarPackageNamesForTests(moduleDir: string): string[]; /** The subset of the graph store this cache needs — narrow so tests need no SQLite. */ export interface Pass1FactStorage { getPass1Facts(filePath: string, contentHash: string, stamp: string): string | undefined; } /** * The production cache: reads through to the graph store, buffers writes. * * Writes are buffered rather than written through because the store handle that may WRITE * only exists at the end of an analyze (the same handle that rebuilds the graph). Opening a * second write handle earlier would widen the window in which a concurrent reader sees a * half-rebuilt store, which is a worse trade than holding the rows in memory — and on the * incremental runs this change exists for, the buffer holds only the diff. */ export declare class BufferedPass1FactCache implements Pass1FactCache { private readonly storage; private readonly stamp; /** * Why this cache will reuse nothing, when that is decided before the build. Present * exactly when reads are off (`--force`/the env escape) or there is no readable store; * absent means the memo is live and any miss is a per-file miss. */ readonly noReuseReason?: NoReuseReason | undefined; private readonly writes; /** * Set once a read throws. A store that cannot answer one lookup cannot answer the next * thousand either — most often it has no memo table at all (an index materialized from a * bundle, or one built before this feature), and occasionally it is locked. Latching stops * the build from raising and swallowing one exception per file for an answer already known. */ private storageUnusable; constructor(storage: Pass1FactStorage | null, stamp: string, /** * Why this cache will reuse nothing, when that is decided before the build. Present * exactly when reads are off (`--force`/the env escape) or there is no readable store; * absent means the memo is live and any miss is a per-file miss. */ noReuseReason?: NoReuseReason | undefined); lookup(file: ExtractionInput): { facts: FileExtractResult | undefined; } | undefined; record(file: ExtractionInput, facts: FileExtractResult | undefined): void; /** * Take the rows to persist, and the stamp they were computed under. The buffer is emptied: * these rows can be tens of megabytes on a full build, and the caller holds them only until * the graph write. Calling twice yields the second call nothing, which is the honest * behavior for a hand-off. */ take(): { stamp: string; rows: Pass1FactRow[]; }; } /** * One line naming what the Pass-1 memo did — always rendered when a memo was consulted, so * the lane is never silent. When nothing was reused, the CAUSE is named: "reused 0" on its * own cannot distinguish an operator who asked for a full re-extraction from a memo that is * quietly unavailable, and only one of those is worth acting on. */ export declare function describePass1Cache(d: Pass1CacheDisclosure | undefined): string | undefined; /** True when the environment escape hatch is set. */ export declare function factCacheDisabledByEnv(): boolean; //# sourceMappingURL=pass1-fact-cache.d.ts.map