import { type RuntimeCommandRunner } from "./runtime-process.ts"; /** * Managed runtime for needle (https://github.com/cactus-compute/needle), a 26M-parameter single-shot * function-calling model. Needle exposes no server — the only integration surface is CLI/library * invocation — so this module owns clone+build of a pinned commit into a contained venv, base * checkpoint download, and one-shot `needle run` invocation/parsing. Mirrors local-runtime.ts's * OllamaRuntime/TransformersRuntime and llamacpp-runtime.ts's PrismLlamaCppRuntime: injectable seams * for exists/fetch/run-command, pi-owned directories under agentDir (runtimes/, models/), onProgress * as best-effort UI feedback, and honest error taxonomies instead of silent fallbacks. * * Investigation findings (verified against a real clone + real venv + real CPU inference run, not * just the README): * * - `needle run` (the CLI's single-inference subcommand) takes a REQUIRED `--checkpoint` and does * NOT auto-download one — unlike `needle playground`/`finetune`, which resolve a checkpoint * internally. The pretrained base checkpoint (`needle.pkl`) lives on `Cactus-Compute/needle` on * Hugging Face and is fetched directly by {@link NeedleRuntime.downloadWeights} (never invoked * implicitly by installManaged/runFunctionCall — this module never silently starts a large * download on the caller's behalf). * - SECURITY (owner-flagged, verified during investigation): `Cactus-Compute/needle` on Hugging Face * publishes THREE weight artifacts — `needle.pkl` (52,633,098 bytes), `model.safetensors` * (reported as a different param count on the HF model card, 30.4M vs the 26,315,421 a real * `needle run` printed — likely not even the same checkpoint), and `needle-cq4.zip`. I grepped the * ENTIRE needle package (cli.py, model/run.py, model/export.py, model/quantize.py, * training/{finetune,pretrain,train}.py, utils/distributed.py, ui/server.py — every `pickle`/`.zip` * hit in the repo) and confirmed every checkpoint load/save path in the codebase — including the * one `needle run` uses — is `pickle.load`/`pickle.dump` on a `.pkl` file. There is NO safetensors * reader anywhere in the package; `model.safetensors` is not consumed by any code this module * drives. `needle-cq4.zip` ("Cactus Quantized 4-bit"?) isn't referenced by any code here either — * the README states production inference runs on the separate Cactus C++ engine * (cactus-compute/cactus), which very plausibly consumes its own format; that repo was not audited * here. Bottom line: **this module has no choice but to fetch and deserialize `needle.pkl` via * Python's `pickle` module, which executes arbitrary code by construction** — there is no * safetensors alternative available in the code path we drive. Given that, {@link * NeedleRuntime.downloadWeights} pins the exact sha256 ({@link NEEDLE_WEIGHTS_SHA256}) and byte * count ({@link NEEDLE_WEIGHTS_BYTES}) of the file as verified on 2026-07-18 (direct `sha256sum` of * a real download, cross-checked against Hugging Face's own `x-linked-etag`/`x-linked-size` * headers on the resolve redirect, at repo commit `5f89b4307696d669c3df1d38ae057e6e1728b107`) and * VERIFIES every download against that pin, refusing to keep any file that doesn't match — this * catches a compromised/tampered/swapped artifact but does NOT make loading a pickle file safe in * the abstract; the actual `pickle.load` execution happens entirely inside needle's own Python * process, outside this module's control. The owner should decide whether serving this model at * all is acceptable given upstream's format choice. * - The upstream `setup` script does more than a contained managed install can honor: it can `sudo * apt-get install` a newer Python or `python3-venv`, `sudo modprobe` TPU kernel modules, toggle * transparent hugepages via a root-owned sysfs write, and interactively prompts for a * WANDB_API_KEY. None of that is available under containment (no writes outside the pi-owned dir, * no sudo, no interactive prompts, no global installs) and none of it is required for the * documented Mac/PC single-inference quickstart path. {@link NeedleRuntime.installManaged} instead * replicates only the containable core: `python3 -m venv`, `pip install -e ` (the * script's own `pip install -e . -q`), and the CPU (or CUDA, when an NVIDIA GPU is detected) JAX * backend install. TPU support and the sudo-gated OS package/kernel-module steps are intentionally * NOT replicated. * - `needle run`'s console-script entry point (`/bin/needle`, installed by * `pip install -e .` from the `[project.scripts]` table) is the only way to invoke the CLI: * `needle/cli.py` has no `if __name__ == "__main__":` guard, so `python -m needle.cli` silently * does nothing — the installed console script is required. * - A verified real stdout capture of `needle run --checkpoint --query "..." --tools '[...]'` * is multi-line diagnostic prose followed by the tool call: * ``` * Loading checkpoint: * Downloading pretrained tokenizer from HuggingFace... (first run only — cached after) * Model parameters: 26,315,421 * * Query: * Tools: ... * * [{"name":"get_weather","arguments":{"location":"SanFrancisco"}}] * ``` * The `` marker is a literal SentencePiece user-defined-symbol token, always emitted as * the first generated token per the model's own answer format. {@link NeedleRuntime}'s stdout * parser locates that marker and strictly `JSON.parse`s everything after it — never guessing a * result from unparseable output. * - Known upstream quirk (verified against a live run with a camelCase tool name): `needle run`'s CLI * wrapper (`main()` in `needle/model/run.py`) streams tokens straight to stdout and never applies * the tool-name restoration its own `generate()` return value receives — a tool name that wasn't * already snake_case (e.g. `getWeatherNow`) is reported back snake-cased (`get_weather_now`), not * in its original casing. This is a limitation of the upstream CLI, not something patched here. * - Pinned commit `ffb1c5144c5a16cb8ec650dbc8a6f6fd3854f8f2` was captured via a real `git clone` of * the repo's default branch HEAD on 2026-07-18 (the repo's own history shows no pushes since * 2026-07-01, so this is not stale). A plain `git clone --depth 1` only reliably lands on a pinned * commit when it still happens to be the remote's current tip; {@link NeedleRuntime.installManaged} * instead fetches the exact commit SHA directly (`git fetch --depth 1 origin `), which GitHub * serves for any reachable commit on a public repo, so the pin holds even after upstream moves on. */ export declare const NEEDLE_REPO_URL = "https://github.com/cactus-compute/needle.git"; /** * Pinned needle commit — captured via a real `git clone` of the repo's default-branch HEAD on * 2026-07-18 (`git ls-remote https://github.com/cactus-compute/needle HEAD` resolves to the same * SHA). Bump here (re-verify with the same command) when a newer commit is needed. */ export declare const NEEDLE_PINNED_COMMIT = "ffb1c5144c5a16cb8ec650dbc8a6f6fd3854f8f2"; /** * Pinned integrity for `needle.pkl` — see the module-level SECURITY note: this is a pickle file * (arbitrary code execution on load by construction), so every download is verified against this * exact sha256+size before being kept. Captured 2026-07-18: `sha256sum` of a real download matched * Hugging Face's `x-linked-etag` header on the resolve redirect at repo commit * `5f89b4307696d669c3df1d38ae057e6e1728b107`. Bump both together (re-verify the same way) only for a * deliberate, reviewed upstream weights update. */ export declare const NEEDLE_WEIGHTS_SHA256 = "40a32e91d1d4197bf15ba559b74f6727c342dc8746918742fc7d8e2c1f18df40"; export declare const NEEDLE_WEIGHTS_BYTES = 52633098; export interface NeedleDetectResult { installed: boolean; installDir?: string; commit?: string; pythonAvailable: boolean; checkpointPresent: boolean; } export interface NeedleFunctionCallRequest { query: string; /** A JSON-serializable tool schema value (never a pre-stringified JSON string) — serialized * exactly once by {@link NeedleRuntime.runFunctionCall}, never by the caller. */ tools: unknown; } export interface NeedleFunctionCall { name: string; arguments: Record; } export type NeedleFunctionCallResult = { ok: true; call: NeedleFunctionCall; } | { ok: false; error: string; rawOutput: string; }; export interface NeedleSmokeTestResult { ok: boolean; latencyMs: number; call?: NeedleFunctionCall; error?: string; } export interface NeedleWeightsDownloadResult { ok: boolean; path?: string; skipped?: boolean; error?: string; } interface NeedleWeightsIntegrity { sha256: string; bytes: number; } export interface NeedleRuntimeDeps { existsFn?: (path: string) => boolean; fetchFn?: typeof fetch; /** Runs git/pip/python commands. Injectable so installManaged/runFunctionCall's orchestration is * testable without a real clone/venv/inference. Defaults to a real spawn-and-collect-output * runner (installs can run long; a single inference call is comparatively quick but still shells * out to a real interpreter). */ runCommand?: RuntimeCommandRunner; /** Whether a named command exists on PATH (git, python3, nvidia-smi). */ hasCommand?: (command: string) => boolean; hasNvidiaGpu?: () => boolean; platform?: () => string; /** Expected weights sha256+size. Defaults to {@link NEEDLE_WEIGHTS_SHA256}/{@link * NEEDLE_WEIGHTS_BYTES} — the real pinned values. Overridable ONLY so tests can exercise the full * download+verify path with small synthetic content instead of the real 52MB pickle; this is a * constructor-level seam for a trusted caller, the same trust boundary every other injectable dep * in this class already relies on (fetchFn/runCommand could equally be used to bypass anything). */ weightsIntegrity?: NeedleWeightsIntegrity; } export declare class NeedleRuntime { private readonly _agentDir; private readonly _exists; private readonly _fetch; private readonly _runCommand; private readonly _hasCommand; private readonly _hasNvidiaGpu; private readonly _platform; private readonly _weightsIntegrity; constructor(args: { agentDir: string; deps?: NeedleRuntimeDeps; }); runtimeDir(): string; modelsDir(): string; checkpointPath(): string; private _srcDir; private _venvDir; private _pythonPath; /** The installed `needle` console script (from `[project.scripts]`, written by `pip install -e`) * — the only invokable entry point, since `needle/cli.py` has no `__main__` guard. */ private _entryPath; private _manifestPath; private _readManifest; private _writeManifest; detect(): Promise; /** * Clone the pinned needle commit and build a contained, pi-owned runtime (consent-gated by the * caller, same contract as OllamaRuntime#installManaged / PrismLlamaCppRuntime#installManaged — * this method only does the mechanical clone+build+verify and reports the outcome honestly). See * the module docstring for exactly which upstream `setup` steps are and are not replicated under * containment. */ installManaged(onProgress?: (status: string) => void): Promise<{ ok: boolean; error?: string; }>; /** * A plain `git clone --depth 1` only lands on {@link NEEDLE_PINNED_COMMIT} when it still happens * to be the remote's current default-branch tip. Fetching the exact SHA directly instead (verified * against the real repo) pins reliably even after upstream moves on: GitHub serves any reachable * commit for a public repo via `git fetch `, not just refs. */ private _cloneAtPinnedCommit; /** * Download the pretrained base weights (`needle.pkl`). `needle run` takes an explicit * `--checkpoint` and does NOT auto-download one (unlike `needle playground`/`finetune`) — see the * module docstring. Never called implicitly by installManaged/runFunctionCall; the caller decides * when to pull weights. * * `needle.pkl` is a pickle file — arbitrary code execution on load by construction (see the * module-level SECURITY note) — so every download is verified against the pinned sha256+size * ({@link NEEDLE_WEIGHTS_SHA256}/{@link NEEDLE_WEIGHTS_BYTES}, overridable only via the * `weightsIntegrity` test seam) computed in the same streaming pass that writes the file, with the * partial/mismatched file deleted on ANY failure — this never leaves an unverified or wrong pickle * sitting where {@link runFunctionCall} would load it. */ downloadWeights(onProgress?: (status: string) => void): Promise; /** * Run one single-shot inference through `needle run --checkpoint ... --query ... --tools ...` * inside the contained venv's installed console script. Argv is passed as an array to the * injected `runCommand` seam — never a shell string — so query/tools text containing quotes, * spaces, or other shell-hostile characters is passed through literally (verified against a real * invocation; see the module docstring). `tools` is serialized with `JSON.stringify` exactly once. */ runFunctionCall(request: NeedleFunctionCallRequest, options?: { checkpointPath?: string; }): Promise; /** * Strictly parse a real `needle run` stdout capture: * ``` * Loading checkpoint: * Model parameters: * * Query: * Tools: ... * * [{"name":"...","arguments":{...}}] * ``` * (diagnostic lines above the marker vary — e.g. a one-time "Downloading pretrained tokenizer..." * line on a cold cache — so parsing locates the `` marker rather than assuming a fixed * line count/position). Unparseable output is always an error carrying the raw text, never a * guessed result. */ private _parseFunctionCall; /** Canned get_weather query/tool through {@link runFunctionCall}; the wiring task uses this as the * post-install verification step. */ smokeTest(onProgress?: (status: string) => void): Promise; /** * No-op: every needle invocation here is a single-shot `runCommand` call that spawns, waits for * exit, and returns — there is no persistent/detached child process this runtime owns between * calls for dispose() to track or kill (contrast OllamaRuntime/PrismLlamaCppRuntime's long-lived * `serve()` child). */ dispose(): void; } export {}; //# sourceMappingURL=needle-runtime.d.ts.map