# Pi magic 8-ball: contract and design ## Contents - [Design](#design) - [Contract](#contract) - [Evidence and usability improvements](#improvements) - [Optional transcripts](#transcripts) - [Builder reasoning and evidence discovery](#discovery) - [Verification and remaining limits](#verification) - [Advisor-derived reliability changes](#reliability) - [Offline verification](#reliability-verification) - [Review regression contract](#review-regressions) - [Publication preparation](#publication) - [Supplemental evidence and usage guidance](#supplemental-evidence) - [Release 0.1.1](#release-0.1.1) - [Incremental builder state](#incremental-state) - [Release 0.1.2](#release-0.1.2) - [Shared behavior release](#shared-behavior-release) ## Design {#design} V1 adds one advisory choice tool. It uses a separate chat-model loop to collect neutral state, then Pi's classifier API. Jev and Clef use the same backend boundary. Model selection stays in extension-owned settings files and user-issued slash commands, not model-issued arguments. The builder has no inherited extensions, filesystem writes, shell, web, or classifier access. Its transcript stays in memory. The parent session stores returned state and results through normal Pi tool-result persistence. Configured providers receive permitted context and state. Read scopes restrict collection, not provider retention. Users must select providers suitable for their data. Prompt instructions request neutrality. Schema validation cannot prove neutrality, factual accuracy, completeness, or secret removal. The builder has no fixed request or tool-call ceilings. Each builder request retains a 2048-token output hint. The invocation deadline defaults to 120000 milliseconds; REQ-21 defines configuration. These ceilings bound work, not monetary cost or confidence calibration. The extension makes no application-level retries or model fallbacks. Conversation input holds at most 24000 UTF-8 bytes. A workspace result holds at most 16000 UTF-8 bytes. State holds at most 12000 UTF-8 bytes. Truncation is explicit. Oversized or malformed final state fails closed. Git execution, research, boolean questions, and score questions are outside V1. ## Contract {#contract} `magic8ball({ question, responses, abstain?, context? })` returns a structured success or error. `context` has boolean `conversation` and `workspace` fields. Both default to true. Optional `context.files` supplies workspace-relative file hints. REQ-12 defines their validation. Optional `context.supplied` supplies bounded caller evidence to the builder. REQ-25 defines its validation and source IDs. Unknown fields and non-booleans are invalid. REQ-22 permits optional diagnostics boolean without sending it to either model. `abstain` defaults to true. Explicit false disables the added `insufficient_evidence` response. Both configured models require provider and model identifiers. The extension never uses the parent model as a fallback. User settings live in `/magic8ball.json`, normally `~/.pi/agent/magic8ball.json`. Trusted project settings live in `/.pi/magic8ball.json`. Each role is a complete `{ "provider": "...", "model": "..." }` pair. A project role overrides the entire user role. Partial pairs do not inherit individual fields. Untrusted project files are not read. The four former `PI_MAGIC8BALL_*` model-selection variables are ignored. Provider credentials remain in Pi's authentication system. `/magic8ball` opens an interactive two-model picker. `/magic8ball show` displays settings and their sources. `/magic8ball builder ` and `/magic8ball classifier ` update one role. A single `--project` or `--global` flag may precede or follow command arguments; saves default to user scope. In noninteractive mode, `/magic8ball` displays usage and settings without writing. Picker cancellation never writes. **REQ-1** The tool MUST validate inputs before model calls. The question and every response description MUST be nonempty strings after trimming. Response identifiers MUST match `[A-Za-z][A-Za-z0-9_]{0,63}` and exclude `__proto__`, `prototype`, `constructor`, and `insufficient_evidence`. The caller MUST supply 2–25 responses with abstention, or 2–26 without it. The tool MUST preserve identifiers and descriptions. The serialized request with only `context.supplied` omitted MUST contain at most 16000 UTF-8 bytes. REQ-25 defines separate supplied-evidence limits. Requests without that field MUST retain the existing size check. Invalid domain inputs MUST return `invalid-input` without model calls. Pi's schema-validation error takes precedence when arguments fail the registered parameter schema before the extension executes. **AC-1** Changed behavior. Valid inputs retain caller criteria and add abstention by default. Explicit false permits 26 caller choices. Empty descriptions, reserved identifiers, invalid counts, unknown fields, and oversized requests produce `invalid-input` before model calls. **REQ-2** The builder MUST run before classification with only permitted conversation, workspace, and caller-supplied evidence under REQ-25. It MUST receive neutral-state instructions and the five local state tools in REQ-27. Only the three workspace evidence tools MAY be declared when workspace scope is true. REQ-28 defines completion from owned state. REQ-29 replaces count-based finalization. It MUST NOT receive classifier output from this invocation or prior `magic8ball` tool results. Conversation extraction MUST retain ordinary assistant text even when the same message contains a `magic8ball` tool call. It MUST omit tool-call blocks, thinking blocks, images, system messages, and `magic8ball` or `magic8ball_*` tool results. Final state MUST contain exactly `goal`, `constraints`, `current_state`, `evidence`, and `uncertainties`. Evidence entries MUST contain exactly nonempty `fact` and `source` strings. Other collections MUST contain only nonempty strings. The tool MUST reject candidate-keyed state, extra fields, or state exceeding 12000 bytes as `invalid-state`. These checks apply to owned state and tool arguments. Final builder prose MUST NOT supply state. **AC-2** Changed behavior. Mock builder and classifier calls show state precedes classification and no classifier feedback reaches the builder. Workspace-disabled requests expose no evidence tools. Conversation-disabled requests expose no prior messages. Valid state reaches classification unchanged. Extra recommendation fields, candidate-keyed state, malformed JSON, and oversized state fail before classification. **REQ-3** The three evidence tools MUST restrict paths to the real workspace root and reject symlinks and special files. They MUST reject dot-prefixed path segments, `node_modules`, and conventional credential files, including PEM, key, and P12 files. Listing MUST omit denied paths. Read and literal search MUST operate on one regular UTF-8 text file per call. REQ-19 permits an absent-path observation without opening a file. This exception does not permit content inspection of restricted or nonregular paths. Read offset defaults to 1. Read limit defaults to 200 and MUST be in the inclusive range 1–200. List results MUST contain at most 200 names. Search results MUST contain at most 200 matching lines. File content inspection MUST read at most 16000 bytes per call. Truncation MUST be explicit. The builder MUST call evidence tools through `ctx.executeTool` so Pi's validation and permission hooks remain active. These tools MUST NOT write files, execute workspace code, or invoke other tools. **AC-3** Changed behavior. Temporary-workspace tests confirm bounded read, list, and literal search. Traversal, external paths, symlinks, hidden credential paths, key files, binary files, and special files fail without exposing content. Builder tests show only allowlisted evidence names reach the nested-tool boundary. Permission failures stop classification. **REQ-4** The classifier adapter MUST use the configured catalog entry and Pi's `choice` interface for both Jev and Clef. A success MUST return `answer`, `probabilities`, backend `confidence`, `abstained`, `advisory: true`, state, models, collection metadata, and usage. The distribution MUST contain exactly the supplied identifiers, including added abstention. Every probability and confidence MUST be finite and in [0,1]. The probability sum MUST differ from 1 by no more than 0.000001, except for the four-decimal allowance below. If every probability p equals Math.round(p * 10000) / 10000, the adapter MUST also accept integer-unit drift within the rounding bound. Let n be the number of supplied probabilities, including added abstention, and U be the sum of Math.round(p * 10000). The four-decimal allowance MUST accept exactly when 2 * abs(U - 10000) <= n. This bounds sum error to 0.00005 * n, inclusive. Non-grid values outside the original 0.000001 tolerance MUST still fail. The adapter MUST NOT round values to qualify for the allowance. The selected identifier MUST have maximal returned probability. The adapter MUST preserve the backend's choice among tied maxima. The adapter MUST NOT renormalize probabilities, substitute confidence, retry classification, or add a model fallback. A malformed answer MUST return `invalid-answer`. Classifier failures MUST return `classifier-failed` without a decision. **AC-4** Changed behavior. Mock Jev and Clef entries receive identical typed questions and state. Successful results preserve all probabilities and confidence, including confidence different from winning probability and tied choices. Missing, extra, negative, nonfinite, nonmaximal, or out-of-tolerance answers fail. AC-24 defines the four-decimal rounding allowance. Abstention is reported without becoming an execution command. **AC-24** Changed behavior. Replay the live Clef probabilities 0.9816, 0.0063, 0.0081, and 0.0039 through the actual registered tool with mocked providers. PASS if the result succeeds, selects the maximal backend choice, and preserves all probabilities, backend confidence, known usage, and usage completeness. For n = 2, 3, 4, and 26, check four-decimal unit drift of both signs at floor(n / 2) and one unit beyond. PASS if the boundary succeeds unchanged and the next unit fails as invalid-answer. A backend choice among rounded tied maxima MUST remain unchanged. Preserved behavior. Non-grid values outside 0.000001 sum tolerance, including a perturbation adjacent to a grid value, MUST fail. Full-precision values within the original tolerance MUST remain valid. Every k / 10000 for integer k from 0 through 10000 MUST satisfy grid recognition. Invalid keys, ranges, nonfinite values, confidence, and nonmaximal choices MUST fail, including replies with otherwise acceptable rounded sums. No provider retry, probability repair, confidence substitution, or model fallback is permitted. Tests use mock providers and no live settings writes. - #REQ-4 verified-by: [#AC-24](#AC-24) **REQ-5** The orchestrator MUST stop on cancellation, timeout, builder failure, permission failure, or exceeded context bounds under REQ-29. It MUST propagate one combined abort signal to builder requests, evidence tools, and classification. It MUST return an error without a fabricated answer and retain usage reported by completed model calls, including failed calls. A valid effective settings object that lacks either model role MUST produce `not-configured` before builder requests. An unavailable catalog entry MUST produce `model-unavailable`. Invalid settings and access failures are governed by REQ-7. The builder MUST be a physical chat-model entry. A Pi virtual-model entry MUST produce `model-unavailable` before model requests. No application-level retry, automatic repair, hidden model fallback, background process, or separate transcript file is permitted. REQ-19 permits a verified, in-bounds `path-not-found` observation. This is not a failed tool call. The model MAY use that observation to select subsequent evidence calls within the existing ceilings; the orchestrator MUST NOT retry automatically. Provider-level retry suppression uses `maxRetries: 0` where supported. Provider adapters can have their own transport behavior. **AC-5** Changed behavior. Fault-injection tests exercise missing configuration, unavailable models, builder and classifier failures, cancellation, timeout, and budget exhaustion. A configured virtual builder model produces `model-unavailable` before any builder or classifier call. No classifier call follows a builder failure. No request or tool call exceeds its ceiling. Completed-call usage survives errors. **REQ-6** The package MUST register `magic8ball` as a model-only tool with a structured output schema. Evidence tools MUST be callable nested tools, inactive by default, with read-only annotations. The main tool description and each success MUST identify decisions as advisory and confidence as distribution concentration, not correctness probability. The extension MUST work without terminal UI and MUST return aggregate nested-model usage in Pi's tool result. **AC-6** Changed behavior. Load the extension against installed Pi without network calls or UI. Inspect registration, schemas, tool annotations, advisory guidance, structured success and error results, and Pi usage accounting. Run mocked model calls through the registered main and evidence tools. Verify host-selected Jev and Clef paths. **REQ-7** Model selection MUST come only from `magic8ball.json` in Pi's agent directory and, when `ctx.isProjectTrusted()` is true, the current workspace's `.pi` directory. Settings MUST contain only optional `builder` and `classifier` roles plus optional root timeoutMs as defined by REQ-21. Each present role MUST contain nonempty `provider` and `model` strings with no whitespace; each identifier MUST contain at most 512 UTF-8 bytes. Only the builder role MAY also contain optional `reasoning`, as specified in REQ-18. No other fields are permitted. Project settings MUST replace complete roles, not merge individual fields. A missing file MUST act as an empty settings layer. Each invocation MUST read a fresh settings snapshot before resolving either model. Missing effective roles MUST return `not-configured` before model requests. Malformed JSON, invalid fields, or files exceeding 16000 UTF-8 bytes MUST return `invalid-config`; inaccessible files, symlink targets or immediate parent directories, and nonregular files MUST return `settings-unavailable`. Settings failure MUST NOT fall back to environment variables, the parent model, or another settings layer. Untrusted project files MUST be ignored without opening them. **AC-7** Changed behavior. Temporary-file tests verify user settings, complete project-role precedence, missing roles, and fresh reads after manual edits. Old model-selection environment variables do not configure an empty settings file. Untrusted malformed project files are ignored; trusted malformed, oversized, partial-pair, unknown-field, symlink, and nonregular files fail before any model call. Preserved behavior. Configured physical builders, Jev and Clef still use the same model boundary, usage accounting, and error handling as AC-5 and AC-6. **REQ-8** The extension MUST register a user-issued `/magic8ball` command with interactive configuration, display, and explicit builder/classifier subcommands. Saves MUST default to user scope; `--project` MUST require current project trust. Conflicting or repeated scope flags and malformed command arguments MUST fail without writes. The picker MUST expose physical available chat models for the builder and available classifier entries for the classifier. Explicit identifiers MUST resolve to the same model types before persistence. Cancelling either picker MUST leave both files unchanged. Noninteractive invocation without identifiers MUST NOT prompt or write. Successful saves MUST preserve the other role in the selected layer and use a same-directory temporary file followed by atomic replacement. Writes MUST serialize through Pi's file-mutation queue and an exclusive lock file. Contention with another process, corrupt existing settings, or detected external modification MUST fail rather than overwrite data. A command MUST report its saved path and effective selections, including project overrides of user settings. The next tool invocation MUST use saved settings without `/reload`. The command MUST NOT change Pi's active conversation model, store credentials, or issue generation or classification requests. Catalog availability uses Pi's normal authentication boundary. Filesystem races with nonparticipating writers and power-loss durability are outside the guarantee. A save error with uncertain outcome MUST direct the user to inspect settings before retrying. **AC-8** Changed behavior. Invoke the registered command against temporary directories and mock model catalogs/UI. Verify interactive configuration, user and trusted-project role updates, picker cancellation at each stage, display, invalid syntax/models, noninteractive behavior, denied project writes, and immediate tool use. Concurrent in-process role updates preserve both roles. Existing lock files and corrupt settings prevent replacement. Successful writes leave no owned temporary or lock files. Assert configuration performs no model request, does not change the active model, and does not alter unrelated Pi settings files. **REQ-9** In TUI mode, each model picker MUST use a viewport bounded by the current terminal height. The selected model MUST remain visible during keyboard navigation and terminal resize. Each model MUST occupy one row. Display truncation MUST NOT change the selected provider or model identifier. The picker MUST retain selection across resize. A terminal too short for all controls MUST prioritize the selected row. RPC mode MUST continue to use the supported select dialog. Cancellation and settings persistence MUST retain REQ-8 behavior. **AC-9** Changed behavior. Render each TUI picker with 100 models at 30 and 80 columns and 6, 10, 24, and 40 rows. Navigate to the last model and resize the terminal. PASS if the selected row remains visible and the picker fits. At one and three terminal rows, PASS if the selected row remains visible within the terminal height. Render long Unicode identifiers. PASS if every line fits the width and selection saves the complete identifiers. Preserved behavior. RPC selection, virtual-builder exclusion, cancellation at either stage, and noninteractive behavior retain AC-8 results. Cancel each TUI picker after navigation. PASS if both settings files remain unchanged and no model request occurs. **REQ-10** Each TUI picker MUST use an inline layout with at most 10 model rows, like Pi's `/model` selector. REQ-9 terminal-height bounds MUST take precedence when fewer rows fit. The picker MUST support editable fuzzy search across the provider, model identifier, and display name. Search MUST use Pi's fuzzyFilter token matching and ranking. A nonempty query edit MUST select the best match. Arrow keys MUST wrap across filtered results. Other text keys, including `j` and `k`, MUST edit the search input. The configured model for the current role MUST appear first before search and carry a current-selection marker. When space permits, the picker MUST show the selected model's display name and navigation hints. An empty result set MUST show `No matching models`. Enter with no match MUST NOT finish the picker. Escape and Ctrl+C MUST cancel without writes. RPC MUST retain REQ-9 behavior. The picker MUST preserve complete model identifiers and MUST NOT change Pi's conversation model or refresh remote catalogs. **AC-10** Changed behavior. Open both 100-model catalogs in an 80-column, 60-row terminal. PASS if the picker is inline and displays at most 10 model rows. Type mixed-case fuzzy queries using display names and provider/model tokens. Edit and clear the query. PASS if filtering matches Pi's ranking, selects the best match, and saves the complete selected identifiers. Type `j` and `k`. PASS if these keys edit the query instead of navigating. Start with configured role models. PASS if each appears first, is marked current, and shows its display name. With no matches, press Enter then clear the query. PASS if Enter does not finish and editing restores results. Verify Up and Down wrap across results. Cancel with Escape and Ctrl+C at either stage. PASS if cancellation leaves both settings files unchanged and performs no generation or classification. Preserved behavior. AC-9 height, resize, Unicode width, identifier preservation, RPC, and cancellation checks MUST continue to pass. - #REQ-10 verified-by: [#AC-10](#AC-10) - #REQ-9 verified-by: [#AC-9](#AC-9) - #REQ-7 verified-by: [#AC-7](#AC-7) - #REQ-8 verified-by: [#AC-8](#AC-8) - #REQ-1 verified-by: [#AC-1](#AC-1) - #REQ-2 verified-by: [#AC-2](#AC-2) - #REQ-3 verified-by: [#AC-3](#AC-3) - #REQ-4 verified-by: [#AC-4](#AC-4) - #REQ-5 verified-by: [#AC-5](#AC-5) - #REQ-6 verified-by: [#AC-6](#AC-6) **REQ-20** The user command MUST provide argument autocomplete through Pi's registered command API. It MUST suggest `show`, `builder`, `classifier`, `reasoning`, `transcripts`, `--global`, and `--project` at the command root. Static choices MUST match the current token by case-sensitive prefix. Completed arguments and separators MUST remain in each replacement value. Each replacement MUST end with a space so the user can complete the next argument without typing a separator. Autocomplete MUST accept one scope flag before, between, or after settings arguments. It MUST NOT suggest repeated or conflicting flags. Scoped commands MUST NOT suggest `transcripts`. Transcript arguments MUST suggest only `on`, `off`, and `show`, without scope flags. Builder provider/model suggestions MUST use available physical chat entries. Classifier suggestions MUST use available classifier entries. Providers MUST be distinct and sorted by identifier. Provider matching MUST use a case-insensitive prefix. Models MUST belong to the completed provider. Model matching MUST use Pi's fuzzyFilter across identifiers and display names. Reasoning suggestions MUST use the supported levels of the builder targeted by the current scope. Omitted scope and `--global` MUST use the global builder. Trusted `--project` MUST use the effective builder, including a global builder when no project builder exists. Autocomplete MUST read a fresh settings snapshot for reasoning. It MUST NOT open untrusted project settings. Unknown syntax, excess arguments, missing context, missing builders, and failed dynamic lookups MUST return no dynamic suggestions. Static suggestions MUST remain available without settings or catalogs. Lookup failures MUST NOT display raw errors or trigger a fallback, retry, or notification. The current session context MUST supply dynamic lookups. Session replacement and shutdown MUST discard it and reject pending results from the prior context. Autocomplete MUST NOT save settings, change transcript capture or the active model, refresh remote catalogs, execute tools, or make generation/classification requests. Existing command validation, trust checks, picker behavior, and transcript clearing MUST remain unchanged. **AC-20** Changed behavior. Invoke the registered autocomplete callback for empty and partial arguments, both scope flags, interleaved flags, and transcript actions. PASS if choices follow REQ-20, replacement values retain prior arguments and separators, and completed or invalid commands suggest no extra positional arguments. Use mock chat/classifier catalogs with duplicate providers, virtual chat entries, display names, and slash-containing model identifiers. PASS if provider suggestions are distinct, role-specific, and prefix-filtered. PASS if fuzzy model suggestions preserve complete identifiers within the selected provider. Use private global/project settings with different builder capabilities. Change files between completions and change project trust. PASS if reasoning choices follow the write target, manual edits apply on the next completion, and unsupported levels do not appear. Apply completions with Pi's actual autocomplete provider, including a cursor before existing text. PASS if the command and untouched suffix remain intact. Inject unavailable settings/catalogs and pending classifier lookups during session replacement and shutdown. PASS if failures return no dynamic suggestions or raw errors, stale results do not return, and static completion still works. PASS if completion leaves settings bytes, active model, transcript capture, notifications, and model/tool request counts unchanged. Preserved behavior. All 53 prior offline checks and strict typechecking MUST pass before and after implementation. Tests MUST use private settings and mock catalogs. No paid request or live settings change is required. Known Pi 1.0.4 host limitation: after Tab completes `/magic8ball `, the next Tab can fail to open argument completion. The host editor routes that key to file completion. Its guard removes the trailing space and rejects the request before the argument callback runs. Forced file completion also skips command arguments. Changing that guard alone would not fix the routing. After name completion, type the first letter of a subcommand, wait for the menu, then press Tab. Use `r`, `b`, `c`, or `t` for reasoning, builder, classifier, or transcripts. For the full argument menu, press Backspace to remove the trailing space, wait for the command menu, then type Space. A separate isolated check verified these sequences with Pi's actual Editor and mock terminal dimensions. Provider-only AC-20 tests do not exercise the editor's keyboard routing. No installed Pi file changed, and the extension does not replace the editor. The user accepted the manual workaround. - #REQ-20 verified-by: [#AC-20](#AC-20) ## Evidence and usability improvements {#improvements} The user requested all six improvements from the code review. This change preserves settings, picker behavior, and advisory-only results. File hints guide collection. They do not authorize reads or replace permission checks. Byte ranges use the existing per-call byte ceiling. Eight calls bound total inspected file bytes to 128000 per invocation. The host packaging contract requires wildcard host peers. Exact development versions and CI define the tested compatibility set instead. **REQ-11** Historical contract, superseded by REQ-29. The previous builder reserved request four for tool-free JSON finalization and stopped collection after eight workspace calls. The current builder has no count-based finalization. REQ-27 and REQ-28 define tool-built state and normal-stop completion. **AC-11** Historical criterion for superseded REQ-11. AC-31 through AC-33 replace its count-based finalization checks. Cancellation, usage aggregation, and no automatic fallback remain required. **REQ-12** The input MAY contain `context.files`, an array of at most eight distinct permitted workspace-relative file paths. Omission MUST mean no hints. Empty arrays are permitted. Null, non-arrays, denied paths, duplicates, and workspace-disabled hints MUST fail before models. The builder MUST receive hints as data and MUST NOT read a hint automatically. Read and search MAY accept `byteOffset` and `byteLength`. Offset MUST default to zero and be a nonnegative safe integer. Length MUST default to 16000 and be an integer from 1 through 16000. Null and invalid ranges MUST fail. The helper MUST inspect only that byte window and report its half-open range and observed file size. Line numbers and read line offsets MUST refer to the byte window, not the whole file. A window that starts inside a UTF-8 sequence MUST fail. An incomplete terminal sequence MUST be omitted when more file bytes remain. The returned range.end MUST exclude the omitted bytes so a later call can resume at a character boundary. The helper MUST NOT read beyond byteLength to complete a character. If no complete character fits, range.end MUST equal range.start, text MUST be empty, and truncated MUST be true. Builder instructions MUST say to resume at range.end and enlarge an empty window to at least four bytes before continuing. Malformed UTF-8 within a window and incomplete sequences at EOF MUST fail as invalid-text. Existing path, type, byte, line, and permission restrictions MUST remain active. **AC-12** Changed behavior. Supply valid and invalid file hints and inspect the first builder message. PASS if valid hints survive unchanged, invalid hints fail before models, and no hint triggers an automatic read. Read and search a fixture beyond byte 16000. PASS if the requested text and exact inspected range return within the byte ceiling. Test defaults, EOF, invalid/null ranges, split UTF-8 boundaries, and denied paths with a range. Preserved behavior. Existing prefix reads, list limits, secret-path rejection, and cancellation checks remain passing. **REQ-13** The collector MUST assign `conversation` to nonempty permitted conversation and sequential `e1`, `e2`, and later IDs to successful workspace calls. REQ-25 adds `supplied1` through `supplied8` for caller-supplied evidence, independently of both scope flags. Collection metadata MUST contain a ledger of IDs, source names, scope, truncation, and inspected byte ranges when available. The builder MUST receive ledger IDs with each source. State evidence sources MUST equal a ledger ID from this invocation. Unknown IDs and citations to disabled or empty conversation MUST produce `invalid-state` before classification. The ledger MUST NOT contain raw file contents. Per-source metadata MUST survive successful results. This check proves source identity, not claim accuracy or neutrality. **AC-13** Changed behavior. Collect truncated workspace and conversation evidence, then inspect state and collection metadata. PASS if citations use collector IDs and ranges and truncation survive the result. Invented IDs, arbitrary paths, and disabled-conversation citations MUST fail before classification. Preserved behavior. State shape, byte ceiling, and separate builder/classifier stages remain enforced. **REQ-14** The package MUST provide an opt-in quality benchmark with synthetic missing, conflicting, injected, and decisive evidence cases. The benchmark MUST compare named builder configurations against a direct-classification baseline using both response orders. All configurations MUST use the same explicit classifier selection. It MUST report decision accuracy, abstention, order sensitivity, builder fact coverage, uncertainty coverage, invalid outputs, and usage. A dry run MUST list the exact finite comparison plan without loading providers or credentials. It MUST report builder-request and total-model-call maxima as null because REQ-29 removes count ceilings. Usage metering MUST remain informational, not an execution limit. A live run MUST require an explicit spending flag and a caller-supplied adapter with explicit model selections. The benchmark MUST use synthetic fixture context only and MUST NOT gather session or workspace data. Reports MUST omit raw provider errors, file content, and transcripts. Benchmark results MUST NOT certify neutrality or calibration. Ordinary tests and CI MUST NOT make model requests. **AC-14** Changed behavior. Inspect the benchmark plan and run it with mock builder and classifier adapters. PASS if all four evidence cases, both response orders, each builder configuration, and direct baseline appear in the report. PASS if known mock answers produce the expected coverage, abstention, accuracy, order-sensitivity, and usage counts. Dry run and a live run without authorization MUST make no adapter calls. Injected adapter errors MUST produce safe error counts without raw messages or automatic retries. **REQ-15** The tool MUST emit safe preparation, collection, validation, and classification progress through Pi's update callback when supplied. Progress MUST contain no raw context or provider errors. No callback MUST be required for completion. Errors MUST expose a finite stage (`preparation`, `collection`, `validation`, or `classification`) and the existing error-kind diagnostic code. The renderer MUST show advisory status, winning choice or abstention, distribution, backend confidence, and reported cost in its compact view. The expanded view MUST also show state evidence, source IDs, truncation, and uncertainties. Partial results and host argument errors MUST render without a crash. Lines MUST fit the supplied terminal width. The model-facing structured result, confidence meaning, and noninteractive behavior MUST remain available. **AC-15** Changed behavior. Invoke the registered tool with and without an update callback using mocked models. PASS if progress stages occur in order and contain no context. Inject model and evidence errors and inspect safe stage/code diagnostics. Render success, abstention, errors, missing details, and partial updates at narrow and wide widths with Unicode data. PASS if compact output contains the distribution and cost, expanded output adds evidence, and every line fits. Preserved behavior. Structured success/error schemas and aggregate usage checks remain passing. **REQ-16** The repository MUST declare exact development dependencies and commit a lockfile. CI MUST run `npm ci --ignore-scripts` and `npm run verify` on a clean checkout with Node 24. Runtime host packages MUST remain optional wildcard peers and MUST NOT become bundled runtime dependencies. Documentation MUST identify Pi 1.0.4 as the tested host version and MUST NOT claim untested later-version compatibility. **AC-16** Changed behavior. Inspect the package, lockfile, and CI workflow. PASS if dependencies are pinned and CI uses the specified commands. Run the same commands in a private clean source copy without worktree symlinks. PASS if strict typechecking and all offline tests pass. No live settings or model requests are permitted. - #REQ-11 verified-by: [#AC-11](#AC-11) - #REQ-12 verified-by: [#AC-12](#AC-12) - #REQ-13 verified-by: [#AC-13](#AC-13) - #REQ-14 verified-by: [#AC-14](#AC-14) - #REQ-15 verified-by: [#AC-15](#AC-15) - #REQ-16 verified-by: [#AC-16](#AC-16) ## Optional transcripts {#transcripts} The user requested optional visible transcripts instead of metadata-only debugging. Capture stays local to the extension runtime. The extension creates no transcript files and adds no transcript to model context. Ordinary conversation, file content, and model text can contain sensitive data. The capture filter is not a secret detector. **REQ-17** The user-only `/magic8ball transcripts` command MUST toggle capture. `on` and `off` MUST set capture explicitly. `/magic8ball transcripts show` MUST display the last captured invocation. Unknown arguments and scope flags MUST fail without settings writes or model requests. Capture MUST default to off. Session start and shutdown MUST turn it off and clear retained text. Off MUST clear retained text and revoke in-flight recorders. Repeated on MUST preserve the existing captured invocation. A new captured invocation MUST replace the prior invocation when it starts. Older concurrent or late completions MUST NOT overwrite it. Capture MUST include logical Pi builder requests (system prompt, messages, tool declarations), visible builder responses, model-facing evidence results, and classifier requests/responses. The filter MUST omit thinking/reasoning blocks, signatures, images, authentication fields, request headers, and raw provider error messages. Failed or aborted builder response text MUST be omitted, because adapters can place raw errors in text blocks. Only allowlisted evidence arguments, state arguments under REQ-30, and classifier decision fields MAY enter structured capture. Arbitrary provider-result fields MUST NOT enter it. The recorder MUST keep at most 128000 UTF-8 bytes of transcript body per invocation. Display headers and the bounded failed-call diagnostic footer in REQ-19 are outside the body ceiling. Overflow MUST clip at a character boundary and report transcript truncation. Capture failures MUST NOT change model work or decision results. Completion MUST close the recorder. No provider result that arrives after completion MAY append text. The extension MUST NOT write transcript files, add transcript fields to tool results, append session entries, or send captured text to either model. The TUI viewer MUST be read-only, scrollable, width/height bounded, and close with Escape or Ctrl+C. RPC show MUST use a user-directed notification. A disabled or empty capture MUST show fixed status text, not old transcript content. Hosts and clients may retain displayed data outside the extension. The registered tool schema MUST NOT allow models to enable capture. Settings and picker behavior MUST remain unchanged. **AC-17** Changed behavior. Invoke the user command without valid settings and inspect default, bare toggle, explicit on/off, show, and invalid syntax. PASS if no command writes settings, performs a model request, or changes Pi's active model. Run mocked builder/evidence/classifier work with capture on. PASS if show includes actual logical inputs, visible outputs, and source IDs in order. Inject hidden blocks, signatures, authentication fields, and raw provider errors. PASS if these fields are absent while visible model text remains. PASS if tool content/details/structuredContent contain no captured transcript or capture-only markers, and disabled capture stores no transcript. Exercise off during a call, older concurrent completion, late completion after timeout, session reset, and oversized multibyte entries. PASS if cleared or superseded text never returns, total retained text stays within 128000 bytes, and truncation is explicit without broken UTF-8. Render and scroll the actual viewer at widths 1, 30, and 80 and heights 1, 6, and 24. PASS if every line fits, selected pages remain bounded, and both cancellation keys close it. Preserved behavior. All 37 existing offline checks and strict typechecking MUST remain passing. No paid model call is required. - #REQ-17 verified-by: [#AC-17](#AC-17) ## Builder reasoning and evidence discovery {#discovery} The user authorized fixes after a captured builder response guessed `test` instead of the existing `tests` directory. The two list calls were planned in one response, before the builder saw the parent listing. Reasoning controls, discovery instructions, and distinguishable evidence outcomes address different parts of that failure. Higher reasoning is not a guarantee of correct paths or neutral extraction. Existing configurations keep their prior reasoning behavior. **REQ-18** The builder settings role MAY contain `reasoning`: `default`, `off`, `minimal`, `low`, `medium`, `high`, `xhigh`, or `max`. Omission and `default` MUST preserve the prior request behavior without inheriting the parent thinking level. `off` MUST omit the simple-stream reasoning option, and MUST be accepted only when the selected model advertises off support. Other levels MUST be forwarded unchanged on every builder request. Unsupported explicit levels MUST fail before model requests; no silent clamping or model fallback is permitted. The classifier MUST NOT receive this option. Existing output-token hints, request/tool limits, timeout and cancellation MUST remain in effect. Provider adapters can apply model-specific token/budget mappings. This is not a fixed monetary budget; choosing more reasoning can increase cost and latency. The user-only `/magic8ball [--global|--project] reasoning` command MUST show the effective builder level and supported choices without model requests or writes. `reasoning ` MUST atomically save a complete builder role, preserving the classifier and obeying existing scope/trust rules. Global writes MUST use the global builder, not copy a project override. A project write MAY copy the effective builder into a complete project override. Existing model-selection commands/pickers continue to replace complete roles. Selecting a model without a level resets that role to its legacy default behavior; users can then set reasoning explicitly. The tool schema MUST NOT give models control of reasoning. Explicit configured reasoning MAY appear in the builder model metadata. The expanded renderer MUST show the configured level or legacy default. Captured builder requests MUST identify requested reasoning and safe token/retry options. Responses MAY show the SDK's provider-native effort label, but MUST NOT show hidden reasoning content or authentication options. No live settings changes or paid evaluation are authorized merely by implementing this feature. **REQ-19** Builder instructions MUST say to list an unknown parent, wait for the returned names, and use those names for dependent child calls. Independent calls MAY still be grouped. Instructions are not proof that a model complies. A read/list/search of a permitted but absent path MUST return a non-error, collector-citable observation with `code: path-not-found`, fixed safe text, and explicit truncation false. This outcome MAY be produced only when ENOENT is observed while walking a syntactically permitted path under a resolved workspace root, after checking all existing ancestors for symlinks and directory type. Invalid arguments MUST fail before a missing-path observation can be returned. Root failures, symlinks, restricted names, traversal, wrong file types, invalid text, access denials and other I/O failures MUST remain hard failures. Missing paths MUST consume evidence-call budget and receive collector IDs. State extraction MUST preserve the absence/uncertainty without treating it as file content. The builder MUST NOT recover from `ctx.executeTool` throws or `isError`, even if an error payload claims `path-not-found`. Hard failures MUST use only fixed diagnostic codes: `invalid-arguments`, `path-denied`, `wrong-type`, `invalid-text`, `permission-denied`, `io-failed`, or `tool-denied-or-failed`. The final error MAY add `evidenceCode` while retaining `kind/code: evidence-failed`. Unknown hook/runtime failures MUST use `tool-denied-or-failed`, not an invented permission diagnosis. Transcripts MUST show a safe failed-call marker linked to the tool call ID, without copying raw hook/provider errors. Negative source status MUST appear in collector metadata and the expanded view. If body clipping omits that marker, a bounded diagnostic footer MUST preserve the fixed failure code and identifiers. Each footer identifier MUST omit control characters, fit within 512 UTF-8 bytes, and report clipping. Off/reset and superseding capture MUST erase this footer along with the body; late or closed recorders MUST NOT restore it. Permission hooks, containment checks, cancellation, and provenance validation MUST remain active. REQ-27 through REQ-29 replace tool-free finalization and request/tool count ceilings. No automatic retry is permitted. **AC-18** Changed behavior. Check all declared reasoning values, invalid/null/unknown values and classifier-role rejection. PASS if legacy configuration is unchanged and project roles still replace whole builder roles. Run user reasoning commands against private settings with mock catalogs. PASS if query makes no model request or write, saves preserve the classifier, global scope does not copy project model selection, project trust is enforced, and unsupported levels do not save. Invoke the real extension boundary with mocked providers. PASS if each builder call receives the selected supported level, omitted/off/default retain the legacy option shape, and the classifier receives no reasoning option. PASS if unsupported explicit levels fail before model work, output schemas accept the added builder metadata, and transcripts show requested reasoning/options without auth fields or hidden thinking. PASS if a clipped transcript still shows the safe failed-call diagnostic and off clears its retained footer. All 44 prior checks and strict typechecking MUST remain passing. A paid evaluation is not required and no quality improvement is inferred from these mocks. **AC-19** Changed behavior. Reproduce the captured parent-list plus guessed missing `test` call against a private workspace that contains `tests`. PASS if the missing call becomes a coded observation, the next mocked builder request sees both results, and a later call can inspect `tests` before classification. PASS if read/list/search missing paths get IDs/status and remain subject to the same per-read bounds and invocation deadline. AC-33 replaces the previous eight-call finalization check. Check missing paths combined with invalid arguments, denied lexical names, symlink ancestors and wrong-type ancestors. PASS if no unsafe/invalid call becomes recoverable absence. Inject a permission-hook throw and an `isError` payload that claims `path-not-found`. PASS if both stop collection, no classifier runs, and safe failure markers/code are shown without raw error text. Check fixed error codes and actual renderer widths. PASS if diagnostics are distinguishable and safe, and existing containment/cancellation/provenance/budget assertions remain unchanged. - #REQ-18 verified-by: [#AC-18](#AC-18) - #REQ-19 verified-by: [#AC-19](#AC-19) ## Verification and remaining limits {#verification} AC-7 and AC-8 change configuration. The existing 14 tests passed before this change and cover preserved AC-1–AC-6 behavior. All nine new settings/command tests failed before implementation against the existing environment-based extension and callable settings scaffolds. After implementation, `npm run verify` passed strict TypeScript checking and all 23 Node tests against Pi 1.0.4. The installed Pi extension loader registered four tools and the `magic8ball` slash command without errors. Settings tests use private temporary directories, scoped environment cleanup, and mock UI/model catalogs. They do not change live user or project settings. | Criterion | Check names in tests | Before settings implementation | After settings implementation | | --- | --- | --- | --- | | AC-1–AC-6 | original validation, neutral-state, evidence, distribution, fault, usage, and registration checks | Passed | Passed | | AC-7 | role precedence, trust and fresh reads; ignored old environment; malformed settings; unsafe files | Failed | Passed | | AC-8 | scoped saves and immediate use; picker; cancellation; invalid/noninteractive commands; concurrent saves and lock contention | Failed | Passed | Historical V1 verification: all original 14 tests failed against callable no-op scaffolds before the first implementation. A separate AC-3 assertion reproduced explicit null values bypassing evidence defaults; the helper now rejects those values. The settings change updates configuration fixtures without weakening the original assertions about model behavior. `docs_validate` reported no diagnostics and publication completed for the project output root when the new requirements were imported. These statements report observed local checks, not authenticated verification records or policy approval. Model responses and the nested-tool boundary are mocked. The tests use Pi's installed session projection and TypeBox schemas. No live calibration check, live terminal check, or context-builder quality evaluation has run. In-process write serialization and pre-existing lock contention are tested. External-editor races and interrupted/power-loss saves are not fault-injected. The settings comparison detects changed bytes before replacement; nonparticipating writers can still race the final validation and filesystem operation. Historical V1 implementation required no package installation, profile activation, live settings edit, or deployment. Development began in an empty bare repository and linked orphan worktree. Profile-wide extension checks do not apply to this independent project. This package uses its own `npm run verify` command. The workspace path filter is not an OS sandbox. Concurrent filesystem mutation can race path validation. The helper rejects known credential paths, but ordinary files and conversation can contain secrets. Direct prior magic8ball results are omitted. Quotations and compaction summaries can still contain prior decisions or probabilities. Configured providers determine external retention. The parent session stores returned state and results through normal Pi persistence. - #REQ-5 references: [#REQ-7](#REQ-7) Picker layout verification (AC-9): the original selector rendered 110 lines for 100 models in a 40-row terminal. Long Unicode labels rendered 610 lines in a 10-row terminal. Both changed-behavior tests failed before the picker change. The TUI cancellation check and all 23 existing checks passed before the change. After the change, `npm run verify` passed strict TypeScript checking and all 26 tests against Pi 1.0.4. The checks render the actual picker component with mock terminal dimensions and real SelectList behavior. They cover both catalogs, keyboard navigation, terminal resize, Unicode width, complete saved identifiers, and cancellation at either stage. RPC keeps the supported select dialog. The first AC-9 implementation used a bounded overlay and removed controls when only the selected row fit. These checks do not establish live terminal appearance or mouse behavior. No provider request or live settings change occurred. Compact search verification (AC-10): the user requested `/model` behavior instead of the screen-sized overlay. The installed Pi 1.0.4 selector uses an inline layout, 10 model rows, Input, and fuzzyFilter. Its constructor requires a chat-session ModelRuntime, which ExtensionCommandContext does not expose. Classifier catalogs require a separate model type. The magic8ball picker uses the host's Input and fuzzyFilter primitives plus SelectList, without the host selector's remote refresh. The four new AC-10 checks failed before implementation. Existing AC-9 and configuration checks passed before implementation. The old picker displayed 53 model rows in a 60-row terminal and had no search or configured-role marker. After implementation, `npm run verify` passed strict TypeScript checking and all 30 tests. AC-10 checks cover inline placement, the 10-row cap, fuzzy names and provider/model queries, text editing, and configured-role selection. They also cover no-match Enter, query clearing, wrapped navigation, and Ctrl+C cancellation at either stage. AC-9 height, resize, Unicode, complete identifiers, Escape cancellation, and RPC checks remain passing. The overlay-only assertion in the old UI fixture was replaced with AC-10's required inline-placement assertion. These checks use mock terminal dimensions. They do not verify live terminal appearance or remote provider behavior. After configuration, one user-requested live smoke call completed both the context builder and classifier stages. The call used conversation context with workspace access disabled. It returned a valid advisory distribution without abstention. This confirms that the loaded extension and configured providers completed one end-to-end call. It does not establish calibration, context-builder quality, workspace evidence collection, or live terminal appearance. Before initial GitHub publication, `npm run verify` again passed strict TypeScript checking and all 30 tests. A scan of the 16 intended project files found no common credential patterns. This scan is not a secret-detection guarantee. Evidence and usability verification (AC-11 through AC-16): the original 30 checks and strict typechecking passed before implementation. All seven new checks in `tests/improvements.test.mjs` failed before implementation for the expected missing behavior. The benchmark core used a callable no-op scaffold for that initial run. No check failed from a missing import or syntax error. | Criterion | Check name in tests/improvements.test.mjs | Before | After | | --- | --- | --- | --- | | AC-11 | reserves final request and finalizes after the evidence ceiling | Failed | Passed | | AC-12 | file hints remain data and invalid hints fail before models | Failed | Passed | | AC-12 | range reads and searches reach beyond the prefix and retain restrictions | Failed | Passed | | AC-13 | collector IDs and source metadata reject invented or disabled citations | Failed | Passed | | AC-14 | benchmark compares configurations and orders without implicit spend | Failed | Passed | | AC-15 | progress, stage diagnostics and compact/expanded rendering are safe | Failed | Passed | | AC-16 | development dependencies and CI support clean-checkout verification | Failed | Passed | Existing state fixtures now cite collector IDs and provide matching ledgers. Their distribution, usage, and stage-order assertions remain unchanged. The request-budget fixture expects three evidence calls instead of four. The fourth request must finalize and still contributes its usage. A benchmark mock initially matched the common question wording instead of its case ID. Its case selector was corrected without changing expected metrics. The final `npm run verify` passed strict typechecking and all 37 checks in the worktree. The same checks passed in a private clean source copy after `npm ci --ignore-scripts`, with no worktree symlinks. The dependency install reported zero known vulnerabilities. That audit is not a security guarantee. The benchmark dry run returned 24 comparisons and an 88-model-call ceiling for two builder configurations. CLI checks proved that dry runs and unauthorized live invocations do not import an adapter. An authorized mock adapter error produced no raw provider message in CLI output. Renderer checks used Unicode data and widths 1, 8, 30, and 80. They did not inspect a live terminal. The benchmark reports keyword coverage for successful builder results. Invalid results have null coverage and separate error counts. Empty keyword sets score 1. Coverage averages exclude null values and remain heuristic, not proof that facts are correct. The benchmark direct baseline uses fixed synthetic state. Custom adapters remain trusted executable code outside a sandbox guarantee. No paid model evaluation, live provider call, hosted CI run, profile change, or deployment occurred during this change. The hosted workflow remains unexecuted. Local clean-copy checks exercised its install and verification commands. Optional transcript verification: before implementation, five AC-17 checks failed against the callable scaffold and existing commands. A separate injected raw-provider-text check failed before the failed/aborted-response filter was added. The final local `npm run verify` passed strict typechecking and all 44 tests, including all 37 prior checks. The tests exercise command isolation from settings/model work, logical request/response capture, hidden-field filtering, clear-on-off and session reset, older and late completion rejection, timeout handling, bounded multibyte text, and the actual read-only viewport with mock terminal dimensions. Only the existing test API mocks gained no-op lifecycle registration; prior assertions were not weakened. No paid model call, live transcript evaluation, or physical-terminal appearance check was run for this transcript change. The transcript feature requires reload or restart; capture then starts off and applies only to subsequent calls. Reasoning/discovery verification: all 44 existing checks passed before this change. All eight initial AC-18/19 checks failed before implementation. A ninth check validates safe option/effort capture and reproduced a lost failure marker after transcript body clipping before the bounded footer fix. The final `npm run verify` passed strict typechecking and all 53 tests. Previous assertions about permission hooks, symlinks, containment, cancellation, source IDs and ceilings were preserved. The new private-workspace replay matches the observed parent-list plus guessed `test` call and verifies a later inspected `tests` call before classification. Missing observations consume the eight-call budget and trigger tool-free finalization; a ninth attempted call still fails without classification. Mocked catalogs verify unsupported levels (including off where unavailable), unchanged legacy options, complete scoped roles, and absence of classifier reasoning changes. These checks use mock model responses and do not establish better reasoning quality or factual neutrality. No paid rerun, live settings write, or real-terminal regression test occurred for this change. The hosted CI workflow remains unexecuted. Profile-only check scripts are not present in this workspace; project validation uses its defined `npm run verify` command. Autocomplete verification (AC-20): all 53 prior checks and strict typechecking passed before this change. All six new checks in `tests/autocomplete.test.mjs` failed before implementation because argument completions were absent. The final `npm run verify` passed strict typechecking and all 59 checks. Existing assertions remained unchanged. The checks cover command grammar, distinct role-specific providers, fuzzy identifiers and names, and supported reasoning for the selected write scope. Private settings fixtures verify fresh manual edits, project trust, lookup failures, unchanged settings bytes, and unchanged transcript capture. Lifecycle checks reject pending catalog results after session replacement or shutdown. Pi's actual autocomplete provider applies replacements with complete argument prefixes, separators, cursor positions, and untouched suffixes. Two initial fuzzy queries matched more fixture models than intended. The fixture now uses unique queries without changing the expected identifiers. A separate excess-token assertion reproduced scope suggestions after an invalid extra positional token. The prefix filter now rejects that case. `git diff --check` passed. No paid call, live settings change, remote catalog refresh, or physical-terminal autocomplete check ran. This change requires `/reload` or restart. It does not change command execution or model-facing tool arguments. Request-only source guidance verification (AC-2 and AC-13): a live synthetic call reproduced a citation to `conversation` with conversation sharing disabled. The collector had no sources. The state validator correctly rejected the citation. The builder prompt now gives the current source-ID list on every request, including an empty list. Request-only facts belong in `current_state` or `constraints`, not collector-cited evidence. An empty list requires `evidence: []`. No request source ID was added. State and classifier validators remain unchanged. Two new guidance checks failed before this change. The preserved invented-citation check passed before and after. After the change, strict TypeScript checking, all 76 tests, and `git diff --check` passed. One isolated live extension/runtime replay with `openai-codex/gpt-6-luna`, high reasoning, and `openrouter/cloudflare/clef-flash` returned valid empty evidence. The classifier chose the expected `longer` response and reported complete known usage. A separate workspace replay passed before the prompt change. The earlier live `invalid-answer` was not localized. These calls do not establish calibration, guaranteed prompt compliance, or physical-terminal behavior. User model settings did not change. A final post-change workspace replay reproduced `invalid-answer` with valid source `e1`. Clef returned probabilities `0.9816`, `0.0063`, `0.0081`, and `0.0039`, which sum to `0.9999`. Their sum differs from one by `0.0001`, above REQ-4's `0.000001` tolerance. The selected choice was maximal and backend confidence was `0.9755`. The validator correctly rejected this result under the current contract. The original rejected payload was not retained. Accepting four-decimal distribution rounding requires a contract decision. No classifier validation change was made in this task. Four-decimal rounding verification (AC-24): the user approved a bounded rounding allowance without renormalization or other validation changes. REQ-4 now permits half a four-decimal unit per probability when every value is on that grid. The implementation compares integer-unit drift. Finer-precision values retain the original sum tolerance. Four changed-behavior checks failed before implementation. Three preserved-behavior checks passed before and after. The checks cover the captured Clef reply, both drift signs, 2/3/4/26 choices, rounded ties, and all 10001 grid values. They also cover adjacent non-grid values, original full-precision tolerance, unchanged values and usage, and invalid keys, ranges, confidence, and winners. After implementation, strict TypeScript checking and all 83 tests passed. `git diff --check` passed. One isolated live workspace replay with the unchanged user models selected `npm_verify`, cited `e1`, and reported complete known usage. That live distribution summed to approximately one. The captured `0.9999` case was verified through the registered tool with mocked providers. These checks do not establish calibration or physical-terminal behavior. Model settings, retries, and fallback behavior did not change. Reload or restart is required to use the changed runtime in an existing session. ## Advisor-derived reliability changes {#reliability} The user authorized reliability and inspection improvements based on advisor revision 38. Magic8ball remains a bounded classifier, not a replacement for open-ended consultation. The structured state and distribution contracts remain mandatory. **REQ-21** Settings MAY contain root timeoutMs, an integer from 1 through 2147483647 milliseconds inclusive. Omission MUST inherit trusted project, global, then the existing 120000 ms default independently of model roles. Invalid values MUST fail closed; untrusted project files MUST remain unread. Each invocation MUST measure one deadline from decide entry, including preparation, builder, evidence, validation, and classification. Preparation MUST initially use the default. After settings resolve, the deadline MUST be rearmed from original entry, not restarted. An elapsed budget MUST fail before further work. Late callbacks MUST NOT rearm timers, change returned usage, emit progress, or start a later stage. Every builder/classifier request MUST receive a positive integer remaining timeout and maxRetries zero. The local deadline MUST return even when a provider ignores abort. No automatic retry or fallback is permitted. The user-only timeout command MUST query effective milliseconds/source without writing or generating. timeout MUST save that scope's timeout; timeout default MUST remove only that scope's timeout. Queries and writes MUST work without models. Saves MUST preserve all other scope-owned fields from the existing locked checkpoint. Model/reasoning/picker saves MUST preserve timeout. Autocomplete MUST offer timeout and timeout default without dynamic reads. Tools MUST NOT accept caller timeouts. This replaces only the fixed-duration rule; all other ceilings remain unchanged. **AC-21** PASS when timeout settings accept both inclusive bounds, reject null/nonnumber/fractional/out-of-range values, inherit independently of model roles, and ignore untrusted projects. Through the registered command, query/save/reset scoped timeouts without model work and preserve unrelated checkpoint fields. Static timeout completion MUST work without a catalog. PASS when preparation consumes the original budget, elapsed preparation prevents generation, every builder/classifier request receives a positive integer remainder, cancellation and ignored-abort timeout return without later stages, and late callbacks cannot change results or progress. Use temporary settings and controlled offline delays only. Preserve all prior settings/trust/locking, resource-ceiling, permission, and transcript tests. **REQ-22** Every result MUST include usageComplete. It MUST be true only when no provider attempt began or every begun attempt returned a usage-bearing result before return. Unknown usage MUST remain incomplete for that invocation, even after a later successful call. Completed rejected replies MUST retain known usage. Late usage MUST NOT change results. Completeness is local reporting, not verified billing or remote cancellation. Input MAY contain diagnostics boolean. Omission/false MUST omit error.diagnostics; true MUST add fixed phase/category hints to failures only. Null/nonboolean values MUST fail validation. No provider input or persistent setting MUST contain this opt-in. Phases MUST be input-validation, preparation, builder, evidence, state-validation, classifier, or answer-validation. Existing error kind/code/stage MUST remain. Diagnostics MUST NOT expose provider text, rejected state, credentials, headers, or thinking. Thrown provider errors MAY use own data status/code descriptors: 401/403 authentication; 400/404/409/413/422 provider-rejection; 429 rate-limit; 408/500/502/503/504 or ECONNRESET/ECONNREFUSED/ETIMEDOUT/ENOTFOUND/EAI_AGAIN transport. Recognized status MUST take precedence. Other thrown provider errors MUST use unknown. Terminal error/aborted results MUST use provider-error/provider-aborted. Extension errors MUST use extension-error; unexpected nonprovider failures MUST use local-error. Accessors and descriptor failures MUST NOT leak or execute getters. No free-form error parsing is permitted. **AC-22** PASS when actual registered tools report complete known usage on success, rejected state/distribution, and evidence failures, and incomplete usage after thrown provider errors, missing usage, timeout, or cancellation during a provider attempt. A pre-provider failure MUST report complete known zero usage. Late settlement MUST change neither results nor progress. PASS when opt-in diagnostics distinguish all seven phases, expose only fixed allowlisted categories, omit raw secret-bearing errors/state/thinking, and preserve default error fields. Test structured HTTP/code precedence, hostile accessors, terminal abort/error, malformed content, and strict input/output schemas. **REQ-23** The call renderer MUST show the full sanitized question and every response identifier/description, preserving line breaks and wrapping at terminal width. This display MUST NOT alter dispatched inputs, add messages, or generate. Successful collapsed and expanded views and known failures MUST show reported input/output/cache-read/cache-write tokens, cost, and usage completeness. Expanded success MUST retain validated facts, source IDs, ranges, truncation, and uncertainties. Partial/fallback/failure views MUST NOT reveal rejected state or private thinking. Diagnostic views MUST show opted-in fixed phase/category only. All lines MUST fit width and neutralize terminal controls with current themes. Tests MUST exercise the installed Responses adapter with synthetic authentication and mock fetch. Checks MUST cover integer timeouts, no retries, builder state tools, and normal stop without an explicit toolChoice override. Offline checks MUST NOT claim live provider interoperability, calibration, or UI appearance. **AC-23** PASS when multiline Unicode question/choices retain their full sanitized tails at widths 1, 8, 30, and 80, both views show known usage and incomplete labels, expanded views retain evidence/uncertainties, and rendering changes no result or provider call count. Controls, partial responses, and rejected/private data MUST remain safe. PASS when a registered builder request reaches the installed Responses adapter mock fetch with a positive integer timeout and maxRetries zero. Builder payloads MUST declare state tools and omit an explicit tool_choice override. A mock HTTP rejection MUST produce no raw payload exposure or classifier call. No real credentials or network requests are permitted. - #REQ-21 verified-by: [#AC-21](#AC-21) - #REQ-22 verified-by: [#AC-22](#AC-22) - #REQ-23 verified-by: [#AC-23](#AC-23) Caching/replay assessment: Pi's default is already short caching. This task does not change cache options, retain new opaque state, or run a paid quality benchmark. A measured benefit is required before adding replay complexity. The existing benchmark requires separate spending authorization. ### Offline verification {#reliability-verification} Baseline 504765b passed TypeScript and all 59 tests before implementation. The first 12 checks in tests/reliability.test.mjs failed before implementation for missing timeout settings, request timeouts, usage completeness, diagnostics, or full question/choice display. All 12 pass afterward. Two further checks cover late preparation and successful fourth-request finalization through the installed Responses adapter with mock SSE. The revised full suite passes TypeScript and all 73 tests. All earlier checks remain, with only the root/scoped autocomplete expectations extended for timeout. A private clean source copy passed npm ci --ignore-scripts --offline and npm run verify. Project git diff --check and npm pack --dry-run --ignore-scripts passed. The tests use temporary settings, synthetic keys, mock fetch/SSE, injected failures, bounded delays, and terminal dimensions. They verify integer remaining deadlines, independent configuration precedence, scoped reset and preservation, known versus incomplete usage, finite private hints, cancellation/late-settlement boundaries, call visibility, and real adapter request construction. The benchmark adapter now also forwards the remaining classifier deadline. These are offline observations, not proof of live provider support, calibrated confidence, billing completeness, remote cancellation, or physical-terminal appearance. No paid provider call, live settings change, commit, push, deployment, hosted CI run, Windows check, or macOS check occurred. Reload is required before exercising the changed runtime. ## Review regression contract {#review-regressions} The review identified a discarded decoder suffix and a discarded assistant message. These fixes preserve byte ceilings and the documented filesystem-race limitation. Review verification: baseline 8de37f3 passed strict TypeScript checking and all 87 tests before these changes. Seven new changed-behavior checks failed before implementation. Two new preservation checks passed before and after. All nine checks in tests/review-regressions.test.mjs now pass. The full suite passes strict TypeScript checking and all 96 tests. AC-26 covers each partial prefix of two-, three-, and four-byte characters for read and search, including the 16000-byte ceiling. It also covers tiny windows, continuation, BOM byte accounting, invalid text, and builder range handling. AC-27 covers mixed assistant prose at extraction and builder input, result exclusion, disabled conversation, and UTF-8 limits. One earlier test expected whole-message exclusion. Its revised assertion requires retained prose and preserves result, image, and argument exclusion. The byte allocator remains bounded by byteLength. No extra reads complete a character. The suite includes package contents, the unpacked Pi 1.0.4 loader, and the offline benchmark check. These are offline observations, not proof of provider quality, calibration, physical-terminal behavior, or filesystem confinement. No builder or classifier request, live settings change, commit, tag, push, or npm publication occurred during these fixes. **AC-26** Changed behavior. Test each partial prefix of two-, three-, and four-byte UTF-8 characters in read and search windows. PASS if range.end excludes the partial prefix and a call at that offset returns the complete character without invalid-text. Test byteLength values 1, 2, and 3, and splits at the 16000-byte ceiling. PASS if a window with no complete character returns empty text and an empty, truncated range. PASS if builder instructions describe continuation and enlargement, and the collector accepts adjusted and empty ranges. Preserved behavior. Complete ASCII and UTF-8 windows, BOM byte accounting, EOF ranges, and output ceilings MUST remain unchanged. Invalid starts, malformed prefixes and interiors, NUL bytes, and incomplete EOF sequences MUST still fail as invalid-text. All tests use private temporary files or mocked model responses. **AC-27** Changed behavior. Give conversation extraction an assistant message with ordinary text and a magic8ball tool call. PASS if the ordinary text survives extraction and reaches the first builder request with conversation sharing enabled. Preserved behavior. Tool-call arguments, thinking, images, system messages, and magic8ball or magic8ball_* results MUST remain absent. Disabled conversation sharing MUST still supply no history or conversation source ID. Existing UTF-8 conversation limits MUST remain active. Tests MUST NOT make provider calls. - #REQ-12 verified-by: [#AC-26](#AC-26) - #REQ-2 verified-by: [#AC-27](#AC-27) ## Publication preparation {#publication} The user requested a code review and publication preparation. This work does not authorize npm publication, commits, tags, or pushes. The user selected @neruok/pi-magic8ball, version 0.1.0, and the MIT license. The release candidate uses public scoped-package access. Registry ownership and publication credentials remain unverified. Release metadata does not grant publication authority. The existing Node 24 and tested Pi 1.0.4 boundaries remain unchanged. Pi loads the TypeScript entry through its extension loader. The package check uses local files and the pinned development host. It makes no model requests and changes no live settings. Publication review checks: the unchanged baseline passed strict typechecking and all 83 tests. The new metadata and focused-command checks failed before the manifest change. The baseline packed-loader and benchmark check passed. After that change, the three AC-25 checks and the full 86-test suite passed. Strict typechecking also passed. The unpacked package registered four tools and the command through the Pi 1.0.4 loader without adjacent node_modules. The packed benchmark dry run reported 24 comparisons and an 88-call ceiling. No compiled entry was needed. The npm audit reported zero known vulnerabilities, including development dependencies. This is not a security guarantee. The review found no confirmed runtime defect in the inspected paths. Runtime code and dependency versions did not change. Physical-terminal checks, paid provider checks, hosted CI, Windows, and macOS checks did not run. No npm publication, commit, tag, or push occurred. **REQ-24** Repository verification MUST pack with lifecycle scripts disabled and inspect the resulting local tarball. The check MUST extract the tarball into a private temporary directory without copying node_modules into the extracted package. Only package.json, README.md, docs/pi-magic8ball.md, magic8ball.ts, lib/*.ts, the two benchmark scripts, and an optional LICENSE file MAY ship. The manifest MUST contain the pi-package keyword, repository URL, homepage, and issue URL. Host peers MUST remain optional wildcard peers without bundled runtime copies. The check MUST load the packed manifest entry through the pinned Pi extension loader and confirm four tools and the magic8ball command register without loader errors. The check MUST confirm the packed benchmark dry run reports 24 comparisons and null builder-request and total-model-call maxima for two configurations, under REQ-29. The check MUST make no model request, install no dependency, change no live setting, and clean its temporary directory on success or failure. The verify command and CI MUST run this package check. A failed check MUST return a nonzero exit code. The package MUST use @neruok/pi-magic8ball and MIT as selected by the user. The initial release used version 0.1.0; subsequent release versions require separate user authorization. The manifest MUST omit private: true and set publishConfig.access to public. The root lockfile identity MUST match the manifest. The tarball MUST include a complete MIT LICENSE file. This check MUST NOT imply authorization to publish. **AC-25** Changed behavior. Before implementation, inspect the manifest and verify workflow. The metadata check MUST fail for absent discovery/repository metadata. The integration check MUST fail because the check:package command is absent. After implementation, both checks MUST pass. Verify MUST include the package test through its test-file glob and CI MUST invoke verify. Preserved behavior. Pack the baseline with scripts disabled into a private temporary directory. PASS if the packed TypeScript entry loads through Pi 1.0.4 without adjacent node_modules and registers the four tools and magic8ball command without errors. PASS if the packed benchmark dry run returns 24 comparisons and null builder-request and total-model-call maxima for small and other configurations, under REQ-29. PASS if every packed path matches REQ-24 and the script cleans its private temporary directory. Release metadata check. PASS if the scoped name, MIT license field, public access, and non-placeholder semantic version match the release candidate. PASS if both root lockfile identity entries match the manifest and the source license contains the MIT grant and disclaimer. PASS if the packed LICENSE matches the source file and the packed manifest retains the release metadata. The release metadata and packed-license checks MUST fail before applying the user-selected release values and pass afterward. These checks MUST use offline host loading, not provider calls or live settings. All 83 existing tests and strict typechecking MUST remain passing. - #REQ-24 verified-by: [#AC-25](#AC-25) User-selected release verification: the new AC-25 release-metadata check failed for the old unscoped name. The packed-license check failed because the baseline tarball had no LICENSE. The two existing metadata/integration checks passed. After applying the selected name, version, MIT license, and public access, all four package checks and all 87 tests passed. Strict typechecking and git diff --check passed. The lockfile update changed only root identity and license metadata. The packed LICENSE matched the source file. Packed host loading and the offline benchmark remained passing. Registry ownership and credentials remain unverified. No publication, commit, tag, or push occurred. ## Supplemental evidence and usage guidance {#supplemental-evidence} The user requested supplemental evidence and static tool guidance. Neither feature changes settings, model selection, or evidence tools. Supplied evidence enters the builder, not the classifier directly. Labels describe caller claims, not verified provenance. The request retains its original 16000-byte limit after omitting the supplied field. Separate limits permit 32 KiB of supplied JSON. Byte limits include JSON escaping and labels. No input truncation is permitted. Offline verification: baseline strict typechecking and all 96 tests passed. Before implementation, ten changed-behavior checks failed for unsupported supplied input, unsupported schemas, or absent promptSnippet. The omitted-input preservation check passed before and after. No existing test assertion changed. After implementation, all eleven checks in tests/supplied.test.mjs passed. Strict typechecking and the full 107-test suite passed. The full suite includes packed-package loading without adjacent dependencies and the offline benchmark check. | Criterion | Check names in tests/supplied.test.mjs | Before | After | | --- | --- | --- | --- | | AC-28 | supplied entries preserve strings/order, optional labels, and duplicate labels independently of scopes | Failed | Passed | | AC-28 | separate serialized entry/array budgets accept boundaries and count UTF-8, labels, and escapes | Failed | Passed | | AC-28 | malformed or over-budget supplied input fails before preparation with no partial work | Failed | Passed | | AC-28 | remaining request keeps the inclusive 16000-byte cap even with supplied evidence | Failed | Passed | | AC-28, AC-29 | preserved omitted input keeps the original cap, scope/hint checks, and source-free flow | Passed | Passed | | AC-28, AC-29 | empty supplied array adds no sources and preserves scope controls | Failed | Passed | | AC-29 | builder separates supplied content and claims from scopes, keeping stable IDs and metadata-only ledger | Failed | Passed | | AC-29 | supplied namespaces coexist with conversation and eight workspace calls without duplicate content or budget changes | Failed | Passed | | AC-29 | exact supplied citations reject labels, unavailable IDs, and disabled scopes before classification | Failed | Passed | | AC-29 | registered tool forwards only validated state to classification and accepts supplied ledger output schema | Failed | Passed | | AC-30 | registration promotes bounded judgment and routes user clarification in the pinned Pi prompt | Failed | Passed | Each check name starts with its criterion IDs. Existing AC-6 registration checks passed before and after. Fixtures use private settings, synthetic evidence, and mocked builder/classifier responses. No live settings changed. These checks establish input boundaries, data flow, and prompt construction, not model compliance, decision quality, or increased usage. The initial offline verification did not include a live builder/classifier call. Quality evaluations, physical-terminal checks, and hosted CI remain unexecuted for this change. After reload, one user-authorized live smoke call used only synthetic context.supplied evidence. Conversation and workspace sharing were false. The supplied median latencies were 18.2 milliseconds for A and 11.4 milliseconds for B. The configured builder was openai-codex/gpt-6-luna with high reasoning. The classifier was openrouter/cloudflare/clef-flash. The result succeeded and selected implementation_b. Validated state cited supplied1 and preserved the synthetic-data limitation. The ledger reported scope supplied, the caller label, truncated false, and zero workspace evidence calls. The returned distribution was 0.0114 for A, 0.9822 for B, and 0.0063 for abstention. Backend confidence was 0.9733. Usage was complete, with 1080 input tokens, 154 output tokens, and reported cost 0.00018226. No live setting changed. This checks one loaded-runtime/provider path, not calibration, general extraction quality, or increased tool usage. **REQ-25** The tool MAY accept `context.supplied`, an ordered array of at most eight objects with only `content` and optional `label`. Each content MUST be a string that is nonempty after trimming. A present label MUST meet the same rule. Validation MUST preserve strings and order. Duplicate labels MUST be permitted. Omission and an empty array MUST supply no evidence. Null, invalid entries, unknown fields, and exceeded limits MUST fail before preparation. Each serialized entry MUST contain at most 8192 UTF-8 bytes. The serialized array MUST contain at most 32768 UTF-8 bytes. The tool MUST measure both limits with JSON.stringify, including escaping, labels, punctuation, and array separators where applicable. The tool MUST reject invalid input as invalid-input without truncation or model calls. Pi schema errors retain REQ-1 precedence. Supplied evidence MUST remain available when conversation and workspace sharing are false. It MUST NOT authorize workspace reads. The collector MUST assign supplied1 through suppliedN by accepted array position, independently of labels and workspace-call counts. Each supplied ledger entry MUST contain scope supplied, fixed source caller-supplied evidence, truncated false, and optional unchanged label. The ledger MUST NOT contain raw content. The builder MUST receive each entry's ID, label when present, and unchanged content. Supplied entries MUST appear once in the initial builder message, separately from permitted scope booleans and file hints. Every builder turn MUST list available source IDs. Exact source-ID validation MUST include supplied entries and reject invented or label-based citations. Supplied entries MUST NOT increment workspace-call counts. REQ-29 removes workspace-call ceilings and count-based finalization. Builder instructions MUST treat supplied content as untrusted evidence and ignore embedded instructions. Instructions MUST identify labels as descriptive, not verified provenance. Conclusions, recommendations, rankings, and preferences MUST remain unverified claims. The classifier adapter MUST continue to receive only validated state plus the question and response descriptions. Raw supplied entries MUST NOT be forwarded separately to classification or copied into the source ledger. Existing state limits, tool permissions, settings, usage, transcript controls, deadlines, and advisory-only behavior MUST remain unchanged. Prompt instructions and source validation MUST NOT be described as proof of factual accuracy or resistance to prompt injection. **AC-28** Changed behavior. Validate labeled and unlabeled supplied entries with both scope flags false and duplicate labels. PASS if content, labels, and order survive unchanged. PASS if full requests above 16000 bytes accept valid independent supplied budgets. Check inclusive 8192/32768-byte JSON boundaries and reject the next byte, with multibyte text, escaping, and long labels. Check eight entries and reject nine. Null, blank, nonstring, malformed, and extra-field entries MUST fail before preparation. Changed behavior. An empty supplied array MUST add no sources. The remaining request MUST accept 16000 bytes and reject 16001. Preserved behavior. Calls that omit supplied evidence MUST add no supplied sources and retain the existing 16000-byte request boundary. Invalid scopes, file hints, and other fields MUST retain their prior validation. **AC-29** Changed behavior. Through the registered tool with private settings and mocked models, supply external observations and instruction-like claims. PASS if the first builder message carries unchanged evidence with stable supplied IDs outside the scope controls. PASS if prompts identify untrusted claims and labels, and supplied content occurs once across retained messages after workspace collection. Collect conversation, supplied evidence, and more than eight workspace observations. PASS if namespaces remain distinct and the next request retains both state tools and permitted workspace tools. PASS if valid supplied citations survive state validation and the output schema accepts the metadata-only ledger. Invented, absent, label-based, and disabled-scope citations MUST fail before classification. PASS if the classifier receives only state, question, and choices, without raw evidence canaries or labels outside state. Preserved behavior. Calls without supplied evidence retain the original scope controls, source IDs, ceilings, usage, and classifier input contract. Tests MUST make no provider request or live settings change. These checks verify data flow and instructions, not model compliance. **REQ-26** The main tool MUST register a one-line promptSnippet for independent judgment when multiple approaches remain. Static promptGuidelines MUST name magic8ball in each bullet and promote use after evidence inspection at consequential judgment forks. Guidelines MUST distinguish user clarification from independent judgment. They MUST NOT assume a user-question tool is installed. They MUST discourage use for explicit user choices, required policy, clear evidence, and trivial decisions. They MUST request concrete alternatives with neutral descriptions and supplemental observations when evidence is otherwise unavailable. They MUST discourage duplicate conversation or workspace content and caller recommendations in context.supplied. They MUST retain advisory responsibility and authorization boundaries and MUST NOT override provider-spending restrictions. The extension MUST NOT add dynamic reminders, eagerness settings, hooks, or automatic calls for this change. Documentation MUST explain routing and supplemental evidence without claiming measured usage or decision-quality improvements. **AC-30** Changed behavior. Inspect main-tool registration and build the pinned Pi default system prompt with its snippet and guidelines. PASS if the active tool has discovery guidance and consequential-fork, clarification, trivial-choice, neutral-description, supplied-evidence, and advisory rules. PASS if each guideline names magic8ball, inactive-tool guidance is absent, and no installed user-question tool is assumed. Preserved behavior. Registration still exposes four tools, unchanged lifecycle hooks and commands, and no automatic model calls. The check MUST use the installed host API offline. It MUST NOT claim increased usage from prompt construction alone. - #REQ-25 verified-by: [#AC-28](#AC-28) - #REQ-25 verified-by: [#AC-29](#AC-29) - #REQ-26 verified-by: [#AC-30](#AC-30) ## Release 0.1.1 {#release-0.1.1} The user authorized the version change, commit, push, and npm publication. The release candidate is @neruok/pi-magic8ball@0.1.1 under MIT. It includes the UTF-8 continuation fix, retained assistant prose, supplemental evidence, and static usage guidance. The manifest and root lockfile identities match version 0.1.1. Development dependencies and tested host versions remain unchanged. Agent-level tool-selection evaluations remain outstanding. The existing prompt tests do not establish appropriate invocation behavior. ## Incremental builder state {#incremental-state} The user authorized builder-only state tools and completion by normal stop. The user rejected fixed request and tool-call ceilings as pricing controls. Locally hosted models have no provider price. Usage metering remains informational. The invocation deadline, cancellation, and data-size checks remain active. **REQ-27** The extension MUST own a new state accumulator for each builder invocation. Only the builder MUST receive `magic8ball_set_goal`, `magic8ball_set_constraints`, `magic8ball_set_observations`, `magic8ball_set_evidence`, and `magic8ball_set_uncertainties`. These tools MUST NOT be registered in the parent tool catalog or sent to the classifier. Each tool MUST accept exactly one `value` field and replace its complete state section. The goal value MUST be a nonempty string. Constraints, observations, and uncertainties MUST contain arrays of nonempty strings. Observations MUST replace `current_state`. Evidence MUST contain exactly nonempty `fact` and `source` strings in each entry. An evidence source MUST equal an ID already collected in this invocation. The extension MUST validate each update before committing it. Unknown fields, invalid values, unknown citations, and oversized state MUST fail as `invalid-state`. The full serialized state MUST contain at most 12000 UTF-8 bytes. An invalid update MUST NOT partially commit. Calls in one response MUST execute in their returned order. Replacements MUST permit correction and removal of earlier entries. A successful update MUST return confirmation to the builder. The next request MUST show the owned state and initialized sections. The extension MUST NOT persist the accumulator or share it between invocations. **AC-31** Changed behavior. Build and correct state through all five tools while collecting evidence. Stop with arbitrary prose or no text. PASS if classification receives only the corrected owned state and ignores final prose. PASS if state tools remain available with both evidence scopes disabled, while workspace tools remain unavailable. PASS if state updates execute locally and never call `ctx.executeTool` or appear in parent registration or classifier context. Check exact schemas, ordered replacements, explicit empty sections, unknown citations, invalid values, extra fields, and the inclusive 12000-byte state boundary. PASS if each invalid update rejects without classification and no state survives a later or concurrent invocation. **REQ-28** A builder response with normal `stop` and no tool calls MUST finish collection from owned state. The extension MUST NOT parse final builder prose as state or use it as a fallback. All five sections MUST have explicit successful updates before classification. An explicit empty array MUST count as initialized. A missing section MUST fail final validation as `invalid-state`. Cancellation, timeout, malformed content, and non-normal stop MUST NOT classify partial state. The extension MUST validate complete state and citation IDs again before classification. The decision result schema and classifier API MUST remain unchanged. **AC-32** Changed behavior. Stop without any updates, after only a goal update, or with final text that contains a complete JSON state. PASS if each run returns `invalid-state` without classification. After valid updates, test `length`, `error`, `aborted`, malformed content, cancellation, timeout, and late completion. PASS if none classifies partial state or changes a returned result. Preserved behavior. Citation identity, usage aggregation, deadline propagation, permission hooks, result schemas, and no automatic retry remain passing. **REQ-29** The builder MUST NOT enforce a fixed model-request, workspace-call, or state-update count ceiling. Workspace source IDs MUST continue as `e1`, `e2`, and subsequent integers without a count ceiling. Supplied IDs and file-hint limits MUST remain independent of workspace-call counts. Every request MUST retain both permitted workspace tools and builder state tools until normal stop, failure, cancellation, or deadline. The existing 2048-token output hint, invocation deadline, input bounds, per-read bounds, and state byte ceiling MUST remain active. When the catalog advertises a context window, the extension MUST check estimated request context before each provider request. The builder check MUST include system prompt, tool schemas, messages, owned state, and source IDs. It MUST reserve the output hint. The estimate MUST use the ceiling of serialized logical-context character length divided by 3.5, consistently across supported host versions. The classifier check MUST include validated state, question, and choices. An estimate above the available window MUST fail as `budget-exhausted` before the provider request. The estimate MUST NOT be described as exact tokenization. Provider context validation remains the exact boundary. The extension MUST NOT silently evict history, summarize it through another model, retry, or switch models. **AC-33** Changed behavior. Complete exploration with more than four builder requests, more than eight workspace calls, and more than eight state updates. PASS if all complete within the deadline and citations beyond `e8` validate. PASS if state updates do not increment workspace counts and missing-path observations do not force finalization. Inject a small advertised context window and a growing transcript. PASS if the next over-window estimated request fails before a provider call or classification. Test the equal-window boundary, reserved builder output, initial request contents, tool declarations, and classifier request contents. Preserved behavior. Cancellation, configured deadline, read restrictions, per-source truncation, and all retained byte ceilings remain active. **REQ-30** User-enabled transcripts MUST capture allowlisted state-update arguments and local confirmation results. State argument capture MUST retain only the declared `value` shape and declared evidence entry fields. Hidden reasoning, signatures, images, authentication fields, raw provider errors, and unknown fields MUST remain excluded. Transcript capture MUST retain its existing memory-only lifecycle, byte ceiling, and user-only control. **AC-34** Changed behavior. Capture a builder invocation with state updates and local confirmations. PASS if the transcript shows each declared value and confirmation but excludes injected unknown fields and hidden provider metadata. Preserved behavior. Capture remains default-off, memory-only, cleared on off and reset, clipped at its existing byte ceiling, and closed before late writes. - #REQ-27 verified-by: [#AC-31](#AC-31) - #REQ-28 verified-by: [#AC-32](#AC-32) - #REQ-29 verified-by: [#AC-33](#AC-33) - #REQ-30 verified-by: [#AC-34](#AC-34) - #REQ-29 supersedes: [#REQ-11](#REQ-11) Context estimation is a preflight check, not a certified hard token cap. The 3.5-character heuristic matches Pi 1.1; Pi 1.0.4 used four characters. The implementation keeps 3.5 across both hosts, rather than importing a host-dependent estimator subpath that fails packed-extension loading. When catalog context-window metadata is absent, the window preflight cannot run. Byte bounds and the configured deadline remain active. No live Mellum compatibility or model-quality claim follows from offline acceptance checks. Implementation verification: the first nine AC-31 through AC-34 checks failed before implementation. The final `npm run verify` passed strict typechecking and all 120 tests on Pi 1.1.0 and in a private clean copy with the pinned Pi 1.0.4 dependencies. Checks include private builder tools, ordered replacement, citation availability, explicit empty initialization, UTF-8 limits, alias isolation, abnormal endings after complete state, late cancellation/timeout results, growth beyond the removed ceilings, context preflight, capture filtering, and usage aggregation. Packed extension loading without adjacent dependencies and mocked native Responses SSE also passed on both hosts. The legacy synthetic fixture adapter is test-only. Dedicated incremental-state tests exercise the actual tool protocol without that adapter. An initial clean-copy check exposed a version-dependent estimator assertion: Pi 1.0.4 used four characters per estimated token, unlike Pi 1.1's 3.5. The corrected test checks the specified 135-character/39-token boundary independently of host version. Offline success does not certify live provider tool-call behavior, semantic neutrality, decision quality, terminal behavior, or exact token counts. This change does not authorize a release, commit, tag, push, or live benchmark. ## Release 0.1.2 {#release-0.1.2} The user separately authorized a version bump, commit, push, and npm publication after the incremental-state implementation. The release candidate is @neruok/pi-magic8ball@0.1.2 under MIT. It includes private state replacement tools, owned-state completion by normal stop, removal of exploration count ceilings, context-window estimates, and updated transcript and benchmark reporting. Usage remains informational. Byte bounds, cancellation, and the configurable invocation deadline remain active. Development dependencies are unchanged. The implementation passed strict typechecking and 120 offline tests on Pi 1.1.0 and a private clean copy with pinned Pi 1.0.4. Live Mellum compatibility, exact token counts, model quality, and agent-level tool-selection evaluations remain unverified. This authorization covers this release; it does not authorize later releases or a live model benchmark. ## Shared behavior release {#shared-behavior-release} The 0.2.0 release uses @neruok/pi-common@0.1.0 as an exact runtime dependency. A clean checkout installs the library from npm. It does not need a sibling library repository. Host SDKs remain peer dependencies. The release adds no provider call, command, domain schema, or spending default. The library owns model selection, reasoning discovery, settings transactions, command reporting, and text sanitization. Advisor and magic8ball also share completion mechanics. Domain model eligibility, trust, paths, budgets, and result schemas remain local. Magic8ball RPC labels use provider/model without slash spacing. Safe settings reports reach non-UI sessions without starting a turn. Compaction settings now refuse foreign locks and immediate-parent symlinks. Explicit reset still repairs malformed regular files. Shared writes do not reclaim stale locks or guarantee power-loss durability. Inspect a failed save before retry. Transcript viewing retains UI-only delivery. Captured transcript text does not enter custom session messages. Release checks use independent clean installs and Pi's package manager. Registry availability and integrity require separate verification after upload. **COMMON-REQ-1** The release MUST declare @neruok/pi-common as its sole runtime dependency, pinned to 0.1.0 without a file or workspace path. Host SDKs MUST remain peers and MUST NOT be bundled. The extension MUST use the common behavior governed by pi-common CC-REQ-1 through CC-REQ-5. Domain command grammar, model eligibility, scope authority, spending defaults, result schemas, and transcript privacy MUST remain local. Release checks MUST install the published common dependency without a sibling checkout and load the consumer through its tested Pi host. README release and installation examples MUST identify the manifest version. Historical release sections are exempt. **COMMON-AC-1** Changed behavior. The release dependency check fails for the earlier local file dependency and passes for exact registry 0.1.0. The source and lockfile identify the same release version and common dependency. The lock records registry resolution and verified integrity without a source link. Clean clone verification, packed npm installation, and isolated Pi package installation pass without sibling sources or provider calls. The installed extension registers its expected tools and command. Safe show and cancellation operations use the shared behavior without model generation. Registry metadata and the downloaded release tarball agree with the inspected artifact and remote source commit. Changed behavior. A README-version regression fails for stale 0.1.2 release examples and passes after the documentation correction. - #COMMON-REQ-1 verified-by: [#COMMON-AC-1](#COMMON-AC-1) - #COMMON-REQ-1 depends-on: [pi-common#CC-REQ-1](../../../../pi-common/worktrees/main/docs/pi-common.md#CC-REQ-1)