# Pi Web Access - Changelog All notable changes to this project will be documented in this file. ## [Unreleased] ## [pi-ext-int-search 1.7.0] - 2026-10-12 (fork release) Synced to upstream v0.38.0, which now carries four fixes this fork sent back (#544 Perplexity deadline, #545 Brave markup, #546 AUTO empty-provider fall-through, #547 snippets in the summary prompt). Upstream's own versions are used; the fork copies are gone. ### Fixed - **History search could return stale sessions and drop today's.** The second pass handed every digest file to one ripgrep call, and ripgrep searches them in parallel and prints whichever finishes first — so the byte budget cut an arbitrary set, not the oldest. On a CI runner the newest session never printed at all while 17 older ones filled the budget. The scan now runs in batches of 8 files, newest batch first, sharing one budget. - **A deadline looked like a cancellation** (upstream's take on #544): a Perplexity request that hits its own 30 s limit is reported as a 408 timeout, so the chain moves on instead of treating it as the caller giving up. - **A clamped snippet overran its budget by one character** (upstream's take on #547): the ellipsis is now charged to the limit. - **CI could not install or run this package off a developer machine.** The lockfile recorded one @esbuild platform instead of 26, so `npm ci` failed on Linux, and the runner had no ripgrep, so a third of the history tests failed for a missing tool. Both are fixed in the workflow and the lockfile. ## [pi-ext-int-search 1.6.4] - 2026-10-10 (fork release) Same content as 1.6.3, which npm left stuck in staging and never served. ### Fixed - **The memory source needed a `sqlite3` binary nobody installs.** On a phone without it the source reported `crashed` and `memory_search` answered that there were no matches — a confident empty answer. The recall database is now read through Node's own SQLite; the CLI stays as the fallback for runtimes without it. ### Added - `scripts/smoke-fleet.mjs`: runs each tool through a real `pi -p` on every machine and treats an empty answer as a failure, because that is how both of this week's field bugs presented. It separates a dead feature from an unreachable host and from a truncated-but-working scan. Its first run found the two bugs fixed in 1.6.2 and 1.6.3 plus the hosts where Perplexity is simply not configured. ## [pi-ext-int-search 1.6.2] - 2026-10-10 (fork release) ### Fixed - **History search found nothing on hosts without a system ripgrep.** Pi installs its own `rg` into the agent bin directory and leaves that directory off `PATH`, so a bare `rg` works on a laptop and fails on a server: two of three `memory_search` sources came back as `source FAILED (tool missing/crashed)` on a host where pi had installed ripgrep itself. The binary is now resolved once — `PATH` first, then pi's own copy. Found by asking the tool on a remote machine, not by a test. ## [pi-ext-int-search 1.6.1] - 2026-10-10 (fork release) ### Fixed - **A proxied Perplexity search reached a route the gateway does not have.** Upstream v0.37.0 moved the provider onto the Search API; measured against the gateway this fork talks to, `POST /v1/search` answers 405 while `POST /v1/chat/completions` answers 200. Under proxy mode the provider now searches through chat completions again, citation trimming included, so `[n]` markers keep pointing at the right source. Direct installs keep upstream's Search API path. Found by probing the gateway after the merge, not by a test. ## [pi-ext-int-search 1.6.0] - 2026-10-09 (fork release) ### Security - **`@modelcontextprotocol/sdk` 1.27.1 → 1.32.1.** The old range (1.12.0–1.30.1) carries a high-severity advisory: the OAuth client could send credentials to an attacker-controlled authorization server. Taken from upstream rather than patched locally. `npm audit` is clean, runtime and full. ### Merged - Upstream `nicobailon/pi-web-access` v0.35.0 → **v0.37.0** (27 commits): the tool implementations moved out of `index.ts` into `web-tool-core.ts`, a standalone core for hosts without Pi, a new `degoog` provider, and Perplexity switched from chat completions to the Search API with `formatSearchResultsAsAnswer`. ### Fork deltas preserved through the merge - `memory_search` registers through upstream's own tool-name machinery — `memorySearch` now lives in `web-tool-core.ts`'s `ToolNames`, so renames and duplicate checks cover it. The standalone core disables it: session history needs the Pi host. - Perplexity keeps the fork's plumbing on top of upstream's rewrite: endpoints through `provider-endpoints.ts` (the Search path is derived from the configured chat URL, so an override moves both), credentials through `resolveProviderKey` (gateway key under proxy mode), `fetchWithCredentialRedirects` stripping `Authorization` on redirects, redacted errors, and the 30-second request deadline. - OpenAI search keeps destination-first credential gating; only the default model follows upstream (`gpt-6-luna`). ## [pi-ext-int-search 1.5.1] - 2026-10-03 (fork release) ### Fixed - **The summary prompt carried no evidence.** Each source reached the summary model as `title — url` only, so with no provider answer text the model honestly reported "no body text" — which is every `memory_search` summary and any raw-result web provider. Snippets now travel with their source, bounded: 600 chars per source / 12 000 per query when there is no answer to lean on, 200 / 3 000 when the provider already answered. Found by running the thing against real questions, not by a unit test. - **Those snippets then blew the generation deadline.** A two-query web search with full snippets overran the default 30 s on `gemini-3.8-flash` and degraded to the deterministic summary. The split budget above keeps the answer-present case near its previous size; both web and history now summarise inside the default deadline. ## [pi-ext-int-search 1.5.0] - 2026-10-03 (fork release) ### Added - **`memory_search` now collects hand-written markdown by default**, not just chat transcripts and recall memories: project docs, `CLAUDE.md`/`AGENTS.md`, and the `~/.claude/memory` notes tree (MEMORY.md, `entities/`, `decisions/`, `journal/`) — the last of which no scope reached before. Git history stays an additive opt-in detected from the question, because walking commits and expanding diffs is the expensive source. The `wantsDocs()` phrase detector is gone: it decided nothing once docs became a default, and a question that happened not to say "docs" no longer misses a decision that was written down rather than chatted about. - **`memory_search` gained the same `workflow` pipeline as `web_search`.** `workflow: "auto-summary"` runs the history matches through the identical summary path — same `summaryModel`, `summaryInstructions`, `summaryGenerationDeadlineMs`, same no-invented-sources guardrails, same deterministic fallback. Only the prompt's domain wording differs (`kind: "history"`), and the Sources section lists `source · date · project` references rather than URLs, because a transcript has no URL to cite. The ranked snippets still follow the summary, and a failed generation degrades to the deterministic summary instead of failing a search that already succeeded. ### Changed - **The session digest advances by tail, not by full re-read.** A manifest entry now records `offset` — how many bytes of a transcript are already digested, always on a line boundary — so the live session, which grows between queries, is read from where the last pass stopped instead of from byte zero. Digest format bumped to v3 (one rebuild on first use). A shrunk or rewritten source, a hand-deleted digest, or an older manifest still fall back to a full re-digest, and a half-written trailing line stays outside the offset until it is complete. ## [pi-ext-int-search 1.4.0] - 2026-10-02 (fork release) ### Changed - **Synced with upstream v0.35.0** (76 commits since our v0.28.0 base). Inherited: Z.ai GLM and You.com search providers, Crawl4AI self-hosted extraction, `web_enable` lazy tool activation (`toolActivation: auto | dynamic | eager`), Brave adaptive rate-limit queue, Tavily 20-key pool with failover, `summaryInstructions`, Cloudflare challenge fallback for fetches, Exa `/search` for every keyed request, OpenAI `openaiUseProviderBaseUrl` and ChatGPT-OAuth routing to the official Responses API, and a precompiled `dist/` for published installs. ### Fork deltas preserved through the merge - **Destination-first key binding** survives upstream's new OpenAI auth paths: model-registry credentials still reach only OpenAI-owned origins — or the user's own provider `baseUrl` when `openaiUseProviderBaseUrl` opts in — and a JWT-shaped proxy key is still never redirected to `chatgpt.com`. - **Unified proxy mode**: Tavily's new numbered key pool (`TAVILY_API_KEY_1…20`) is disabled when the Tavily destination resolves to the gateway origin — i.e. when `tavilyBaseUrl` points at it, since Tavily has no gateway route of its own — so vendor keys never reach a gateway that has its own bound credential. - `redactProviderError` (bounded + pattern-redacted upstream error bodies) reapplied to the rewritten Tavily and OpenAI request paths; Brave keeps the fork's HTML-entity `stripHtml` for titles and snippets. - AUTO-mode empty-result fallback (`EmptyResultError` / `AutoFallback`) now composes with upstream's `allowedProviders` gating instead of being replaced by it. - `memory_search` keeps its own registration gate and renamable tool name alongside upstream's `web_enable`; the tool-definition compatibility hashes for the four shared tools still match upstream byte for byte. Because `memory_search` searches local history only, it is registered up front instead of waiting behind the activation loader, so the dormant-schema budgets are raised from 700 B / 11 924 B to 2 600 B / 13 900 B. ### Note - The test suite now needs **Node 24+** (upstream CI's version). On Node 22 the ten upstream `brave-rate-limit` tests fail with `Promise resolution is still pending but the event loop has already resolved` — verified identical on a pristine upstream checkout, so it is a runtime artifact, not a fork regression. Full suite on Node 25: **1122/1122**. ## [pi-ext-int-search 1.3.2] - 2026-10-02 (fork release) ### Fixed - **Pi startup warning "Host-provided extension packages must be declared in peerDependencies".** `typebox` moved from `dependencies` to `peerDependencies` with a `"*"` range (pinned as an exact devDependency for reproducible typecheck), so the installed copy can no longer bypass Pi's extension module mapping and create a duplicate TypeBox registry. Port of upstream PR [#448](https://github.com/nicobailon/pi-web-access/pull/448) (issue [#442](https://github.com/nicobailon/pi-web-access/issues/442)); the packed-install test now asserts the peer contract instead of the old bundled-copy contract. - **Runtime security: `undici` bumped to `^8.11.2`** (was `^8.9.0`, which resolved into the vulnerable `8.0.0–8.10.1` range: TLS validation bypass in `BalancedPool`, response splitting/truncation via interceptors, cache poisoning and several DoS advisories). `npm run audit:runtime` is now clean. ## [pi-ext-int-search 1.3.1] - 2026-09-05 (fork release) ### Changed - Synced with upstream **v0.28.0** (our PRs #353 / #354 landed there with credit; upstream's unreleased-code cleanup; xAI `x_search`, Mistral search, parallel batch queries — see the 0.28.0 entry). - Fixed a fifth upstream test race (`github-issue-pr` REST-abort test raced process spawn; now asserts the ordering invariant instead of wall-clock) — upstream PR #359. ## [pi-ext-int-search 1.3.0] - 2026-09-04 (fork release) ### Fixed - **`memory_search` sessions source silently dropped ~97 % of matches.** The scan OR-ed every query token; corpus-wide words ("model", "error") match >50 % of transcript lines and a line is a whole API message (toolResult dumps up to 2.7 MB), so rg emitted 10.8 GB for one query, Node killed it at the output buffer and kept whatever came first on disk — never the newest session. A question whose answer was in the same day's transcript came back `partial` with no relevant hit. Now a two-pass funnel: the rarest content token filters lines, scanned newest-first under a byte budget, so a cut drops OLD data and is reported. Russian question words (`какой/когда/почему…`) and bare numbers never become the filter. - `web_search` now prints the stored `responseId` so the model can call `get_search_content` on search results (upstream PR #354). ### Changed - **Sessions are searched through a text-only digest cache** (`~/.pi/agent/memory-search-cache/sessions/`): one small file per session with `{ts, role, text}` of user/assistant turns — 2 % of raw transcript size (toolResult dumps are never searchable and are not stored). Rebuilt incrementally by size/mtime; cold start is budgeted per call, newest-first, and labelled `partial` until complete (~12 min for 15 GB). Warm searches: 8–20 s → ~1–3 s, no truncation. - Upstream test races fixed (upstream PR #353). ## [pi-ext-int-search 1.2.0] - 2026-09-03 (fork release) Merged upstream `nicobailon/pi-web-access` **v0.14.0 → v0.27.0** (180 commits, 13 releases; see the entries below for everything inherited: 17 new search providers incl. keyless DuckDuckGo and self-hosted SearXNG/Firecrawl, `provider:"all"` / provider arrays, `fetch_content` `mode:"raw"|"answer"`, `get_search_content findText`, GitHub PR/issue specialization, Gemini ADC + PDF→Markdown, external fetch cache, transport `proxy` via curl, per-provider `*BaseUrl` overrides with redirect credential stripping). ### Breaking (fork users) - `braveBaseUrl` / `tavilyBaseUrl` / `exaBaseUrl` are now **API bases** (upstream 0.24 semantics: the provider appends `/web/search`, `/search`, …). In 1.1.0 brave/tavily were full endpoint URLs. A config like `"braveBaseUrl": "https://gw/v1/brave/search"` must become `"https://gw/res/v1"`-style. Unified proxy (`proxyBaseUrl`) users are unaffected — the gateway routes are unchanged (`/v1/brave/search` stays full). - Per-provider `*BaseUrl` / `*_BASE_URL` and `proxyBaseUrl` / `WEB_SEARCH_PROXY_URL` are now validated: absolute **HTTPS**, no embedded credentials, no query/fragment. `openaiResponsesUrl` is HTTPS-only (upstream accepts `http:`). An explicitly empty `*_BASE_URL=""` is an error, not "unset". ### Fixed (security) - **Destination-first key binding now branches on destination, not on key presence.** A proxied destination with a missing `proxyApiKey` is *unavailable*; it never falls back to a personal key (env / `$ENV` / `!cmd` / model-registry). Previously `WEB_SEARCH_PROXY_URL` set + key missing sent the personal key to the gateway. - `proxyBaseUrl` itself is validated (it was the one URL receiving every key and had no HTTPS/credential check). - Redirect credential stripping extended to Perplexity, OpenAI and **Gemini** (`x-goog-api-key` / `cf-aig-authorization` are custom headers undici does not strip cross-origin); upstream covers brave/exa/tavily. - Direct-Google detection for the ambient `GEMINI_API_KEY` now requires `https:`. - Model-registry OpenAI/Codex tokens are still gated to OpenAI-owned origins (`isOpenAIAuthOrigin`) — upstream sends them to any `openaiResponsesUrl`; a JWT-shaped proxy key is never redirected to chatgpt.com. ### Changed - `provider-endpoints.ts` v2: `kind: "base" | "full"` + `overrideSuffix`; `resolveProviderKey()` / `providerHasCredential()` / `isProxiedDestination()` are the only proxy-aware helpers — providers hold no proxy logic. Dead sync `providerApiKey` / `providerProxyApiKey` / `normalizeBaseUrl` removed. - Auto-mode empty-result fallback (`nonEmptyOrThrow` / `EmptyResultError`) now covers **every** auto provider (18), not the 9 that predate the merge; explicit / `all` / routing modes stay strict. - Pattern redaction (`redactError`) applied at the auto-chain aggregation point, so every provider's error text is scrubbed, not only the five with per-provider `redactProviderError`. - `memory_search` registers through upstream's tool-gate machinery: `tools.memorySearch.enabled`, `toolNames.memorySearch` (participates in rename + duplicate detection). - Tests are hermetic by default (`node --import ./test/hermetic-env.mjs --test`): a developer's real `~/.pi/web-search.json` can no longer reroute mocked-fetch assertions. New `provider-endpoints` (15) and `proxy-key-binding` (3 configurations × 6 providers) suites. - Dropped the fork's "keep every Perplexity citation" patch in favour of upstream's `citationsToKeep` (keeps up to `max(numResults, highest cited [n])`, capped at 20). - Removed dead `.npmignore` (`files[]` governs the tarball); demo mp4 no longer shipped (referenced by URL). ## [pi-ext-int-search 1.1.0] - 2026-07-26 (fork release) Merged upstream `nicobailon/pi-web-access` v0.14.0 (see the 0.14.0 entry below for everything inherited: SearXNG/SERPdive/AnySearch providers, Firecrawl extraction, `source_check`, `$ENV`/`!command` credential sources, ordered search routing, fetch-content domain policy, configurable tool names). ### Fork changes on top of the merge - Rewired unified proxy mode onto upstream's async credential resolution: new `providerProxyApiKey()` (destination-first) is checked BEFORE `resolveCredential` in exa/brave/tavily/perplexity/openai, so a personal per-provider key never leaks to the proxy gateway. - Gemini destination-first binding preserved: ambient `GEMINI_API_KEY` only counts for direct Google hosts (`getApiKey`, `isGeminiApiAvailable`); `buildKeyParam` kept for key-binding guarantees. - `redactError()` (pattern-based secret scrubbing) layered on top of upstream's `redactCredential` in provider error paths. - Perplexity: kept the fork's own 30s request timeout and preservation of ALL citations regardless of `numResults` (answers reference sources by index). - Auto-mode empty-result fallback extended to the new searxng/serpdive providers: an HTTP-200-but-empty response keeps falling through the chain. - Kept `memory_search` (internal search), the bundled `librarian` skill (upstream removed theirs; ours ships in `files[]`), and `pi-ext-int-search` packaging. - Made `source_check` tests hermetic (they no longer pick up the developer's real `~/.pi/web-search.json`). ### Added - PDFs can be converted by a document parser you run yourself. Set `pdf.provider: "endpoint"` and `pdf.endpointUrl`, and `fetch_content` sends the PDF to that service and uses the Markdown it returns, with no API key or per-page cost. In `auto`, a configured endpoint is tried before Datalab, Gemini and the built-in parser, and a failed request falls back to them. The URL passes the same SSRF checks as other fetches, so a LAN host needs its range in `ssrf.allowRanges`. Thanks to [@goodman-b](https://github.com/goodman-b) for [PR #553](https://github.com/nicobailon/pi-web-access/pull/553). ### Changed - Search summaries now show the summary model each source's snippet, not just its title and URL, so providers that return ranked results without an answer get a summary of what the pages say. Snippets are capped per source and per query (600 and 12,000 characters, or 200 and 3,000 when the provider already gave an answer) to keep summaries within their deadline. Thanks to [@axelbaumlisto](https://github.com/axelbaumlisto) for [PR #547](https://github.com/nicobailon/pi-web-access/pull/547). ### Fixed - Single-URL fetches and completed search summaries now include model-visible storage IDs and retrieval calls, so stored content remains reachable without access to host metadata. Guidance uses the configured retrieval tool name, respects disabled retrieval, and leaves raw bodies and approved summary text unchanged. Thanks to [@mothieras](https://github.com/mothieras) for [issue #548](https://github.com/nicobailon/pi-web-access/issues/548) and [PR #551](https://github.com/nicobailon/pi-web-access/pull/551). - Truncated `fetch_content` answers now clarify that stored content is the original source, not the remaining answer, and offer source retrieval from offset zero instead of an answer offset. Thanks to [@mothieras](https://github.com/mothieras) for [issue #550](https://github.com/nicobailon/pi-web-access/issues/550) and [PR #552](https://github.com/nicobailon/pi-web-access/pull/552). - When a model sends `queries`, `provider` or `domainFilter` as a string that starts with `[` but isn't valid JSON, `web_search` now returns a tool error saying the field must be an array of strings, instead of running the whole string as one literal search. Thanks to [@tschnorpfeil](https://github.com/tschnorpfeil) for [issue #542](https://github.com/nicobailon/pi-web-access/issues/542) and [@MatrixNeoKozak](https://github.com/MatrixNeoKozak) for [PR #543](https://github.com/nicobailon/pi-web-access/pull/543). - In `auto` search, a provider that answers successfully but with no results and no answer no longer ends the search. The next available provider is tried, and the first empty response is returned only when every provider came back empty. Thanks to [@axelbaumlisto](https://github.com/axelbaumlisto) for [PR #546](https://github.com/nicobailon/pi-web-access/pull/546). - Brave search results no longer show raw markup such as `` and `—` in titles and snippets. Numeric entities outside the Unicode range stay as written instead of failing the search. Thanks to [@axelbaumlisto](https://github.com/axelbaumlisto) for [PR #545](https://github.com/nicobailon/pi-web-access/pull/545). - A Perplexity request that stalls now gives up after 30 seconds instead of holding the search for the whole turn. The timeout is reported as a provider error, so `auto` search and configured routing move on to the next provider instead of treating it as a cancelled search. Thanks to [@axelbaumlisto](https://github.com/axelbaumlisto) for [PR #544](https://github.com/nicobailon/pi-web-access/pull/544). - DuckDuckGo search now tells a bot check apart from an empty search. When DuckDuckGo answers with its bot check (HTTP 202 or a challenge page), the error says DuckDuckGo blocked the request and to use another provider, and configured `searchRouting` treats it like a rate limit (`quota`). A search that found nothing now returns no results instead of failing. Only a page with neither results nor a "no results" message still fails with "no parseable results", which now says the page format may have changed. Thanks to [@tschnorpfeil](https://github.com/tschnorpfeil) for [issue #541](https://github.com/nicobailon/pi-web-access/issues/541). - `fetch_content` with `mode: "raw"` on a PDF now says raw mode doesn't decode PDFs and suggests the default Markdown mode or `mode: "answer"` with a prompt, instead of only reporting an unsupported content type. Thanks to [@goodman-b](https://github.com/goodman-b) for [PR #555](https://github.com/nicobailon/pi-web-access/pull/555), split out of [PR #553](https://github.com/nicobailon/pi-web-access/pull/553). - `fetch_content` on a GitHub pull request or issue no longer repeats long bodies and comments: the start shows inline once and the rest is stored at the end of the document. Comments, review comments, files, commits, checks and reviews left out of the inline view are now stored too, so the retrieval hint that points at them finds them. Items GitHub was never asked for point at the `gh` command instead. Hints use the retrieval tool's configured name, or only `gh` when retrieval is turned off. Thanks to [@mothieras](https://github.com/mothieras) for [issue #549](https://github.com/nicobailon/pi-web-access/issues/549). - `fetch_content` answer mode with several URLs now shows each URL's answer, instead of only a list of URLs with character counts. The answers share the same output limit as a single-URL answer and say when one is cut short. The retrieval hint now says the stored content is the original pages, not the answers. See [issue #556](https://github.com/nicobailon/pi-web-access/issues/556). ## [0.38.0] - 2026-10-09 ### Highlights - Search with Ceramic, a new paid keyword search provider you can select. - OpenAI search now defaults to the newest Luna model, which costs about 20 times less per token than Terra. - Answer mode and query rewriting work with models whose provider signs in without an API key, such as pi-multiprovider models. - Requests through a proxy keep their method, headers, body and `Content-Length`, and HEAD requests stay HEAD requests. ### Added - New Ceramic search provider for [Ceramic](https://www.ceramic.ai)'s keyword web search API, used only when you select it. Set `ceramicApiKey` or `CERAMIC_API_KEY` to your Ceramic key. Ceramic is a paid API with free starter credits, so it is never picked by `auto` or `all`. Allowed domains are added to the query as `site:` filters, and excluded domains are removed from the results. Ceramic has no date filter, so `recencyFilter` is ignored. Thanks to [@sweepies](https://github.com/sweepies) for [issue #523](https://github.com/nicobailon/pi-web-access/issues/523). ### Changed - The `web_search` tool description is now four sentences on how to use the tool, instead of a paragraph listing every provider, the `all` policy and each workflow mode. The model still sees the providers and the `all` policy in the `provider` parameter and the modes in the `workflow` parameter. Thanks to [@mcwalrus](https://github.com/mcwalrus) for [issue #528](https://github.com/nicobailon/pi-web-access/issues/528). - OpenAI `web_search` now runs on the newest Luna model by default, such as `gpt-6-luna` on a ChatGPT subscription, instead of `gpt-5.6-terra`. Terra costs about 20 times as much per token as `gpt-6-luna` and is a generation older, so searches were using a large share of subscription usage. With only an API key, the default is `gpt-6-luna`. Set `openaiSearchModel` to pick a different model. Thanks to [@sslotin](https://github.com/sslotin) for [issue #520](https://github.com/nicobailon/pi-web-access/issues/520). - Search summaries and `fetch_content` answer mode now give Pi the session ID and let its `opencode` and `opencode-go` providers add OpenCode's `x-opencode-session` header, instead of building OpenCode's headers themselves. They no longer send `x-opencode-client`, and custom providers pointed at `opencode.ai` no longer get the session header from pi-web-access. See [issue #537](https://github.com/nicobailon/pi-web-access/issues/537). ### Fixed - With a proxy set, `fetch(new Request(...))` now sends the Request's own method, headers, body, redirect setting and abort signal. It used to arrive as a bare GET without them. HEAD requests through the proxy are now real HEAD requests with an empty response body instead of GETs. Thanks to [@xiaozhu1337](https://github.com/xiaozhu1337) for [issue #535](https://github.com/nicobailon/pi-web-access/issues/535) and [PR #536](https://github.com/nicobailon/pi-web-access/pull/536). - With a proxy set, HEAD responses keep the server's `Content-Length`, and a POST or PUT with no body sends `Content-Length: 0` as native fetch does, instead of omitting it (which some servers answer with 411). See [issue #539](https://github.com/nicobailon/pi-web-access/issues/539). - Keyless Parallel MCP searches no longer fail with Parallel's "free-tier rate limit" error. Requests went out with Node's default `undici` User-Agent, which Parallel rate-limits as one shared bucket, so they now send `User-Agent: pi-web-access`. Thanks to [@kestermcullough](https://github.com/kestermcullough) for [PR #531](https://github.com/nicobailon/pi-web-access/pull/531). - `fetch_content` answer mode and search query rewriting now work with models whose provider signs in without giving Pi an API key, such as models added by [pi-multiprovider](https://github.com/monotykamary/pi-multiprovider). Answer mode used to fail with "No API key available for answer model" even though Pi could call the model. Thanks to [@eikopf](https://github.com/eikopf) for [issue #530](https://github.com/nicobailon/pi-web-access/issues/530). - A host that serves several conversations at once from one process, with one extension instance per session (an app or chat server built on the Pi SDK), no longer loses results across conversations. Each instance now keeps its own session's results, background fetches and active flag: one conversation starting or ending used to clear every stored result, so a `get_search_content` in another conversation answered "No stored results for responseId …", and its background `includeContent` fetches were aborted. Sessions forked from one history hold the results they share until the last of them ends, and deleting a shared result from `/search` removes it only from that session. Pi itself, which runs one instance and switches its session, behaves as before. GitHub clones are removed when the last live session ends, and while sessions keep overlapping, a session's end removes the clones no fetch has returned since the oldest live session started. A session that restores results from history marks the clones they point at as used, so a session forked from another keeps its clone paths after the original ends. Thanks to [@kid7st](https://github.com/kid7st) for [PR #521](https://github.com/nicobailon/pi-web-access/pull/521). - `@modelcontextprotocol/sdk` is updated from 1.27.1 to 1.32.1, which fixes [GHSA-6qxp-vccf-f47h](https://github.com/advisories/GHSA-6qxp-vccf-f47h), so `npm audit` no longer reports it. That advisory covers the SDK's OAuth client, which pi-web-access doesn't use: its HTTP providers send API keys in headers and its MCP server runs over stdio. Thanks to [@M1racleShih](https://github.com/M1racleShih) for [PR #522](https://github.com/nicobailon/pi-web-access/pull/522) and [@SidShaytay](https://github.com/SidShaytay) for [issue #519](https://github.com/nicobailon/pi-web-access/issues/519). ## [0.37.0] - 2026-10-05 ### Highlights - Search through degoog, a self-hosted or public metasearch engine, with no API key needed. - Perplexity search results now include real page snippets and cost less per search. - Pi `codemode` scripts can work with search results and fetched pages as data instead of reading formatted text. - Failed tool calls are now marked as errors, so Pi and MCP clients can tell a failure from a success. ### Added - New degoog search provider for a self-hosted or public [degoog](https://degoog.org) metasearch instance, used only when you select it. It works without an API key and uses `https://degoog.org` by default. Set `degoogBaseUrl` or `DEGOOG_URL` to use your own instance, `degoogApiKey` or `DEGOOG_API_KEY` if your instance requires a key, and `degoogEngines` to limit which engines it queries. It supports domain and recency filters, routing, and the curator. Thanks to [@zhouzhuojie](https://github.com/zhouzhuojie) for [PR #515](https://github.com/nicobailon/pi-web-access/pull/515). - In Pi `codemode` scripts, `web_search` and `fetch_content` return data instead of the text the model sees: each query's results and errors, and each URL's full content and error, even when the call fails. Searches from a script don't open the curator unless the script asks for it, and `includeContent` waits for the pages instead of fetching them in the background. What the model sees and the MCP output are unchanged. See "In codemode scripts" in the README. [Issue #516](https://github.com/nicobailon/pi-web-access/issues/516). ### Changed - Perplexity `web_search` results now come from Perplexity's Search API, so each result has a real page snippet instead of an empty one, and searches cost less ($1 per 1,000 searches, with no token billing). Results no longer include an answer written by Sonar. YouTube summaries still use Sonar. Thanks to [@DWalland](https://github.com/DWalland) for [issue #512](https://github.com/nicobailon/pi-web-access/issues/512). - Failed tool calls are now marked as errors (`isError: true`) in Pi and over MCP. This covers invalid arguments and calls where nothing succeeded, such as a `web_search` or `source_check` where every search failed. Pi versions before 1.0 ignore the marker. `fetch_content` called without a URL now names the parameters to use, as `web_search` already did. Other messages are unchanged. Thanks to [@j-koester](https://github.com/j-koester) for [PR #514](https://github.com/nicobailon/pi-web-access/pull/514). ## [0.36.0] - 2026-10-04 ### Highlights - Use pi-web-access's search and fetch tools from Claude Code, Codex, Cursor, and other MCP clients with `npx -y pi-web-access`. - Docs sites that publish markdown versions of their pages now come back as clean markdown. - Brave prepaid keys keep working after the first search, and OpenAI search shows the real error, such as a usage limit. - Keep Firecrawl PDF costs down by reading only the first few pages of each PDF. - New Keenable search provider that works without an API key. ### Added - The search and fetch tools (`web_search`, `fetch_content`, `get_search_content`, and `source_check`) run as a local MCP server for Claude Code, Codex, Cursor, and other MCP clients. Start it with `npx -y pi-web-access`. It reads the same `web-search.json` and provider keys as the Pi extension and does not need Pi installed. Pi-only features such as the curator, summaries, and Kimi search are not available there. See "Use from other agents (MCP)" in the README. Thanks to [@Avg8888](https://github.com/Avg8888) for [issue #496](https://github.com/nicobailon/pi-web-access/issues/496). - `fetch_content` asks servers for markdown first, so sites that publish markdown versions of pages (Cloudflare, Mintlify, and other docs hosts) return clean markdown directly. `mode: "raw"` still asks for the server's normal representation. Thanks to [@erwinkramer](https://github.com/erwinkramer) for [issue #495](https://github.com/nicobailon/pi-web-access/issues/495). - `web_search` can restrict Exa results to a category such as `news` or `research paper` with the new `category` parameter. Other providers ignore it. Without an Exa API key, if Exa's filtered search is unavailable, the category is added to the query text instead. Thanks to [@SuTang-vain](https://github.com/SuTang-vain) for [PR #493](https://github.com/nicobailon/pi-web-access/pull/493). - New Keenable search provider, used only when you select it. It works without an API key through Keenable's public endpoint; set `keenableApiKey` or `KEENABLE_API_KEY` for higher limits. It supports domain and recency filters, routing, and the curator. Thanks to [@ilya-bogin-keenable](https://github.com/ilya-bogin-keenable) for [PR #506](https://github.com/nicobailon/pi-web-access/pull/506). - Set `firecrawlPdfMaxPages` to have Firecrawl read only the first N pages of each PDF, which caps per-page PDF billing on fetches and `includeContent` searches. Cut-off PDFs end with a note giving both page counts. Leaving it unset keeps the current behavior. Thanks to [@Dangooy](https://github.com/Dangooy) for [issue #507](https://github.com/nicobailon/pi-web-access/issues/507). ### Fixed - Brave search with a prepaid (pay-as-you-go) key no longer fails every call after the first with "quota exhausted; estimated reset in ~702h". Prepaid plans report a monthly rate limit of 0, which was read as an exhausted quota. Thanks to [@kk-code-lab](https://github.com/kk-code-lab) for [issue #501](https://github.com/nicobailon/pi-web-access/issues/501). - OpenAI web search shows the server's real error when a response fails partway, such as a ChatGPT subscription usage limit, instead of "no parseable response output". Falling back to the next provider still works. Thanks to [@ShinoharaHaruna](https://github.com/ShinoharaHaruna) for [PR #505](https://github.com/nicobailon/pi-web-access/pull/505). - `web_search` with `includeContent` keeps page content the search provider already returned and only fetches the pages it didn't cover. Before, it fetched every page again, and a failed fetch replaced usable content. - `web_search` accepts `provider`, `queries`, and `domainFilter` lists that the model sent as a JSON string, such as `provider: "[\"parallel-mcp\"]"`. Before, these calls were rejected before the search ran. Thanks to [@advaitpaliwal](https://github.com/advaitpaliwal) for [PR #491](https://github.com/nicobailon/pi-web-access/pull/491). - `fetch_content` reports a tool error when its only URL failed or every URL in a batch failed. Before, Pi recorded these as successful calls even though the text said `Error: ...`. Batches where some URLs succeed are unchanged. Thanks to [@MDGChamomile](https://github.com/MDGChamomile) for [PR #504](https://github.com/nicobailon/pi-web-access/pull/504). - `get_search_content` reads a fetched page without `url` or `urlIndex` when the fetch holds only one page. Before, calls like `{ responseId, findText }` failed with "No URL specified" and the model had to retry with `urlIndex: 0`. Thanks to [@j-koester](https://github.com/j-koester) for [PR #494](https://github.com/nicobailon/pi-web-access/pull/494). - When the model reads a long `source_check` result with `get_search_content`, each page now says how to get the next one, as search and fetch pages already did. - Local video analysis through the Gemini Web cookie fallback no longer comes back as "garbled characters", because uploads now send the file's type and name. Thanks to [@leonzyb](https://github.com/leonzyb) for [issue #508](https://github.com/nicobailon/pi-web-access/issues/508). ## [0.35.0] - 2026-09-30 ### Highlights - Tell the summarizer what matters to you, such as keeping prices, limits, and versions exactly as written. - Brave search holds up on low-rate plans instead of failing when several searches run at once. - OpenAI web search works when you signed in to Pi with your ChatGPT account. ### Added - Add your own instructions to search summaries with `summaryInstructions` in `web-search.json`, for example "keep prices, limits, and versions verbatim." They apply to summaries from the curator and the `auto-summary` workflow, and the built-in safety rules still apply. Thanks to [@rcharrisg](https://github.com/rcharrisg) for [PR #489](https://github.com/nicobailon/pi-web-access/pull/489). ### Fixed - Brave searches now wait their turn according to the rate limits Brave reports, so running several searches at once no longer overruns low-rate plans. If Brave says to slow down and the wait is short, the search retries once. If the wait is long, it fails right away and later searches respect the same wait. The 30-second search timeout includes this waiting. Thanks to [@saschaSpoonbill](https://github.com/saschaSpoonbill) for [PR #487](https://github.com/nicobailon/pi-web-access/pull/487). - OpenAI web search now works when you signed in to Pi with "Sign in with ChatGPT" on the `openai` provider. Searches go to OpenAI's official API, and `auto` search tries OpenAI first for this login, as it already did for `openai-codex`. With an `openai` API key, `auto` still tries Exa first. Thanks to [@kasumikira](https://github.com/kasumikira) for [issue #488](https://github.com/nicobailon/pi-web-access/issues/488). ## [0.34.0] - 2026-09-30 ### Highlights - Turning on web tools no longer costs a prompt-cache miss on models like DeepSeek. New sessions now pick the activation style that suits the model. - Search with Z.ai's GLM Coding Plan and use your plan quota instead of paying per call. - `fetch_content` now gets past Cloudflare "Just a moment..." pages by handing off to your other fetch providers. - Fetching works on networks where only your configured proxy can resolve hostnames. - `undici` is updated to pick up fixes for known security advisories. ### Added - Search with Z.ai's GLM Coding Plan web search when you select `provider: "zai"`. Set `ZAI_API_KEY` or `zaiApiKey` to your plan key, and set `zaiEndpoint: "china"` if your key comes from bigmodel.cn. Searches use your plan quota instead of per-call billing. Thanks to [@nailuoGG](https://github.com/nailuoGG) for [issue #475](https://github.com/nicobailon/pi-web-access/issues/475) and for live-testing the China endpoint in [issue #479](https://github.com/nicobailon/pi-web-access/issues/479). ### Changed - `toolActivation` now defaults to `"auto"`. On models that can take tools added mid-conversation, new sessions start with `web_enable` as before. On other models, such as DeepSeek, every enabled web tool is available from the first request, so turning them on no longer makes Pi resend the whole conversation and miss the prompt cache. Set `"dynamic"` to always start with `web_enable`, or `"eager"` to never use it. Resumed sessions keep the tools they already had. The README's Tool activation section explains the trade-off. Thanks to [@tinoy1336](https://github.com/tinoy1336) for [issue #484](https://github.com/nicobailon/pi-web-access/issues/484) and [@jordi9](https://github.com/jordi9) for [issue #481](https://github.com/nicobailon/pi-web-access/issues/481). - Keyed Exa searches with default options or `numResults: 5` now call Exa's `/search` endpoint instead of `/answer`, like every other keyed Exa search. Their answer text now comes from the search results instead of Exa's generated answer. If you route Exa through a custom `exaBaseUrl` gateway, it must support `/search`. Thanks to [@SuTang-vain](https://github.com/SuTang-vain) for [issue #470](https://github.com/nicobailon/pi-web-access/issues/470). - When `fetch_content` can't get a page, its list of fallback options now explains how to turn on the keyless Jina Reader fallback. It names only the setting you still need and keeps your current or default provider order. The hint doesn't appear if Jina already ran for that fetch, and it warns that Jina's servers fetch the target URLs. Thanks to [@SuTang-vain](https://github.com/SuTang-vain) for [PR #471](https://github.com/nicobailon/pi-web-access/pull/471). ### Fixed - `fetch_content` now treats a Cloudflare "Just a moment..." challenge page returned with HTTP 200 as a failed fetch, so your configured fallback providers can retrieve the real page. Detection needs Cloudflare's `cf-mitigated: challenge` header or its challenge-page scripts, so a page that only says "Just a moment..." is unaffected. Raw mode still returns the response unchanged, and authenticated fetches report the challenge without falling back to other providers. Thanks to [@SuTang-vain](https://github.com/SuTang-vain) for [issue #472](https://github.com/nicobailon/pi-web-access/issues/472). - With `ssrf.trustEnvProxy: true`, the `proxy` configured in `web-search.json` now resolves hostnames for web tools instead of your local DNS, so `fetch_content` works where only the proxy can resolve names. A per-call `proxy` with a different value is still checked against local DNS. When local resolution fails for a request going through the configured proxy, the error now names this setting. Thanks to [@ChenAuCarre](https://github.com/ChenAuCarre) for [issue #476](https://github.com/nicobailon/pi-web-access/issues/476). - Exa results without a title are now labeled with their site's hostname, such as `cdn.jsdelivr.net`, instead of `Source N`. Results whose URL has no hostname, such as `mailto:` or `file:` links, still use `Source N`. Thanks to [@SuTang-vain](https://github.com/SuTang-vain) for [PR #469](https://github.com/nicobailon/pi-web-access/pull/469). - `undici` is updated to 8.11.2. Earlier 8.x releases, including the 8.10.0 this package previously installed, have known security advisories such as decompression denial of service and a TLS certificate validation bypass. Thanks to [@setanta00](https://github.com/setanta00) for [issue #483](https://github.com/nicobailon/pi-web-access/issues/483). ## [0.33.0] - 2026-09-27 ### Highlights - Keep every web tool available from the first message with `"toolActivation": "eager"`, for models that rarely call `web_enable` on their own. - Search with You.com when you select it explicitly. - Give each Pi process its own fetched-page cache so parallel sessions stop evicting each other's pages. - Model allowlists in `enabledModels` or `--models` now work with every pattern Pi accepts. - Resumed older sessions keep the tools they had instead of gaining `web_enable`. ### Added - Set `"toolActivation": "eager"` in `web-search.json` to skip `web_enable` and keep every enabled web tool available from the first message. With the default `"dynamic"` setting, `web_enable` now tells the model to call it first whenever current, external, or linked information could help. Thanks to [@ackalker](https://github.com/ackalker) for [issue #458](https://github.com/nicobailon/pi-web-access/issues/458). - Set `PI_WEB_ACCESS_CACHE_ROOT` to a directory to give a Pi process its own fetched-page cache. The cache goes in a `web-search-cache` folder inside that directory, and `web-search.json` stays where it is. Parallel sessions then stop evicting each other's cached pages. Thanks to [@goodman-b](https://github.com/goodman-b) for [issue #457](https://github.com/nicobailon/pi-web-access/issues/457). - You.com is available as an optional search provider. Set `youApiKey` or `YDC_API_KEY`, then select `provider: "you"`, add it to a provider list, or list it in `searchRouting`. It also appears in the Curator. `auto` and `provider: "all"` never pick it. Thanks to [@mouse-value-add](https://github.com/mouse-value-add) for [PR #461](https://github.com/nicobailon/pi-web-access/pull/461). ### Fixed - Resuming a session that never had `web_enable`, for example one started before pi-web-access was installed or upgraded, no longer adds it partway through the conversation. The session keeps the tools it already had. Thanks to [@nilsoskar](https://github.com/nilsoskar) for [issue #462](https://github.com/nicobailon/pi-web-access/issues/462). - Answer mode, query rewrite, and summaries now follow the model list Pi builds from `enabledModels` or `--models`, so every pattern Pi accepts works here too, including patterns such as `@(!(openai-codex))/*` and partial names such as `sonnet`. Before, an allowlist made of those patterns blocked every model. Thanks to [@IdrisGit](https://github.com/IdrisGit) for [issue #463](https://github.com/nicobailon/pi-web-access/issues/463) and [PR #464](https://github.com/nicobailon/pi-web-access/pull/464). - `web_enable` now depends on the version of Pi you are running, not an older Pi package that may be installed next to the extension, so those setups no longer lose it. It needs Pi 0.86.0 or newer. On older Pi, every enabled web tool stays available from the start. Thanks to [@PhrZer](https://github.com/PhrZer) for [PR #456](https://github.com/nicobailon/pi-web-access/pull/456) and [@nguyenchiencong](https://github.com/nguyenchiencong) for [issue #444](https://github.com/nicobailon/pi-web-access/issues/444). ## [0.32.0] - 2026-09-26 ### Highlights - Spread Tavily searches across up to 20 API keys, with automatic failover when one runs out of quota. - Use Gemini Web browser cookies on KDE Linux, where Chrome keeps its password in KWallet. - Search with OpenAI through OpenCode and `opencode-go` without session errors or the wrong model. - See full search queries and URLs in tool-call labels. - Globally installed Pi no longer shows false version or duplicate TypeBox warnings. ### Added - Tavily can use a pool of up to 20 API keys. Set `TAVILY_API_KEY_1` through `TAVILY_API_KEY_20`, and when a key hits a quota or auth error the search moves on to the next one. `TAVILY_API_KEY_INDEX` picks the key to try first, and `TAVILY_API_KEY` / `tavilyApiKey` is tried last. Thanks to [@apoapostolov](https://github.com/apoapostolov) for [PR #430](https://github.com/nicobailon/pi-web-access/pull/430). ### Fixed - On KDE Linux, Chrome and Chromium cookies for Gemini Web now decrypt with the password stored in KWallet when Secret Service doesn't have it. Thanks to [@dianzuan](https://github.com/dianzuan) for [issue #436](https://github.com/nicobailon/pi-web-access/issues/436). - OpenAI search requests to `opencode.ai` now include OpenCode's session headers, fixing `400 MissingSessionID` errors. Other destinations never receive these headers. Thanks to [@juliocc](https://github.com/juliocc) for [PR #445](https://github.com/nicobailon/pi-web-access/pull/445). - When `openaiSearchModel` isn't set, OpenAI search through gateways such as `opencode-go` now picks the newest GPT model instead of a non-OpenAI model. - Globally installed Pi no longer shows a false "requires Pi 0.86.1 or newer" warning, and dynamic tool activation now works there. Thanks to [@samsimsom](https://github.com/samsimsom) for [issue #428](https://github.com/nicobailon/pi-web-access/issues/428). - TypeBox now comes from Pi instead of being installed separately, which avoids duplicate copies and Pi startup warnings. Thanks to [@ksreenivasan](https://github.com/ksreenivasan) for [issue #442](https://github.com/nicobailon/pi-web-access/issues/442) and [PR #443](https://github.com/nicobailon/pi-web-access/pull/443). - Tool-call labels show full search queries and fetch URLs when the terminal is wide enough, instead of cutting them off at 60 characters. Thanks to [@TheBestPessimist](https://github.com/TheBestPessimist) for [issue #440](https://github.com/nicobailon/pi-web-access/issues/440). ## [0.31.0] - 2026-09-22 ### Highlights - Start fresh sessions with a smaller web-tool list, then enable the tools when you need them. - Keep long search responses manageable without losing access to the full results. - Search with Baizhi MCP when you explicitly configure it. - Connect Brave, Exa, and Tavily to local HTTP services on loopback addresses. ### Added - Fresh sessions now show `web_enable` instead of the full web-tool list when supported. Call it to make your configured web tools available. Thanks to [@Knimoms](https://github.com/Knimoms) for [PR #424](https://github.com/nicobailon/pi-web-access/pull/424). - Added Baizhi MCP as an optional search provider, with search routing and Curator support. It is used only when explicitly selected. Thanks to [@ct-jaryn](https://github.com/ct-jaryn) for [PR #422](https://github.com/nicobailon/pi-web-access/pull/422). ### Changed - Long `web_search` responses now show a shorter result and the providers used. Full results remain available through `get_search_content`. Thanks to [@theSprog](https://github.com/theSprog) for [issue #423](https://github.com/nicobailon/pi-web-access/issues/423). ### Fixed - Allow explicitly configured Brave, Exa, and Tavily API base URLs to use HTTP on true loopback hosts while continuing to require HTTPS remotely. Thanks to [@aaschmid](https://github.com/aaschmid) for [issue #421](https://github.com/nicobailon/pi-web-access/issues/421). ## [0.30.0] - 2026-09-19 ### Highlights - Start installed copies of Pi Web Access much faster with a precompiled bundle. - Use standalone OpenAI search or reuse an existing Pi provider URL without duplicating gateway configuration. - Control which search providers and fetch modes are available. - Search Google through the new explicit Serply provider. - Get more reliable proxy handling, OpenCode requests, PDF answers, source checks, and stored-content retrieval. ### Added - Added opt-in `openaiUseProviderBaseUrl` to reuse a selected Pi provider's URL and credentials for OpenAI search. An explicit `openaiResponsesUrl` still takes precedence. - Added opt-in `openaiUseAlphaSearch` for standalone OpenAI search, with source limits, recency filtering, allowed-domain filtering, and normal provider fallback. Existing Responses search remains the default. Thanks to [@ZacharyQin](https://github.com/ZacharyQin) for [PR #420](https://github.com/nicobailon/pi-web-access/pull/420). - Added `fetch.defaultMode` and `fetch.allowedModes` for choosing the default `fetch_content` mode and disabling unwanted modes. Thanks to [@Slooz](https://github.com/Slooz) for #395. - Added `webSearch.allowedProviders` to restrict providers consistently across search, source checks, routing, aggregation, schemas, and Curator. Thanks to [@Slooz](https://github.com/Slooz) for #396. - Added an explicit-only Serply Google Search provider with domain filtering, recency filtering, routing, and Curator support. Configure it with `serplyApiKey` or `SERPLY_API_KEY`. Thanks to Serply vendor [@googio](https://github.com/googio) for PR #386. ### Changed - Published packages now load a precompiled bundle for faster startup, while source checkouts continue loading TypeScript directly. Thanks to [@Yisus423](https://github.com/Yisus423) for [issue #418](https://github.com/nicobailon/pi-web-access/issues/418) and [PR #419](https://github.com/nicobailon/pi-web-access/pull/419). - Fresh installs now use the silent `none` web search workflow by default. Explicit and configured Curator or summary workflows are unchanged. Thanks to [@ducaoya](https://github.com/ducaoya) for issue #416. ### Fixed - Preserve legacy `~/.pi/web-search.json` configuration when `~/.pi/agent/web-search.json` is absent in default environments without `XDG_CONFIG_HOME`. Thanks to [@fancyboi999](https://github.com/fancyboi999) for issue #411. - List Crawl4AI in the README provider summary and the package description, which both still omitted it after the provider shipped in 0.29.0. Thanks to [@bergheim](https://github.com/bergheim) for [PR #382](https://github.com/nicobailon/pi-web-access/pull/382). - Removed the unconditional global `fetch` replacement during extension initialization; proxy transport is now installed lazily when a proxied web-tool operation runs. Thanks to [@AdrianJ20](https://github.com/AdrianJ20) for [issue #388](https://github.com/nicobailon/pi-web-access/issues/388). - Send OpenCode session attribution headers when generating summaries, preventing configured `opencode` and `opencode-go` summary models from silently falling through to another candidate. Thanks to [@damozhang](https://github.com/damozhang) for issue #385. - Send the `x-opencode-session` / `x-opencode-client` attribution headers when `fetch_content` answer mode uses an `opencode` or `opencode-go` model. The answer path dispatches through the model registry, which bypasses the attribution headers Pi merges in the main agent loop, so those requests were rejected with `400 MissingSessionID`. Thanks to [@MrSerious0](https://github.com/MrSerious0) for PR #381. - Pass extracted PDF Markdown to `fetch_content` answer mode instead of the saved-file notice, while preserving readable-mode file output and stored-content retrieval. Thanks to [@MDGChamomile](https://github.com/MDGChamomile) for PR #390. - Prevent malformed `fetch_content` auth, mode, or proxy parameters from breaking tool-call rendering while preserving strict execution validation. Thanks to [@tekumara](https://github.com/tekumara) for issue #387. - Clarified that a negative `fetch_content` answer-mode result means the answer was not found in the extracted content, rather than asserting that it is absent from the whole page. Thanks to [@Slooz](https://github.com/Slooz) for issue #394. - Stop inferring claim support or contradiction from unrelated lexical markers in `source_check`; retrieved passages now require manual semantic review. Thanks to [@wayenchan](https://github.com/wayenchan) for issue #383. - Return bounded `get_search_content` excerpts instead of dropping oversized merged match ranges, using compact query IDs to reserve representative ranges when discovered spans fit the output budget. Thanks to [@MDGChamomile](https://github.com/MDGChamomile) for PR #391. ## [0.29.0] - 2026-09-10 ### Highlights - Search Google through SerpApi with domain and recency filters. - Use a self-hosted Crawl4AI instance as an extraction fallback. - Connect through SOCKS proxies and resolve credentials with 1Password service accounts. - Start Pi faster through on-demand loading of extraction and AI features. - Get more reliable provider fallback, credential routing, caching, and config discovery. ### Added - Added a Curator action that approves the current summary and uses auto-summary for later default-workflow searches in the same prompt run. The choice is not saved between prompts. Thanks to [@thomak-dev](https://github.com/thomak-dev) for issue #376. - Added a self-hosted Crawl4AI extraction fallback for `fetch_content`, configured with `crawl4aiBaseUrl` / `CRAWL4AI_BASE_URL` and `crawl4aiApiToken` / `CRAWL4AI_API_TOKEN`. It runs after Firecrawl and before hosted providers, and can be selected with `crawl4ai` in `fetchRouting.providers`. Thanks to [@bergheim](https://github.com/bergheim) for [PR #375](https://github.com/nicobailon/pi-web-access/pull/375). - Added an explicit SerpApi Google Search provider with `serpapiApiKey` / `SERPAPI_KEY`, domain and recency filters, search routing, and Curator support. Thanks to [@tanysheng](https://github.com/tanysheng) for PR #363. - Added SOCKS4, SOCKS4A, SOCKS5, and SOCKS5H proxy support in config and per-call overrides. Thanks to [@phillipzink](https://github.com/phillipzink) for PR #365. - Added 1Password service-account support for credential resolver commands through `OP_SERVICE_ACCOUNT_TOKEN`. Thanks to PR author [@Avg8888](https://github.com/Avg8888) and commit author [@xapids](https://github.com/xapids) for [PR #364](https://github.com/nicobailon/pi-web-access/pull/364). ### Changed - Load extraction and AI features only when first used, reducing extension startup time. Thanks to [@ducaoya](https://github.com/ducaoya) for PR #366. ### Fixed - Treat Tavily monthly plan exhaustion as a quota error so search routing can fall back to another provider. Thanks to [@simbel](https://github.com/simbel) for issue #378. - Refuse to send Pi-resolved OpenAI credentials to the official Responses endpoint when they belong to a provider with a custom `baseUrl`. Configure `openaiResponsesUrl` explicitly to search through a gateway. Thanks to [@projectkite](https://github.com/projectkite) for issue #367. - Remove expired fetched content from memory when pruning the cache, while preserving the retrieval window and session history. Thanks to [@MDGChamomile](https://github.com/MDGChamomile) for issue #362. - Use Pi's agent directory (`~/.pi/agent/web-search.json`) for the default web-search config instead of falling back to the legacy `~/.pi/web-search.json`. Existing `PI_CODING_AGENT_DIR` and XDG behavior is unchanged. Thanks to [@lJoublanc](https://github.com/lJoublanc) for issue #360. ## [0.28.0] - 2026-09-04 ### Highlights - Search X posts through xAI or choose Mistral for web searches. - Batch searches run in parallel for faster multi-query research. - Follow-up retrieval can access stored search results, and Perplexity keeps all cited sources. - Web-tool proxy settings no longer affect unrelated Pi requests. ### Added - Documented the Linux `xdg-utils` dependency for automatic curator browser launch and the manual URL fallback. Thanks to [@wickedTangent](https://github.com/wickedTangent) for issue #336 and PR #337. - Added opt-in X post search through xAI alongside web search, configured with `xaiSearchTools`. Thanks to [@Jerry2003sky](https://github.com/Jerry2003sky) for issue #342. - Added Mistral web search with `provider: "mistral"`, using `mistralApiKey` / `MISTRAL_API_KEY` credentials. Premium web search is available as a separate opt-in. Thanks to [@jaudiger](https://github.com/jaudiger) for issue #346. ### Changed - Run batch searches in parallel with a concurrency limit, keeping results in query order and preserving provider fallback. Thanks to [@Noir-Lime](https://github.com/Noir-Lime) for PR #345. ### Fixed - Included `responseId` in search output so the model can retrieve stored results with `get_search_content`. Thanks to [@axelbaumlisto](https://github.com/axelbaumlisto) for PR #354. - Fixed flaky tests on slower Node startups. Thanks to [@axelbaumlisto](https://github.com/axelbaumlisto) for PR #353. - Kept all sources cited in Perplexity answers, even beyond `numResults`. Answers without citations still respect the result limit. Thanks to [@schlessera](https://github.com/schlessera) for issue #340 and PR #341. - Limited configured proxies to web-tool requests so unrelated Pi requests are unaffected. Thanks to [@alexei-ciobanu](https://github.com/alexei-ciobanu) for PR #339 and [@mystery4f](https://github.com/mystery4f) for PR #343. - Cleaned up abandoned GitHub clone directories after crashes, only when their owning process is confirmed dead. Thanks to [@yazanabuashour](https://github.com/yazanabuashour) for issue #331. - Kept using an existing `~/.pi/web-search.json` when `XDG_CONFIG_HOME` is set but no config file exists there. Thanks to [@hu3rror](https://github.com/hu3rror) for issue #333. ## [0.27.0] - 2026-08-28 ### Highlights - `fetch_content` now lets you tune direct HTTP and Jina Reader timeouts from config. - Answer mode can use a configured default model while still allowing per-call overrides. - HTML fallback parsing is quieter for pages with relative canonical links. - GitHub repository fetching is safer when more than one Pi process is running. ### Added - Added opt-in `fetch.timeout` configuration in seconds for the direct HTTP and Jina Reader `fetch_content` paths, with per-call timeout overrides taking precedence. Thanks to [@linuxtextadventurer](https://github.com/linuxtextadventurer) for PR #327. - Added opt-in `fetch.answerProvider` and `fetch.answerModel` defaults for `fetch_content` answer mode, with per-call `answerModel` overrides taking precedence. Thanks to [@linuxtextadventurer](https://github.com/linuxtextadventurer) for PR #328. ### Fixed - Set the Defuddle fallback document URL before parsing pages with relative canonical links, preventing `ERR_INVALID_URL` warnings. Thanks to [@bin115885](https://github.com/bin115885) for issue #322. - Isolated GitHub clone workdirs per extension runtime so cleanup in one process cannot delete another process's clone. Thanks to [@MDGChamomile](https://github.com/MDGChamomile) for PR #323. ## [0.26.0] - 2026-08-28 ### Highlights - `web_search` now includes XCrawl as an explicit search provider. - XCrawl results now produce safer clickable links, including relative redirect links from the API. - Local models can send multiple web-search queries as a JSON string and still get separate searches. - HTML fallback extraction is quieter and more accurate when Defuddle cannot process a page. ### Added - Added XCrawl as an explicit-only search provider (`provider: "xcrawl"`) with `xcrawlApiKey` / `XCRAWL_API_KEY` credentials, client-side domain filtering, fallback titles, and retriable provider-timeout errors. Thanks to [@zeroicey](https://github.com/zeroicey) for #312 and #313. ### Changed - Refined the XCrawl provider docs, parser shape, availability metadata, and focused tests. ### Fixed - Expanded JSON-array strings supplied in the singular `web_search` `query` field into independent searches, with a clear no-query error for empty arrays. Thanks to [@alex-rs](https://github.com/alex-rs) for #317 and [@zeroicey](https://github.com/zeroicey) for PR #319. - Kept Defuddle selector-processing failures out of the Pi console and stopped treating the raw page body as a successful extraction. Thanks to [@riabiy](https://github.com/riabiy) for #315. - Resolved API-origin-relative XCrawl result links to absolute URLs and rejected blank result links instead of emitting fabricated source URLs. Thanks to [@zeroicey](https://github.com/zeroicey) for #318. ## [0.25.0] - 2026-08-25 ### Highlights - `web_search`, `source_check`, and `fetch_content` can now use an explicit HTTP(S) proxy for restricted networks. - `fetch_content` now gives useful GitHub PR and issue summaries, including comments, review threads, anchors, and truncation markers. - Gemini users can choose browser cookie profiles more predictably and can use Google Application Default Credentials for Gemini generate-content calls. - Kimi Code Plan users can run explicit Kimi web searches through Pi's `kimi-coding` login. - HTML extraction, missing-page guidance, and stored-content lookup are more reliable. ### Added - Added an optional `proxy` parameter to `web_search`, `source_check`, and `fetch_content`. When set, search APIs, page fetches, and content extraction use `curl` through that proxy. Localhost and `NO_PROXY` hosts still bypass the proxy. A default proxy can also be set with `"proxy"` in `~/.pi/web-search.json`. Thanks to [@mystery4f](https://github.com/mystery4f) for PR #307. - Added deterministic Chromium cookie selection with `browserCookies.browser` and `browserCookies.profile`; arbitrary profile paths remain unsupported. Thanks to [@lmilojevicc](https://github.com/lmilojevicc) for issue #297. - Added GitHub PR and issue specialization in `fetch_content`, with `gh`-first metadata, bounded REST fallback, comment anchors, review threads, truncation markers, and the `githubPrIssue.enabled` opt-out (#294). - Added explicit-only Kimi Code Plan search using Pi's refreshed `kimi-coding` OAuth credentials. Thanks to [@lushangkan](https://github.com/lushangkan) for PR #275. - Added opt-in `searchRouting.useCurrentModel` routing for automatic searches. Official OpenAI GPT Responses models can now fund Hosted Search with their current endpoint, credentials, and headers before configured fallbacks. Thanks to [@nyankosama](https://github.com/nyankosama) for PR #293. - Added strict Hosted Search response validation and better fallback classification for unsupported Hosted Search tools. - Added current-model Hosted Search routing for official `openai-codex` GPT Responses models through the Codex Responses endpoint. - Added Google Application Default Credentials support for Gemini generate-content calls with `geminiAuth: "adc"`. This covers Gemini search, URL context, PDF, and inline-data extraction through Vertex AI. YouTube and local video analysis still require `GEMINI_API_KEY` because they use the Gemini Files API. Thanks to [@smazurov](https://github.com/smazurov) for PR #301. ### Fixed - Added Defuddle as a local fallback when Readability and RSC extraction cannot recover useful HTML content. Thanks to [@tobru](https://github.com/tobru) for issue #300. - Kept Gemini ADC config and credential errors, GitHub PR/issue REST fallback failures, and Defuddle fallback failures visible instead of silently downgrading them. - Pointed definitive `fetch_content` 404/410 failures to search guidance instead of provider configuration. Thanks to [@Daniishkhan](https://github.com/Daniishkhan) for PR #308. - Improved Gemini Web browser-cookie diagnostics so `/google-account` shows sanitized attempted browser/profile entries and distinguishes missing required cookies, password-store access, and decryption failures. Thanks to [@lmilojevicc](https://github.com/lmilojevicc) for issue #296. - Made `get_search_content` tolerate bridge defaults when `findText` is supplied, while preserving ordinary pagination. Thanks to [@ZacharyQin](https://github.com/ZacharyQin) for PR #295. ## [0.24.2] - 2026-08-22 ### Highlights - `auto` search now uses Codex-backed OpenAI search when Pi is running a Codex model. - Non-Codex sessions still prefer Exa first, so zero-config search keeps its fast keyless path. - Gemini Web browser-cookie access is more reliable on Windows Chrome and Edge profiles. ### Changed - Prefer Codex-backed OpenAI search in `auto` mode when the active Pi model is `openai-codex`; otherwise prefer Exa before OpenAI. ### Fixed - Fixed Windows Gemini Web browser-cookie extraction when Chromium cookie expiry values are too large for JavaScript numbers or PowerShell DPAPI key unprotect needs encoded command transport. Thanks to [@laixuanthoi](https://github.com/laixuanthoi) for issue #290. ## [0.24.1] - 2026-08-21 ### Added - Added `pdf.maxPages` to limit Datalab, Gemini, and local PDF extraction to the first N pages. Thanks to [@jaudiger](https://github.com/jaudiger) for issue #277. - Added optional `openaiSearchProviders` config to choose which Pi model providers fund OpenAI `web_search`, in priority order. Thanks to [@hank-warren](https://github.com/hank-warren) for PR #276. - Added Windows Chrome and Edge browser-cookie support for Gemini Web. Thanks to [@laixuanthoi](https://github.com/laixuanthoi) for issue #286. ### Fixed - Hardened GitHub clone cache path handling. Thanks to [@spikelab](https://github.com/spikelab) for the responsible disclosure. - Updated the local `fetch_content` HTTP User-Agent for wider compatible content retrieval. Thanks to [@_can1357](https://x.com/_can1357) for [the User-Agent observation](https://x.com/_can1357/status/2090837707069014224). - Replaced inline RFC 2397 `data:` URIs in extracted page content with explicit bounded omission markers (MIME type, encoding, encoded/decoded byte counts, SHA-256 digest, `retrieval=not-retained`) before content reaches tool results, the fetch cache, or session persistence. Readable prose and Markdown image alt text are preserved; typed thumbnail/frame image blocks are unaffected. Thanks to [@bbbRye007](https://github.com/bbbRye007) for #281 and #282. - Detached Linux curator browser launches so `xdg-open` cannot block `web_search` until the browser exits. Thanks to [@nguyenphivn](https://github.com/nguyenphivn) for issue #279. - Allowed configured Firecrawl API base URLs to use loopback addresses without opening loopback for submitted fetch targets. Thanks to [@ackalker](https://github.com/ackalker) for issue #280. ## [0.24.0] - 2026-08-18 ### Highlights - Added more provider choices, including Parallel Search MCP, Valyu, and Serper. - Made `fetch_content` recover useful article text from more Next.js and React Server Components pages. - Updated summary and query rewrite model defaults to newer fast models. - Made nested summary and rewrite model calls follow Pi's model registry routing. - Added more control over summary thinking levels and provider API base URLs. ### Added - Added `summaryModel` thinking-level suffix support. Thanks to [@pkos98](https://github.com/pkos98) for issue #264. - Added explicit-only Parallel Search MCP support for keyless search and opt-in hosted fetch. Thanks to [@happytomatoe](https://github.com/happytomatoe) for #257. - Added explicit-only Valyu and Serper search providers. Thanks to [@mikhel01](https://github.com/mikhel01) for issues #259 and #260. - Added configurable API base URLs for Brave, keyed Exa, and Tavily requests, with credential stripping across redirect origins. Thanks to [@XWIlluDelu](https://github.com/XWIlluDelu) for #265. ### Changed - Refreshed default summary and query rewrite model preferences. Thanks to [@hyein-cbio](https://github.com/hyein-cbio) for #266. ### Fixed - Route nested model calls through Pi's model registry. Thanks to [@rany2](https://github.com/rany2) for #263. - Recover useful Next.js RSC content when Readability only extracts a loading shell, and report full, partial, or failed background content fetches accurately (#272, #273). - Keep valid AnySearch results when the API omits or nulls result content. Thanks to [@mikhel01](https://github.com/mikhel01) for #258. ## [0.23.0] - 2026-08-15 ### Added - Added opt-in `authFetch` profiles for local browser-cookie `fetch_content` requests. Thanks Luka (`@lmilojevicc`) for issue #254. - Added Firecrawl as a `web_search` provider using the existing Firecrawl configuration. Thanks Andrés Sanabria (`@andy-spike`) for issue #246. - Added macOS Brave profile support for opt-in Gemini Web browser-cookie access. Thanks Rajyavardhan Singh (`@imrajyavardhan12`) for PR #248. ### Changed - Made local-video detection use an explicit result state in extraction code. ### Fixed - Preserve summary draft completion routing through the model registry. Thanks `@limwa` for PR #249. - Constrain numeric tool parameters to supported integer ranges. Thanks `@jaudiger` for issue #250 and PR #251. ## [0.22.0] - 2026-08-11 ### Added - Added Bocha web search provider support. Thanks @jingyulong for PR #243. - Added `maxInlineContentChars` to configure the direct content and stored-content slice limit, with a 200,000-character maximum. Thanks @be4zad for issue #244. ### Fixed - Hardened the fetched-content cache against symlink traversal and unsafe permissions, and bounded it to 128 entries and 128 MiB with oldest-entry eviction. Thanks `@HerbertGao` for issue #240 and PR #241. ## [0.21.0] - 2026-08-10 ### Added - Added per-tool and per-command registration gates plus image and PDF extraction gates. Thanks @jaudiger for issue #234. - Added `summaryGenerationDeadlineMs` to configure the summary model deadline for curator and auto-summary workflows. Thanks @cataldoc for issue #237. ### Fixed - Store full fetched content in an external cache instead of embedding it in session JSONL entries, preventing large search-heavy sessions from ballooning on restore. Thanks Igor Samokhovets (`@samohovets`) for issue #236. - Document `get_search_content` parameter constraints in the tool schema. Thanks `@iwangjie` for PR #233. ## [0.20.0] - 2026-08-10 ### Added - Added keyless DuckDuckGo HTML search as an explicit and routing provider. Thanks @lmilojevicc for issue #228. - Added Datalab hosted PDF-to-Markdown extraction as an optional PDF provider. Thanks José Antonio Galiano Sandoval (`@jagaliano`) for PR #226. ### Changed - Tightened Datalab JSON response validation and Gemini Web fetch initialization internals. ### Fixed - Fix Gemini Web "fetch failed" (`UND_ERR_HEADERS_OVERFLOW`) when running inside a host agent whose global undici dispatcher uses HTTP/1.1 with the default 16 KiB `maxHeaderSize`: Google's `/app` page exceeds that budget. Gemini Web requests now use a dedicated undici agent with a 4 MiB header budget. Thanks José Antonio Galiano Sandoval (`@jagaliano`) for PR #230. - Preserve collapsed `web_search` result background padding. Thanks `@SheffeyG` for PR #224. ## [0.19.0] - 2026-08-08 ### Added - Added Jina Search as a normal configured `web_search` provider with `jinaApiKey` / `JINA_API_KEY`, explicit/auto/routing/all-provider support, domain and recency constraints, optional inline page content, and Curator integration. Thanks Orbio Agent (`@Gabrielgvl`) for PR #214. ### Fixed - Added a tracked npm lockfile for reproducible contributor installs, and updated model-registry auth header typing for current Pi peer packages. Thanks `@dougEfresh` for PR #218. - Keep manual `/websearch` curator pages in sync if the live SSE stream disconnects or misses events while searches are still running. Thanks `@Whisperfall` for issue #215. - Updated Kagi Search and Extract requests for the current v1 API contracts. Thanks `@mattgaff` for PR #217. - Recognize `:max` thinking suffixes when matching scoped summary models. Thanks `@justin8ty` for PR #219. - Route local video Gemini upload, polling, and deletion requests through configured `geminiBaseUrl` / `GOOGLE_GEMINI_BASE_URL` relays. Thanks Mr. (`@Liemo99`) for PR #213. - Classify xAI `403` spending-limit and quota-exhaustion responses as quota errors so configured search routing can fall back, while preserving ordinary `403` responses as authentication errors. Thanks `@0xmarcinz` for PR #212. ## [0.18.0] - 2026-08-03 ### Added - `fetch_content` can now return the original text response with `mode: "raw"`, which is useful for JSON APIs, error pages, and debugging what a server actually sent. - `fetch_content` can now answer a question about a single fetched page with `mode: "answer"`, while still saving the original page text so you can inspect it later. - Direct image links now work for PNG, JPEG, WebP, and GIF files. The tool downloads them safely, resizes large images, and returns an inline image result. - `get_search_content` now accepts `findText` and `findMode`, so you can search saved content for the passage you need instead of paging through a long page by hand. Inspired by `@xl0`'s `pi-lovely-web` project. - Added optional `searxngHeaders` so self-hosted SearXNG requests can carry reverse-proxy or Zero Trust auth headers. Thanks `@preinpost` for PR #202. - Added explicit-only xAI/Grok search with `xaiApiKey` / `XAI_API_KEY`, Pi model-registry auth, and optional `xaiSearchModel`. Thanks `@join3r` for PR #196. - Added explicit-only Bright Data SERP search with `brightdataApiKey` / `BRIGHTDATA_API_KEY` credentials and a required `brightdataSerpZone` / `BRIGHTDATA_SERP_ZONE` zone of Bright Data type `serp`. Bright Data is never chosen by `auto` and never participates in `provider: "all"`. Thanks `@mo-root` for PR #198. - Added Bright Data Web Unlocker as a paid `fetch_content` extraction fallback with `brightdataApiKey` / `BRIGHTDATA_API_KEY` credentials and a required `brightdataUnlockerZone` / `BRIGHTDATA_UNLOCKER_ZONE` zone of Bright Data type `unblocker`. Thanks `@mo-root` for PR #199. - Added Kagi Search API support with `kagiApiKey` / `KAGI_API_KEY`, plus Kagi Extract as a `fetch_content` fallback. Thanks `@imlonghao` for issue #197. - Added Ollama Cloud Web Search support with `ollamaApiKey` / `OLLAMA_API_KEY`, plus Ollama Web Fetch as a `fetch_content` fallback. Thanks `@bradley-holt` for issue #203. - Added explicit-only SerpBase Google SERP search with `serpbaseApiKey` / `SERPBASE_API_KEY`, domain filters as `site:` clauses, and recency mapped to Google `tbs`. Thanks `@gefsikatsinelou` for issue #195. ### Changed - Long `fetch_content` results are easier to continue reading. The first response now stops on cleaner line boundaries and tells you the character, byte, and line totals plus the exact offset to request next. ### Fixed - Fixed the `fetch_content` call header showing `fetch (no URL)` when Pi supplied `url` together with an empty `urls` array. Thanks `@Vergil824` for issue #192. - Resolve preferred summary and query-rewrite models through routed provider registrations such as OpenRouter, preserving the registered provider and model ID instead of falling back when the native provider is unavailable. Thanks `@robzolkos` for issue #200 and PR #201. - GitHub clone subprocesses now disable interactive credential prompts and terminate their process trees on timeout or cancellation, preventing orphaned Git helpers from capturing terminal input. Thanks `@MDGChamomile` for PR #193. ## [0.17.1] - 2026-07-31 ### Fixed - Removed the unsupported JSON Schema `uniqueItems` keyword from provider-array tool schemas so Gemini-compatible tool validators can register pi-web-access tools. Thanks `@akmaldira` for PR #191. ## [0.17.0] - 2026-07-30 ### Added - Added Gemini API PDF-to-Markdown conversion before local `unpdf` fallback, with inline PDF upload, page-marker validation, configurable `pdf.maxSizeMB`, and streamed size enforcement. Thanks José Antonio Galiano Sandoval (`@jagaliano`) for PR #180. - Added Searchinfinity (Byteplus Searchinfinity / 豆包搜索 Global edition) as a search provider with `searchinfinityApiKey` / `SEARCHINFINITY_API_KEY` credentials, native domain and recency filters, model-generated result summaries, HTTP-semantics mapping for business error codes, provider-array/all-provider routing, and curator selection. Thanks `@cyzlmh` for PR #186. - Added Querit as a search and hosted content provider with `queritApiKey` / `QUERIT_API_KEY` credentials, native domain and recency filters, optional inline Contents retrieval, provider-array/all-provider routing, curator selection, and a `fetch_content` fallback. Thanks `@MCapricorns` for PR #185. ### Changed - The 5MB `fetch_content` response size limit is now enforced while streaming the decoded body, not just via the `Content-Length` header. Chunked or compressed pages whose decoded content exceeds 5MB now fail with `Response too large (5MB)` instead of being buffered unbounded and parsed. Shared config parsing for SSRF and fetch-content domain policy loaders also removes duplicate reads on the fetch hot path. ### Fixed - Fixed filtered zero-config Exa search silently losing its request options: the default keyless MCP tool (`web_search_exa`) only accepts `query` and `numResults`, so `type`, `livecrawl`, and `contextMaxCharacters` were dropped server-side and `includeContent` never returned page text. Filtered or content-carrying keyless searches now use `web_search_advanced_exa` — served by the same keyless free tier — which applies `includeDomains`/`excludeDomains`, `startPublishedDate`, highlights, and text limits as real parameters instead of `site:` / "past week" strings appended to the semantic query. Plain searches stay on `web_search_exa`, and the advanced path falls back to it if it is unavailable. - Stopped requesting citation text on keyed Exa `/answer` calls, which was fetched and then discarded, and stopped requesting page text on keyed `/search` calls that do not ask for content. - Exa MCP rate-limit responses (429) now explain that adding `exaApiKey` to `~/.pi/web-search.json` removes the free-tier limit, instead of surfacing a bare status code. Thanks `@kesku` for PR #187. ## [0.16.0] - 2026-07-30 ### Added - Added Search1API as a first-class search and extraction provider with `search1apiApiKey` / `SEARCH1API_KEY` credentials, native domain and recency filters, opt-in Deep Search inline content, provider-array/all-provider routing, curator selection, and a hosted `fetch_content` fallback through the Crawl API. Thanks `@fatwang2` for PR #176. ### Fixed - Standardized Gemini API defaults on `gemini-3.6-flash` for search, URL context, YouTube, and local video paths, set Gemini Web’s separate browser-cookie default to `gemini-3.1-pro`, and stopped unsupported Web models from silently falling back to 2.5 Flash. Thanks `@jagaliano` for issue #181. - Accept explicit provider arrays for `web_search` and `source_check`, running only the selected providers concurrently while preserving `auto`, `all`, and sequential routing behavior. Thanks `@XWIlluDelu` for PR #179. - Added `curatorRemote` and `autoOpenBrowser` controls for remote-accessible curator sessions, defaulting remote mode to a printed manual URL unless browser auto-open is explicitly requested. Thanks `@tylerdavis` for PR #178. - Resolve the OpenAI search model from Pi’s model registry, with an `openaiSearchModel` override and preserved API-key fallback for partial registries. Thanks `@ahalekelly` for PR #182. - Surfaced RFC Link / HTML discovery relations (`service-desc`, `service-doc`, `service-meta`, `api-catalog`, `describedby`) from HTTP `Link` headers and matching `link`/`a[rel]` markup during `fetch_content` HTML extraction, including empty SPA shells, without broad `/docs` URL heuristics. Thanks `@XWIlluDelu` for PR #175. - Expand leading `~` and `$HOME`-style environment variables in `githubClone.clonePath` before cloning repositories. Thanks `@unship` for PR #184. - Write extracted PDF markdown to a temporary `pi-web-pdf` directory by default instead of `~/Downloads`, while preserving explicit output directories. Thanks `@ahalekelly` for PR #183. ## [0.15.0] - 2026-07-28 ### Added - Added `openaiResponsesUrl` for routing OpenAI `web_search` and `source_check` calls through third-party Responses-compatible gateways while keeping the official OpenAI endpoint as the default. Thanks The Loki (`@the-loki`) for issue #174. - Added `provider: "all"` to search every available provider except AnySearch simultaneously, render one independently selectable Curator card per provider before the final summary, deduplicate sources and inline content, and preserve partial successes with per-provider diagnostics. Thanks José Antonio Galiano Sandoval (`@jagaliano`) for PR #173. - Added TinyFish as a first-class search and extraction provider with `tinyfishApiKey` / `TINYFISH_API_KEY` credentials, domain and recency filters, paginated result counts, batched inline content, ordered routing, curator selection, and a hosted `fetch_content` fallback before Parallel. Thanks José Antonio Galiano Sandoval (`@jagaliano`) for PR #172. ## [0.14.0] - 2026-07-25 ### Added - Added optional `fetchContent.domainPolicy` hostname allow/deny rules for `fetch_content`, checked before target requests and redirects while preserving SSRF protection and local-source behavior. Thanks Joseph Maliksi (@jmaliksi) for #79. - Added explicit-only AnySearch direct search provider with anonymous access, optional `anysearchApiKey` / `ANYSEARCH_API_KEY` credentials, strict response validation, and request-time credential sources. Thanks Robin (@choronz) for #130. - Added SERPdive search provider with request-time credential sources, a free-tier `krill` default that never spends on install, opt-in `mako`/`moby` retrieval depth, locally applied domain filters, and recency documented as a query hint rather than a guaranteed freshness filter. Thanks Eden d'Alexis (@edendalexis) for #139. - Added opt-in ordered search routing through `searchRouting.providers` with explicit transient, quota, and network fallback kinds; named providers remain strict, single-provider config retains precedence, and exhausted routes preserve per-provider diagnostics. Thanks @smithyyang for #77. - Added optional self-hosted SearXNG search with SSRF-guarded base URLs, local-first auto selection, result filters, and documented `ssrf.allowRanges` opt-ins. Thanks to Marcos A. Núñez (@marnunez) for PR #107 and Avinash Kanaujiya (@avinashkanaujiya) for issue #105. - Added optional self-hosted Firecrawl extraction for `fetch_content`, using `/v2/scrape` by default (or `/v1/scrape` for older images), request-time credential sources, cache-only requests by default, cross-origin redirect credential stripping, and local-first fallback behavior when configured. Thanks Florian Kinder (@fank) for PR #123 and Avinash Kanaujiya (@avinashkanaujiya) for issue #105. - Added opt-in configurable public tool names for environments where another extension or model reserves the defaults, while keeping `web_search`, `source_check`, `fetch_content`, and `get_search_content` unchanged by default. Thanks Kaiqiang (@youkq95) for reporting #138. - Added `SECURITY.md` guidance for private vulnerability reporting. Thanks Aurelio Ribeiro (@aurelio-ribeiro) for #128. - Added request-time `$ENV_VAR` and `!command` credential sources for every provider API-key field, including escaped `$$` and `$!` literals, while preserving legacy environment precedence for literal config values. Thanks to Eugene Strizhok (@estrizhok) for #159. - Added request-time `$ENV_VAR` and `!command` credential sources for Exa and Gemini API configuration, with bounded output, redacted failures, strict source precedence, and shell-local 1Password session forwarding limited to the resolver command. Thanks to Ezra Miller (@ezmill) for #137. - Added `source_check` machine-readable research artifacts with passage-level provenance, bounded page fetching, durable session retrieval, and conservative claim assessments. Thanks Clark Everson (@gr3enarr0w) for PR #111 and issue #108. ### Changed - Limited the published package contents to runtime files, docs, and declared assets, keeping internal tests out of the npm tarball. - Deferred the heavy content extraction module until the first `fetch_content` or `includeContent` search call, reducing extension startup work. Thanks Kaushik Gopal (@kaushikgopal) for PR #125. - Send direct Gemini API credentials in `x-goog-api-key` headers for generate, upload, status, and delete requests instead of URL query parameters. ### Removed - Removed the bundled `librarian` skill from this package instead of keeping a second research workflow coupled to GitHub clone internals. Thanks mcwalrus (@mcwalrus) for #136 and gravewhisper (@gravewhisper) for #23. ### Fixed - Prevented opt-in Gemini Web browser cookies from crossing origins on redirects, disabled automatic redirects for cookie-bearing generation and upload requests, and restored local file reads for Gemini Web uploads. - Preserved caller cancellation in non-curated multi-query searches instead of continuing with later queries, and passed the active extension context into curator-added searches. - Hardened new provider result boundaries and removed a dead auto-routing condition without changing provider behavior. - Hardened unreleased config/result boundaries by rejecting malformed config roots, reporting malformed SSRF JSON, normalizing invalid Perplexity result counts, and rejecting contradictory curator summary metadata. - Reworked opt-in Gemini Web browser-cookie extraction to scan non-default Chromium profiles, preflight required cookie names before Keychain/secret-tool access, and cache encryption passwords only in-process. Thanks Kevin Truong (@kevinQTruong) for the originating PR #14, Jessica Black (@jssblck) for reporting #9, János Veres (@jveres) for reporting #2, and RimuruW (@RimuruW) for reporting #15. - Added read-only `sqlite3` CLI and Python stdlib fallbacks when `node:sqlite` is unavailable, with sanitized actionable diagnostics instead of silently reporting Gemini Web as unavailable. - Bounded curator summary-model generation and returned a deterministic, phase-labeled fallback when a draft exceeds its deadline, while preserving caller cancellation and model-resolution errors. Thanks torn1147 (@torn1147) for reporting #43. - Used `gemini-2.5-flash` as the Gemini API grounded-search default while preserving configured `searchModel` values and the shared URL/video defaults. Thanks lajarre for reporting #11. - Used the configured search provider when tool calls omit `provider` or emit the schema's `auto` default, while preserving explicit provider overrides and auto fallback without configuration. Thanks Pavlo (@fxposter) for reporting #17. - Clarified that `githubClone.enabled: false` disables GitHub clone/API specialization while leaving generic URL extraction available. Thanks Carlos Peralta (@cperalt) for issue #26. - Added an opt-in `ssrf.trustEnvProxy` setting that skips local DNS preflight only for hostnames routed through configured HTTP(S) proxy environment variables, while retaining localhost, literal private-IP, and `NO_PROXY` protections. This addresses the DNS failure described in #104 without claiming full proxy transport support. Thanks Daniel Birn (@DBPhoenix) for PR #109 and mystery4f for reporting #104. - Kept PDF extraction compatible with runtimes without native `Promise.try` and limited PDF.js output to errors. Thanks Guido Witt-Dörring (@guwidoe) for PRs #33 and #34. - Returned fetched URL content from `get_search_content` in bounded slices with explicit `offset`/`limit` continuation metadata, preventing large stored pages from overflowing the next model request. Thanks @manfredlift for reporting #112. - Kept oversized local videos recognizable for ffmpeg frame/timestamp extraction while preserving the Gemini analysis upload size limit. Thanks @Xandaar for reporting #135. - Declared `typebox` as a regular runtime dependency so clean/global installs can load the extension without relying on Pi or Feynman to provide that peer. Thanks @DuskyElf, @tonytziorvas, and @53able for reporting #59, #63, and #66. - Preserved OpenAI/Codex answers even when the provider returns no source citations instead of replacing them with `No results found`. Thanks AstroHan (@Astro-Han) for reporting #117. - Prevented browser auto-open failures from crashing the curator fallback path with `ReferenceError: sendCuratorFallbackUpdate is not defined`. Thanks Nisarg Patel (@pnisarg) for PR #121; Apostol Apostolov (@apoapostolov), @2seoik, and Demetre Dzmanashvili (@demetere) for related PRs #133, #120, and #114; and @ruanlinxin, @bluewatercg, @tonydiep, and @pinion05 for reports #103, #116, #126, and #127. - Added a focused SSRF error hint for TUN/fake-IP `198.18.0.0/15` addresses that points users to the existing `ssrf.allowRanges` opt-in. Thanks Aaron (@BenjaminAaron196) for PR #141 and AstroHan (@Astro-Han) for reporting #134. - Registered the web activity widget with Pi's supported string-array API to prevent `content is not a function` crashes when toggling non-default shortcuts. Thanks @llllllllqq for reporting #158. - Registered the web activity widget as a Pi component factory instead of passing a `Text` instance directly. Thanks Trey Hoover (@treyhoover) for PR #132. ## [0.13.0] - 2026-06-25 ### Added - Added `ssrf.allowRanges` config to exempt CIDR ranges (e.g. `198.18.0.0/15`) from the SSRF guard, so `fetch_content`/`web_search` work on hosts whose network proxy runs in TUN + fake-IP mode (Surge, Clash, Mihomo, Stash, ...) where public domains resolve into a synthetic reserved range. Off by default. Thanks @TianZuo555 for reporting #101 and PR #102. ### Fixed - Hardened `ssrf.allowRanges` validation to reject all-address `/0` CIDRs and non-string entries. ## [0.12.0] - 2026-06-24 ### Added - Added `webSearch.enabled` config to disable `web_search` tool registration. Thanks @webwarrior-ws for PR #38 and @knocte for reporting #22. - Added `workflow: "auto-summary"` for `web_search` to generate a summary without opening the browser curator. Thanks @Ilm-Alan for PR #57 and @baflow for reporting #32. - Added Tavily Search API provider support with `TAVILY_API_KEY` / `tavilyApiKey`, domain filters, recency filters, and auto-provider fallback integration. - Added Parallel search provider support, auto-provider fallback integration, and Parallel `fetch_content` extraction fallback. - Resolved `web-search.json` from `PI_CODING_AGENT_DIR`, then `XDG_CONFIG_HOME/pi`, before falling back to `~/.pi`. - Added `GOOGLE_GEMINI_BASE_URL` / `geminiBaseUrl` for routing Gemini generate-content calls through compatible gateways, plus `CLOUDFLARE_API_KEY` / `cloudflareApiKey` authentication for Cloudflare AI Gateway. Thanks @meatballhat-cf for PR #76 and @tynril for reporting #74. ### Removed - Removed the deprecated `code_search` tool that duplicated the Exa search provider from `web_search`. Thanks @picasso250 for PR #62 and reporting #61. ### Fixed - Kept curator sessions alive when browser auto-open fails and surfaced a copyable curator URL for Docker, WSL, and headless environments. Thanks @rca, @runningman84, and @k0valik for reporting #92, #93, and #55. - Updated `@mozilla/readability` to `^0.6.0` for GHSA-3p6v-hrg8-8qj7. Thanks @omar-elmountassir for reporting #86, and @henriquebastos and @av1155 for PRs #35 and #68. - Limited curator summary model selection to Pi `enabledModels` when configured, and fall back to a deterministic no-billing summary instead of silently calling unrelated catalog models. Thanks @Horace1423 for reporting #73. - Let configured Exa API keys use Exa's own account limits instead of blocking at the legacy local 1,000-request counter. Thanks @totoDoP for reporting #80. - Supported parallel `web_search` curator tool calls with per-call browser and cancellation state. - Prevented curator sessions from hanging after searches finish when no browser connects, and finalized connected idle sessions once the curator timeout elapses. - Added explicit `role: "user"` fields to Gemini generate-content requests for Vertex AI-backed proxy compatibility. Thanks @meatballhat-cf for PR #76 and @tynril for reporting #74. ## [0.11.0] - 2026-06-24 ### Added - Added OpenAI Responses API web search with Codex subscription auth, OpenAI API key auth, required web-search tool use, model-registry header forwarding, and source extraction from citations and web-search sources. - Added Brave Search API provider with `BRAVE_API_KEY` / `braveApiKey`, freshness mapping, and domain include/exclude handling. - Added OpenAI and Brave to `auto` provider routing, explicit `provider` selection, curator provider buttons, provider tags, and provider availability checks. ### Changed - Updated extension imports for Pi 0.80.x, including the `@earendil-works` package namespace, `.ts` local module specifiers, and the `@earendil-works/pi-ai/compat` entrypoint for legacy model helpers. - Declared Pi-bundled packages as peer dependencies so Pi can provide its runtime aliases when loading the extension. ### Fixed - Aligned custom message display values and widget clearing with the current Pi extension API types. - Surfaced real YouTube Gemini API failures instead of replacing them with the generic Chrome/API-key guidance. - Kept Brave snippets out of `inlineContent` so `includeContent` searches still fetch full page content through the background extraction pipeline. - Prevented constrained auto searches from silently losing recency/count constraints by preferring Exa/Brave over OpenAI when OpenAI cannot enforce them strictly. ### Removed - Removed the committed `package-lock.json` from the extension package and ignored future lockfiles. ## [0.10.7] - 2026-05-02 ### Added - Added `summaryModel` config for choosing the default curator summary draft model from `~/.pi/web-search.json`. ### Fixed - Made Gemini Web browser-cookie access opt-in via `allowBrowserCookies` or `PI_ALLOW_BROWSER_COOKIES=1`, preventing surprise macOS Keychain prompts during provider checks. - Restored `code_search` after Exa removed the `get_code_context_exa` MCP tool by falling back to `web_search_exa` with code-focused queries. - Migrated extension tool schemas from `@sinclair/typebox` to Pi's bundled `typebox` 1.x import path. ## [0.10.6] - 2026-04-04 ### Changed - Added `promptSnippet` metadata for `web_search`, `code_search`, `fetch_content`, and `get_search_content` so Pi 0.59+ includes these tools in the default prompt tool section and improves discoverability of research/fetch flows. ## [0.10.5] - 2026-04-03 ### Fixed - Forward dynamic request `headers` from `ctx.modelRegistry.getApiKeyAndHeaders()` into `complete()` for query rewriting and summary generation, finishing the pi 0.63+ auth migration for providers that require per-request headers. - Removed legacy `session_switch`/`session_fork` lifecycle listeners and rely on immutable-session `session_start` reinitialization. ## [0.10.4] - 2026-03-27 ### Added - **Workflow-based curator hard cutover (`workflow`).** Replaced `curate` with `workflow: "none" | "summary-review"`, added summary-review approval flow with `POST /summarize`, made summary text the primary returned output while retaining raw curated evidence in `details`, and switched timeout handling to submit-first with deterministic summary fallback when no approved draft exists. - **Auto-open curator for all `web_search` runs (single + multi query).** Searches now open the curator window immediately and stream results live for review workflows; the old countdown/auto-condense fallback path was removed. - **Exa.ai search provider.** Neural/semantic search available alongside Perplexity and Gemini. 1,000 free requests/month. Set `EXA_API_KEY` env var or `exaApiKey` in `~/.pi/web-search.json`, or select explicitly with `provider: "exa"`. Includes built-in content extraction — when `includeContent` is true, full page text comes back with search results instead of requiring a separate background fetch. Monthly usage tracked in `~/.pi/exa-usage.json` with a warning at 80%. - **Exa MCP fallback.** When no Exa API key is configured, search routes through `mcp.exa.ai` with zero setup. Supports basic search and `includeContent` but not domain/recency filtering (falls through to Gemini for those). - **`code_search` tool.** Code/documentation search via Exa MCP (`get_code_context_exa`). No API key required. Returns code examples, docs, and API references from GitHub, Stack Overflow, and official documentation. - **Glimpse native curator window.** On macOS with Glimpse installed, the search curator opens in a native WKWebView window instead of a browser tab. Faster launch, closer integration. Falls back to browser automatically when Glimpse is unavailable. - **Curator provider UX rewrite.** Replaced the provider dropdown with provider buttons (hidden when unavailable), made provider re-search additive, added provider badges on all cards (including errors), and switched button states to coverage-based logic keyed by logical query slots so duplicate query text is handled correctly. - **Per-card provider re-search.** Completed result cards now show "Also try" chips for other available providers. Clicking one searches the same query with that provider and adds a new card below, keeping both results for comparison. - **Query rewrite with magic wand.** The "Add a search" input now has a ✨ button that rewrites the entered query using a fast LLM (haiku/flash) to make it more specific and effective. The improved query replaces the input text for review before searching. - **Summary model chooser redesign.** Summary review now follows a provider-first flow: provider dropdown first, then provider-scoped model dropdown. - **Active summary-generation loading state.** Summary review now shows a dedicated animated in-progress panel while `/summarize` is running (panel sweep, pulse + shimmer placeholders, staged copy updates, and active model label) instead of only a static disabled textarea. - **Model/provider guard retry for summary generation.** If a selected summary model fails with model/provider configuration errors, curator now retries once with Auto model selection before surfacing a terminal error. - **Feedback input for summary regeneration.** Optional text field in the summary review panel for providing instructions when regenerating a summary. Only the Regenerate button passes feedback to the prompt; auto-generation and the initial Generate button do not. Feedback is cleared on success and persisted on error for retry. - **`/curator` command.** Toggle or configure the curator workflow at runtime: `/curator on`, `/curator off`, `/curator summary-review`, or `/curator` to toggle. Persists to `~/.pi/web-search.json` and takes effect on the next `web_search` call. - **Config-based workflow default.** Added `workflow` field to `~/.pi/web-search.json` for persistent curator preference. Per-call `workflow` parameter on `web_search` takes priority, then config, then the built-in default (`summary-review` with UI, `none` without). - **"Send selected results without summary" button.** New secondary button in the curator that submits curated results directly without generating or approving a summary. Works from any stage when results are selected. Output uses the raw curated results format with per-query detail. - **Summary preview modal.** Preview button in the summary actions opens a full-page modal with the summary rendered as formatted markdown. Includes Approve and Regenerate actions with an inline model selector for switching models without leaving the preview. - **"Also try" provider chips on searching cards.** Provider re-search chips now appear on cards still in-flight, not just completed ones, so alternative-provider searches can be kicked off in parallel without waiting. - **Live search progress in heading.** Hero heading shows "2 of 4 Searches Complete" while searches are running, with status line showing "2 completed, 2 searching". Reverts to "N Searches Complete" when all finish. - **Summary subtitle with selection count.** Subtitle now shows "Summary of N selected results" and reacts to selection changes ("Selection changed — regenerating summary…"). - **Summary model selector relocated.** Moved the provider/model dropdowns from the hero area into the summary panel header, next to the title, so the model choice is adjacent to the summary it controls. - **Improved collapsed TUI preview.** Collapsed search result cards now show adaptive content: summary text when available, curated query titles with source counts when results were sent without summary, or a fallback text line otherwise. Line count hint matching pi's built-in pattern: `... (X more lines, Y total, ctrl+o to expand)`. - **Inline annotation feedback in preview modal.** Select any text in the rendered summary preview to get a popover with a quoted excerpt and a feedback textarea. Regenerate from the popover to send targeted feedback like `Regarding: "" — `. Supports Cmd/Ctrl+Enter to submit and Escape to dismiss. - **Concurrent add-search and alt-chip searches.** The "Add a search" input and "Also try" provider chips are no longer locked while other searches are in-flight. Multiple searches can run in parallel. - **Batch provider search shows searching cards immediately.** Clicking a provider button now creates placeholder cards with loading animations upfront instead of waiting for results to arrive. ### Changed - Exa search now always requests text content from both direct API and MCP paths (3000 chars default, 50000 with `includeContent`) instead of requesting highlights only. Ensures consistent answer quality regardless of whether Exa returns highlight snippets. - Adapted model registry calls to pi SDK changes: `getApiKey()` → `getApiKeyAndHeaders()` in `index.ts` and `summary-review.ts`, and `getAvailable()` from async to sync. - Hoisted dynamic `await import()` calls to static top-level imports in `gemini-web.ts`, `video-extract.ts`, and `youtube-extract.ts`. - Removed legacy `session_switch`/`session_fork` lifecycle listeners and rely on immutable-session `session_start` reinitialization. ### Removed - **`result-review` workflow.** Hard cutover — only `"none"` and `"summary-review"` remain. Removed from `WebSearchWorkflow` type, `resolveWorkflow()`, tool schema, `/websearch` command, and `/curator` command. ### Fixed - Summary generation no longer hard-fails on empty model payloads (`content parts: none`): empty-response failures now fall back to deterministic summary output with explicit fallback metadata (`fallbackReason: "summary-model-empty-response"`) instead of surfacing a terminal UI error. - Deterministic fallback summaries now strip trailing `Source:`/`Sources:` boilerplate from provider answer text before building query previews, preventing noisy source-list dumps from replacing actual summary prose. Fixed regex matching so `Source:` tokens at the start of provider answers are correctly detected and removed. - Curator now allows provider switching and add-search actions while summary generation is running. User-initiated search mutations supersede the in-flight summary request client-side and return the UI to results mode so searching can continue without waiting for draft completion. - Curator client now handles non-2xx server responses consistently across `/provider`, `/search`, `/submit`, `/cancel`, and heartbeat requests, and no longer leaves timeout/heartbeat POST promises unhandled. - Prevented duplicate completion counting when the same result card is updated more than once. - Fixed background fetch abort detection to avoid crashing on non-`Error` rejection values. - Fixed YouTube detection for protocol-less links (`youtu.be/...`) by allowing regex fallback after URL-parse failures. - Fixed README probing in GitHub clone mode to continue scanning alternate README filenames when one candidate is unreadable. - Removed dead `search-filter.ts` code path and its stale README file-table entry. - Gemini provider routing now preserves provider failure context in explicit/final Gemini paths instead of silently collapsing errors to `null`. - Hardened auto-provider fallback diagnostics: when Exa/Perplexity/Gemini are available but fail at runtime, the thrown error now includes all provider-specific failure reasons instead of dropping context. - Prevented queued SSE event loss in curator reconnect flows by preserving unsent buffered messages when an SSE flush write fails. - Hardened curator server provider validation (`/provider`, `/search`) so invalid/unavailable provider names are rejected explicitly instead of mutating session state. - Fixed `file://` local-video path handling to decode URL-escaped paths and treat malformed file URLs as invalid inputs instead of throwing. - Prevented path-escape reads in GitHub clone rendering by constraining blob/tree paths to remain within the cloned repository root. - Prevented symlink-escape traversal in clone tree/list rendering (`buildTree` / `buildDirListing`) by skipping entries that resolve outside the repository root. - Config parse errors from YouTube/video/Gemini fallback paths are now surfaced explicitly to users instead of silently collapsing to generic fallback messages. - Fixed provider switching during streaming curator searches (`web_search` + `/websearch`) so remaining queued searches use the latest selected provider instead of the initial one. - Fixed `fetch_content` timestamp behavior to fail explicitly on invalid timestamp formats and non-video/non-YouTube targets instead of silently ignoring `timestamp` and falling through to generic extraction. - Removed `Promise.withResolvers` from `web_search` curation flow for broader Node compatibility (no ES2024 runtime requirement). - Hardened PDF metadata handling (`pdf-extract.ts`) with typed metadata guards and safe `maxPages` clamping. - Normalized configured/default provider values in `index.ts` and `gemini-search.ts` (including case-insensitive values) so invalid provider strings no longer leak into curator state and now safely fall back to `auto` resolution. - Hardened config string/number normalization (`index.ts`, `gemini-search.ts`, `gemini-web.ts`, `youtube-extract.ts`, `video-extract.ts`): whitespace-only model/profile/provider values now safely fall back to defaults, and invalid/non-positive video `maxSizeMB` no longer disables local video detection accidentally. - Hardened API key/config handling (`gemini-api.ts`, `perplexity.ts`, `exa.ts`, `github-extract.ts`): whitespace/invalid key values are no longer treated as configured credentials, and invalid GitHub clone config booleans/numbers/paths now safely fall back to defaults instead of causing silent misconfiguration. - Fixed mid-flight abort behavior in extraction fallbacks (`extract.ts`, `github-extract.ts`): aborted YouTube/local-video/GitHub extraction no longer degrades into misleading fallback guidance and now returns explicit `Aborted` results instead of continuing with fallback network work. - Fixed abort lifecycle consistency in GitHub clone extraction (`github-extract.ts`): aborted clone attempts now correctly close activity entries as aborted and avoid persisting failed-abort clone cache entries that could force stale API-only fallback on later requests. - Fixed activity-monitor lifecycle for shared-clone races (`github-extract.ts`): callers that race onto an already-started clone now properly close their own activity entry (success/error/aborted) instead of leaving stale pending entries. - Fixed oversized-repo activity status accuracy (`github-extract.ts`): API fallback paths now mark activity success only when API fetch succeeds and correctly log an error when API fallback is unavailable instead of reporting false-positive success. - Fixed clone-failure fallback telemetry (`github-extract.ts`): when clone fails but API fallback succeeds, activity now reports success instead of remaining an error, and aborted clone-failure paths now short-circuit without extra fallback fetches. - Fixed GitHub URL host matching (`github-extract.ts`) so `https://www.github.com/...` URLs are recognized as clone/API candidates instead of silently falling through to generic HTTP extraction. - Hardened curator markdown rendering (`curator-page.ts`) against HTML/script injection by escaping provider answer text before markdown rendering, preserving markdown formatting while blocking raw HTML execution in the UI. - Closed additional curator link-safety gaps (`curator-page.ts`): sanitized markdown-rendered `href`/`src` protocols and source-link URLs to block `javascript:`/non-http schemes, enforced safe link attrs (`noopener noreferrer`), and stripped inline event-handler attributes from rendered markdown DOM. - Hardened inline script data serialization in curator page generation (`curator-page.ts`) by escaping Unicode line/paragraph separators (`U+2028`, `U+2029`) in `safeInlineJSON`, preventing malformed script blocks or injection edge cases from unescaped JSON payloads. - Fixed abort telemetry misclassification in media extractors (`youtube-extract.ts`, `video-extract.ts`): canceled extractions now log activity as aborted (`status: 0`) instead of incorrectly reporting `all ... paths failed` errors after abort races. - Fixed GitHub URL path decoding in clone/API extraction (`github-extract.ts`): percent-encoded path segments (for example `%20`) are now decoded before blob/tree resolution, so URLs that point to files/directories with encoded characters no longer fall through to incorrect "path not found" output. - Fixed error-signal downgrade in local-video API fallback (`video-extract.ts`): `tryVideoGeminiApi` now rethrows config parse failures (`Failed to parse ~/.pi/web-search.json`) instead of swallowing them as `null`, preserving actionable root-cause errors. - Fixed provider config type hardening in search routing (`index.ts`): `normalizeProviderInput` now guards non-string config values before trimming so malformed `provider` entries in `~/.pi/web-search.json` no longer crash runtime provider resolution with `value.trim is not a function`. - Simplified provider typing flow in curator/search orchestration (`index.ts`): narrowed `resolveProvider` and `PendingCurate.defaultProvider` to resolved provider types, normalized incoming provider strings at callback boundaries, and removed redundant `as SearchProvider | undefined` casts while preserving search behavior. - Improved curator loading experience in `curator-page.ts`: added animated skeleton loading panel in the content area while searches are in-flight, upgraded searching card visuals with shimmer/active-state styling, and wired loading visibility to real search state transitions (including add-search, done, submit/cancel, and timeout paths). - Updated curator session timeout defaults in `index.ts`: curator now starts at 20 seconds by default (down from 60) and can be configured via `curatorTimeoutSeconds` in `~/.pi/web-search.json` (capped at 600 seconds). - Hardened `/websearch` startup error handling in `index.ts`: config/provider bootstrap now runs behind explicit error handling so malformed `~/.pi/web-search.json` no longer throws uncaught command errors before the existing server-start try/catch; users now receive a direct UI error with parse context. - Hardened extension bootstrap config handling in `index.ts`: shortcut initialization now uses guarded config loading, logging parse errors and falling back to default shortcuts instead of crashing extension registration on malformed `~/.pi/web-search.json`. - Simplified curator-timeout config plumbing in `index.ts` by removing an unused `getCuratorTimeoutSeconds(config)` parameter path and keeping a single config-read code path. - Simplified curator bootstrap wiring in `index.ts` by extracting shared provider/timeout setup (`ProviderAvailability`, `CuratorBootstrap`, `getProviderAvailability`, `loadCuratorBootstrap`) and removing duplicated availability assembly across `web_search` and `/websearch` flows. - Hardened SSE event parsing in curator client (`curator-page.ts`): malformed JSON payloads from SSE `data:` lines now surface as user-visible errors instead of crashing the page via uncaught `JSON.parse` exceptions. - Fixed "Send results" producing a deterministic summary instead of raw curated results. The submit payload now uses a `rawResults` flag to distinguish explicit "Send results" clicks from timeout-via-submit, which correctly falls back to a deterministic summary. - Exa search results with no highlight snippets now fall back to `item.text` (truncated to 1000 chars) instead of producing empty answers. Empty snippets are also skipped during MCP answer assembly. - Exa MCP result parsing now handles `Highlights:` response blocks in addition to `Text:` blocks, and strips trailing `---` separators from parsed content. - Fixed stale heading count after a user-added search fails and its card is removed. `updateSummaryText()` is now called in all card-removal error paths. - Fixed heading not reflecting new in-progress searches immediately. Adding a search via "Also try" or "Add a search" now updates the heading to show the new total (e.g., "4 of 5 Searches Complete") right away instead of waiting for completion. ## [0.10.3] - 2026-03-12 ### Added - `/google-account` command to report the active Google account currently authenticated for Gemini Web. - `chromeProfile` config support for targeting a non-default Chromium profile when reading Gemini Web cookies. - `searchModel` config support for overriding the Gemini API model used by `web_search`. ### Changed - Chromium cookie extraction now tries Helium, Chrome, and Arc on macOS, plus Chromium and Chrome on Linux, with profile-aware cookie paths and per-platform key handling. - Gemini Web availability checks now pass required cookie names into cookie extraction and can look up the active signed-in Google account without changing existing `isGeminiWebAvailable()` callers. - README documentation now covers macOS/Linux cookie extraction limits, the new config fields, the `/google-account` command, and the expanded `chrome-cookies.ts` role. ## [0.10.2] - 2026-02-18 ### Added - **Interactive search curation.** Press Ctrl+Shift+S during or after a multi-query search to open a browser-based review UI. Results stream in live via SSE. Pick which queries to keep, add new searches on the fly, switch providers — then submit to send only the curated results to the agent. - **Auto-condense pipeline.** When the countdown expires without manual curation, a single LLM call (Claude Haiku by default) condenses all search results into a deduplicated briefing organized by topic. Preprocessing enriches the prompt with URL overlap, answer similarity, and source quality analysis. Configure via `"autoFilter"` in `~/.pi/web-search.json`. Full uncondensed results stored and retrievable via `get_search_content`. - **Configurable keyboard shortcuts.** Both shortcuts (curate: Ctrl+Shift+S, activity monitor: Ctrl+Shift+W) can be remapped via `"shortcuts"` in `~/.pi/web-search.json`. Changes take effect on restart. - **`/websearch` command** — opens the curator directly from pi without an agent round-trip. Accepts optional comma-separated queries or opens empty. - **Task-aware condensation.** Optional `context` parameter on `web_search` — a brief description of the user's task. The condenser uses it to focus the briefing on what matters. - **Provider selection** — global dropdown in the curator UI to switch between Perplexity and Gemini. Persists to `~/.pi/web-search.json`. - **Live condense status in countdown.** Shows "condensing..." while the LLM is working, then "N searches condensed" once complete. - Markdown rendering in curator result cards via marked.js. - Query-level result cards with expandable answers and source lists. Check/uncheck to include or exclude. - SSE streaming with keepalive, socket health checks, and buffered delivery. - Idle-based timer (60s default, adjustable). Timeout sends all results as safe default. - Keyboard shortcuts: Enter (submit), Escape (skip), A (toggle all). - Dark/light theme via `prefers-color-scheme` with teal accent palette. ### Changed - **Curate enabled by default.** Multi-query searches show a 10-second review window; single queries send immediately. Pass `curate: false` to opt out. - **Curate shortcut opens browser immediately, even mid-search.** Remaining results stream in live via SSE. - **Tool descriptions encourage multi-query research.** The `queries` param explains how to vary phrasing and scope across 2-4 queries, with good/bad examples. - **Curated results instruct the LLM.** Tool output prefixed with an instruction telling the LLM to use curated results as-is. - Expanded view shows full answer text per query with source titles and domains. - Non-curated `web_search` calls now respect the saved provider preference. - Config helpers generalized from `loadSavedProvider`/`saveProvider` to `loadConfig`/`saveConfig`. ### Fixed - Curated `onSubmit` passed the original full query list instead of the filtered list, inflating `queryCount`. - Collapsed curated status mixed source URL counts with query counts. ### New files - `curator-server.ts` — ephemeral HTTP server with SSE streaming, state machine, heartbeat watchdog, and token auth. - `curator-page.ts` — HTML/CSS/JS for the curator UI with markdown rendering and overlay transitions. - `search-filter.ts` — auto-condense pipeline: preprocessing, LLM condensation via pi's model registry, and post-processing (citation verification, source list completion). ## [0.7.3] - 2026-02-05 ### Added - Jina Reader fallback for JS-rendered pages. When Readability returns insufficient content (cookie notices, consent walls, SPA shells), the extraction chain now tries Jina Reader (`r.jina.ai`) before falling back to Gemini. Jina handles JavaScript rendering server-side and returns clean markdown. No API key required. - JS-render detection heuristic (`isLikelyJSRendered`) produces more specific error messages when pages appear to load content dynamically. - Actionable guidance when all extraction methods fail, listing steps to configure Gemini API or use `web_search` instead. ### Changed - HTTP fetch headers now mimic Chrome (realistic `User-Agent`, `Sec-Fetch-*`, `Accept-Language`) instead of the default Node.js user agent. Reduces blocks from bot-detection systems. - Short Readability output (< 500 chars) is now treated as a content failure, triggering the fallback chain. Previously, a 266-char cookie notice was returned as "successful" content. - Extraction fallback order is now: HTTP+Readability → RSC → Jina Reader → Gemini URL Context → Gemini Web → error with guidance. ### Fixed - `parseTimestamp` now rejects negative values in colon-separated format (`-1:30`, `1:-30`). Previously only the numeric path (`-90`) rejected negatives, while the colon path computed and returned negative seconds. ## [0.7.2] - 2026-02-03 ### Added - `model` parameter on `fetch_content` to override the Gemini model per-request (e.g. `model: "gemini-2.5-flash"`) - Collapsed TUI results now show a 200-char text preview instead of just the status line - LICENSE file (MIT) ### Changed - Default Gemini model updated from `gemini-2.5-flash` to `gemini-3-flash-preview` across all API, search, URL context, YouTube, and video paths. Gemini Web gracefully falls back to `gemini-2.5-flash` when the model header isn't available. - README rewritten: added tagline, badges, "Why" section, Quick Start, corrected "How It Works" routing order, fixed inaccurate env var precedence claim, added missing `/v/` YouTube format, restored `/search` command docs, collapsible Files table ### Fixed - `PERPLEXITY_API_KEY` env var now takes precedence over config file value, matching `GEMINI_API_KEY` behavior and README documentation (was reversed) - `package.json` now includes `repository`, `homepage`, `bugs`, and `description` fields (repo link was missing from pi packages site) ## [0.7.0] - 2026-02-03 ### Added - **Multi-provider web search**: `web_search` now supports Perplexity, Gemini API (with Google Search grounding), and Gemini Web (cookie auth) as search providers. New `provider` parameter (`auto`, `perplexity`, `gemini`) controls selection. In `auto` mode (default): Perplexity → Gemini API → Gemini Web. Backwards-compatible — existing Perplexity users see no change. - **Gemini API grounded search**: Structured citations via `groundingMetadata` with source URIs and text-to-source mappings. Google proxy URLs are resolved via HEAD redirects. Configured via `GEMINI_API_KEY` or `geminiApiKey` in config. - **Gemini Web search**: Zero-config web search for users signed into Google in Chrome. Prompt instructs Gemini to cite sources; URLs extracted from markdown response. - **Gemini extraction fallback**: When `fetch_content` fails (HTTP 403/429, Readability fails, network errors), automatically retries via Gemini URL Context API then Gemini Web extraction. Each has an independent 60s timeout. Handles SPAs, JS-heavy pages, and anti-bot protections. - **Local video file analysis**: `fetch_content` accepts file paths to video files (MP4, MOV, WebM, AVI, etc.). Detected by path prefix (`/`, `./`, `../`, `file://`), validated by extension and 50MB limit. Two-tier fallback: Gemini API (resumable upload via Files API with proper MIME types, poll-until-active and cleanup) → Gemini Web (free, cookie auth). - **Video prompt parameter**: `fetch_content` gains optional `prompt` parameter for asking specific questions about video content. Threads through YouTube and local video extraction. Without prompt, uses default extraction (transcript + visual descriptions). - **Video thumbnails**: YouTube results include the video thumbnail (fetched from `img.youtube.com`). Local video results include a frame extracted via ffmpeg (at ~1 second). Returned as image content parts alongside text — the agent sees the thumbnail as vision context. - **Configurable frame extraction**: `frames` parameter (1-12) on `fetch_content` for pulling visual frames from YouTube or local video. Works in five modes: frames alone (sample across entire video), single timestamp (one frame), single+frames (N frames at 5s intervals), range (default 6 frames), range+frames (N frames across the range). Endpoint-inclusive distribution with 5-second minimum spacing. - **Video duration in responses**: Frame extraction results include the video duration for context. - `searchProvider` config option in `~/.pi/web-search.json` for global provider default - `video` config section: `enabled`, `preferredModel`, `maxSizeMB` ### Changed - `PerplexityResponse` renamed to `SearchResponse` (shared interface for all search providers) - Extracted HTTP pipeline from `extractContent` into `extractViaHttp` for cleaner Gemini fallback orchestration - `getApiKey()`, `API_BASE`, `DEFAULT_MODEL` exported from `gemini-api.ts` for use by search and URL Context modules - `isPerplexityAvailable()` added to `perplexity.ts` as non-throwing API key check - Content-type routing in `extract.ts`: only `text/html` and `application/xhtml+xml` go through Readability; all other text types (`text/markdown`, `application/json`, `text/csv`, etc.) returned directly. Fixes the OpenAI cookbook `.md` URL that returned "Untitled (30 chars)". - Title extraction for non-HTML content: `extractTextTitle()` pulls from markdown `#`/`##` headings, falls back to URL filename - Combined `yt-dlp --print duration -g` call fetches stream URL and duration in a single invocation, reused across all frame extraction paths via `streamInfo` passthrough - Shared helpers in `utils.ts` (`formatSeconds`, error mapping) eliminate circular imports and duplication across youtube-extract.ts and video-extract.ts ### Fixed - `fetch_content` TUI rendered `undefined/undefined URLs` during progress updates (renderResult didn't handle `isPartial`, now shows a progress bar like `web_search` does) - RSC extractor produced malformed markdown for `
` blocks (backticks inside fenced code blocks) -- extremely common on Next.js documentation pages
- Multi-URL fetch failures rendered in green "success" color even when 0 URLs succeeded (now red)
- `web_search` queries parameter described as "parallel" in schema but execution is sequential (changed to "batch"; `urls` correctly remains "parallel")
- Proper error propagation for frame extraction: missing binaries (yt-dlp, ffmpeg, ffprobe), private/age-restricted/region-blocked videos, expired stream URLs (403), timestamp-exceeds-duration, and timeouts all produce specific user-facing messages instead of silent nulls
- `isTimeoutError` now detects `execFileSync` timeouts via the `killed` flag (SIGTERM from timeout was previously unrecognized)
- Float video durations (e.g. 15913.7s from yt-dlp) no longer produce out-of-range timestamps — durations are floored before computing frame positions
- `parseTimestamp` consistently floors results across both bare-number ("90.5" → 90) and colon ("1:30.5" → 90) paths — previously the colon path returned floats
- YouTube thumbnail assignment no longer sets `null` on the optional `thumbnail` field when fetch fails (was a type mismatch; now only assigned on success)

### New files
- `gemini-search.ts` -- search routing + Gemini Web/API search providers with grounding
- `gemini-url-context.ts` -- URL Context API extraction + Gemini Web extraction fallback
- `video-extract.ts` -- local video file detection, Gemini Web/API analysis with Files API upload
- `utils.ts` -- shared formatting and error helpers for frame extraction

## [0.6.0] - 2026-02-02

### Added
- YouTube video understanding in `fetch_content` via three-tier fallback chain:
  - **Gemini Web** (primary): reads Chrome session cookies from macOS Keychain + SQLite, authenticates to gemini.google.com, sends YouTube URL via StreamGenerate endpoint. Full visual + audio understanding with timestamps. Zero config needed if signed into Google in Chrome.
  - **Gemini API** (secondary): direct REST calls with `GEMINI_API_KEY`. YouTube URLs passed as `file_data.file_uri`. Configure via `GEMINI_API_KEY` env var or `geminiApiKey` in `~/.pi/web-search.json`.
  - **Perplexity** (fallback): uses existing `searchWithPerplexity` for a topic summary when neither Gemini path is available. Output labeled as "Summary (via Perplexity)" so the agent knows it's not a full transcript.
- YouTube URL detection for all common formats: `/watch?v=`, `youtu.be/`, `/shorts/`, `/live/`, `/embed/`, `/v/`, `m.youtube.com`
- Configurable via `~/.pi/web-search.json` under `youtube` key (`enabled`, `preferredModel`)
- Actionable error messages when extraction fails (directs user to sign into Chrome or set API key)
- YouTube URLs no longer fall through to HTTP/Readability (which returns garbage); returns error instead

### New files
- `chrome-cookies.ts` -- macOS Chrome cookie extraction using Node builtins (`node:crypto`, `node:sqlite`, `child_process`)
- `gemini-web.ts` -- Gemini Web client ported from surf's gemini-client.cjs (cookie auth, StreamGenerate, model fallback)
- `gemini-api.ts` -- Gemini REST API client (generateContent, file upload/processing/cleanup for Phase 2)
- `youtube-extract.ts` -- YouTube extraction orchestrator with three-tier fallback and activity logging

## [0.5.1] - 2026-02-02

### Added
- Bundled `librarian` skill -- structured research workflow for open-source libraries with GitHub permalinks, combining fetch_content (cloning), web_search (recent info), and git operations (blame, log, show)

### Fixed
- Session fork event handler was registered as `session_branch` (non-existent event) instead of `session_fork`, meaning forks never triggered cleanup (abort pending fetches, clear clone cache, restore session data)
- API fallback title for tree URLs with a path (e.g. `/tree/main/src`) now includes the path (`owner/repo - src`), consistent with clone-based results
- Removed unnecessary export on `getDefaultBranch` (only used internally by `fetchViaApi`)

## [0.5.0] - 2026-02-01

### Added
- GitHub repository clone extraction for `fetch_content` -- detects GitHub code URLs, clones repos to `/tmp/pi-github-repos/`, and returns actual file contents plus local path for further exploration with `read` and `bash`
- Lightweight API fallback for oversized repos (>350MB) and commit SHA URLs via `gh api`
- Clone cache with concurrent request deduplication (second request awaits first's clone)
- `forceClone` parameter on `fetch_content` to override the size threshold
- Configurable via `~/.pi/web-search.json` under `githubClone` key (enabled, maxRepoSizeMB, cloneTimeoutSeconds, clonePath)
- Falls back to `git clone` when `gh` CLI is unavailable (public repos only)
- README: GitHub clone documentation with config, flow diagram, and limitations

### Changed
- Refactored `extractContent`/`fetchAllContent` signatures from positional `timeoutMs` to `ExtractOptions` object
- Blob/tree fetch titles now include file path (e.g. `owner/repo - src/index.ts`) for better disambiguation in multi-URL results and TUI

### Fixed
- README: Activity monitor keybinding corrected from `Ctrl+Shift+O` to `Ctrl+Shift+W`

## [0.4.5] - 2026-02-01

### Changed
- Added package keywords for npm discoverability

## [0.4.4] - 2026-02-01

### Fixed
- Adapt execute signatures to pi v0.51.0: reorder signal, onUpdate, ctx parameters across all three tools

## [0.4.3] - 2026-01-27

### Fixed
- Google API compatibility: Use `StringEnum` for `recencyFilter` to avoid unsupported `anyOf`/`const` JSON Schema patterns

## [0.4.2] - 2026-01-27

### Fixed

- Single-URL fetches now store content for retrieval via `get_search_content` (previously only multi-URL)
- Corrected `get_search_content` usage syntax in fetch_content help messages

### Changed

- Increased inline content limit from 10K to 30K chars (larger content truncated but fully retrievable)

### Added

- Banner image for README

## [0.4.1] - 2026-01-26

### Changed
- Added `pi` manifest to package.json for pi v0.50.0 package system compliance
- Added `pi-package` keyword for npm discoverability

## [0.4.0] - 2026-01-19

### Added

- PDF extraction via `unpdf` - fetches PDFs from URLs and saves as markdown to `~/Downloads/`
  - Extracts text, metadata (title, author), page count
  - Supports PDFs up to 20MB (vs 5MB for HTML)
  - Handles arxiv URLs with smart title fallback

### Fixed

- Plain text URL detection now uses hostname check instead of substring match

## [0.3.0] - 2026-01-19

### Added

- RSC (React Server Components) content extraction for Next.js App Router pages
  - Parses flight data from `` tags
  - Reconstructs markdown with headings, tables, code blocks, links
  - Handles chunk references and nested components
  - Falls back to RSC extraction when Readability fails
- Content-type validation rejects binary files (images, PDFs, audio, video, zip)
- 5MB response size limit (checked via Content-Length header) to prevent memory issues

### Fixed

- `fetch_content` now handles plain text URLs (raw.githubusercontent.com, gist.githubusercontent.com, any text/plain response) instead of failing with "Could not extract readable content"

## [0.2.0] - 2026-01-11

### Added

- Activity monitor widget (`Ctrl+Shift+O`) showing live request/response activity
  - Displays last 10 API calls and URL fetches with status codes and timing
  - Shows rate limit usage and reset countdown
  - Live updates as requests complete
  - Auto-clears on session switch

### Changed

- Refactored activity tracking into dedicated `activity.ts` module

## [0.1.0] - 2026-01-06

Initial release. Designed for pi v0.37.3.

### Added

- `web_search` tool - Search via Perplexity AI with synthesized answers and citations
  - Single or batch queries (parallel execution)
  - Recency filter (day/week/month/year)
  - Domain filter (include or exclude)
  - Optional async content fetching with agent notification
- `fetch_content` tool - Fetch and extract readable content from URLs
  - Single URL returns content directly
  - Multiple URLs store for retrieval via `get_search_content`
  - Concurrent fetching (3 max) with 30s timeout
- `get_search_content` tool - Retrieve stored search results or fetched content
  - Access by response ID, URL, query, or index
- `/search` command - Interactive browser for stored results
- TUI rendering with progress bars, URL lists, and expandable previews
- Session-aware storage with 1-hour TTL
- Rate limiting (10 req/min for Perplexity API)
- Config file support (`~/.pi/web-search.json`)
- Content extraction via Readability + Turndown (max 10k chars)
- Proper session isolation - pending fetches abort on session switch
- URL validation before fetch attempts
- Defensive JSON parsing for API responses