# Local inference outage fallback

Status: source implementation complete and verified. Installed-runtime activation
is separate; the running daemon has not been upgraded or restarted.

Request: reuse the free inference onboarding catalogue at
https://robit-man.github.io/inference/ when local inference or its broker fails.
Provide this behavior by default in the shared runtime used by terminal, Telegram,
daemon chat and native workers, with host-side credential references.

## Anchored findings

- Onboarding `FreeInferencePool` has validated anonymous discovery and seeds, but
  its setup-only limits and tool stripping make its request method unsuitable
  for normal agent tasks.
- Normal inference uses `OllamaAgenticBackend`, including Telegram and workers.
  The previous TUI-only history cascade races full prompts across endpoints,
  reuses the local model name and can replay partial streamed output.
- Telegram can reject local admission before calling the backend, and previously
  acquired a local GPU slot even for remote providers.
- Runtime credentials have a generic primary key; fallback must never inherit it.

## Work orders and acceptance

1. Share the bounded anonymous catalogue parser and seed definitions between
   onboarding and runtime. Never evaluate catalogue JavaScript or send secrets
   while discovering routes.
2. Add a sequential, provider/model-specific default cascade for local runtime
   transport failures and broker unavailability. Bound local waiting, each remote
   attempt and the whole cascade; honor cancellation and dispatch ownership.
3. Preserve complete context, tool schemas and structured output requirements.
   Never drop tools or retry a stream after any output has escaped. Preserve
   explicit configured-endpoint requests and local hardware ownership gates.
4. Store fallback credential environment variable names, resolve each only for
   its canonical HTTPS provider, deny redirects and scrub known secret values
   from every outbound context field. Return host-owned errors and route metadata.
   An authenticated provider receives only its own key in its auth header.
5. Remove the incompatible automatic TUI history cascade. Keep Telegram remote
   requests independent of local GPU admission and route admission failures
   through the same shared fallback mechanism.
6. Verify normal production backend construction with mocked local outages and
   remote HTTP responses, unary and streaming, Stop, exhaustion, context/tool
   preservation, env rotation, cross-provider isolation and credential echoes.
   Run onboarding and relevant existing transport regressions and workspace build.
7. Commit and push only task-owned files after verification. Record deployment
   separately from source completion; do not infer a running-bot upgrade.

No local live inference or model loading is required for these regression tests.
Earlier desktop/browser work remains recorded under
`rapid-computer-use-20261011/` and is outside this repair.

## Delivered behavior

- Shared `OllamaAgenticBackend` performs bounded local-first fallback for normal
  loopback requests. Configured provider/model profiles precede compatible
  anonymous routes; exact-endpoint requests and zero-token probes remain pinned.
- Failed local routes cool down across fresh backend instances without synthetic
  generation probes. Remote routes have bounded attempts and a shared deadline.
- Fallback profiles contain environment references only. Credentials resolve
  again after destination validation, never inherit primary credentials, and
  authenticate only their canonical provider over HTTPS without redirects.
  Known credential values and common encodings are scrubbed from context,
  responses, streaming fragments and host route receipts.
- TUI uses the shared cascade. Telegram passes local admission failures through
  it, bounds admission waiting, releases local slots before remote dispatch and
  reports the selected remote model. Stop and dispatch ownership remain enforced.
- Partial streams and exhausted fallback are terminal throughout controller,
  transient retry and brute-force rescue paths. Existing changed-request context
  and output-capacity recovery remains available. Image fallback currently has
  no verified routes and fails closed.

## Verification

- `pnpm -r build`: all 11 workspace build projects passed.
- Orchestrator: 244 tests passed across 13 suites, including policy, credentials,
  production backend unary/streaming fallback, terminal runner retries,
  canonical dispatch, interruption, pinned endpoints and network inventory.
- CLI: 191 tests passed across 13 suites, including actual Telegram backend
  admission, Stop and slot release, onboarding, split-stream transport,
  delayed/subprocess failure, managed Stop and web client runtime.
- Three synthetic remote checks used the compiled production backend with a
  host-injected local outage: LLM7 text (2.873 s), Hermes native tool call
  (1.985 s), Hermes streamed native tool call (2.139 s). No local inference,
  model loading, user-task context or executed tool action was involved.
  Evidence: `inference-fallback-live-20261012.json`; reproduction:
  `scripts/inference-fallback-smoke.mjs`.
- Source delivery follows the scoped commit/push policy; the final handoff
  records its commit and remote branch. No npm publication, global installation
  or active-process restart was performed by this work order.
