# Bring Your Own Inference

Omnius routes local and hosted models through an explicit provider descriptor.
The descriptor determines the wire protocol, URL paths, authentication
headers, and supported operations. It is the source of truth used by setup,
model discovery, TUI runs, REST chat, summaries, voice, Telegram, sponsors,
and endpoint probes.

## Discover The Provider Contract

```bash
omnius discover "bring your own inference"
omnius show provider.anthropic
omnius show provider.gemini
omnius discover "provider protocol" --kind provider
```

From the daemon:

```bash
curl -s "http://127.0.0.1:11435/v1/discovery?q=provider&kind=provider"
curl -s "http://127.0.0.1:11435/v1/discovery/provider.anthropic"
```

## Protocol Matrix

| Provider | Protocol | Canonical base | Authentication |
| --- | --- | --- | --- |
| Ollama | `ollama` | `http://127.0.0.1:11434` | none by default |
| LM Studio, vLLM | `openai-chat` | local configured URL | optional Bearer |
| OpenAI | `openai-chat` | `https://api.openai.com/v1` | Bearer |
| Anthropic / Claude | `anthropic-messages` | `https://api.anthropic.com/v1` | `x-api-key` plus `anthropic-version` |
| Google Gemini | `openai-chat` | `https://generativelanguage.googleapis.com/v1beta/openai` | Bearer plus Omnius client header |
| OpenRouter, Groq, Chutes, DeepInfra, Together, Fireworks, Mistral, Cerebras, SambaNova, NVIDIA NIM, Hyperbolic | `openai-chat` | provider preset | Bearer |
| Sponsor and COHERE peers | negotiated Omnius endpoint | discovered peer | peer/session policy |
| Custom | explicitly selected | user supplied | configured for that endpoint |

Known aliases include `claude` for Anthropic and `google` / `google-ai` for
Gemini.

## Anthropic Messages Translation

Anthropic is not treated as an OpenAI-shaped URL. Omnius sends
`POST /v1/messages` with `x-api-key`, `anthropic-version: 2023-06-01`, and
JSON content type. The shared adapter translates:

- the OpenAI-style system message into Anthropic's top-level `system`;
- ordinary user and assistant messages into Messages content blocks;
- tool definitions into Anthropic tool declarations;
- assistant tool calls into `tool_use` blocks;
- tool results into `tool_result` blocks;
- Anthropic text, tool calls, streaming events, stop reasons, and usage back
  into Omnius' common completion shape.

This behavior follows Anthropic's
[Messages API](https://platform.claude.com/docs/en/api/messages/create) and
[authentication contract](https://platform.claude.com/docs/en/manage-claude/authentication).

## Gemini OpenAI Compatibility

Gemini uses Google's official OpenAI-compatible prefix:

```text
https://generativelanguage.googleapis.com/v1beta/openai
```

Omnius appends compatible resources such as `/models`,
`/chat/completions`, and `/embeddings`, authenticates with
`Authorization: Bearer <key>`, and identifies the client with
`x-goog-api-client: omnius/<version>`. See Google's
[OpenAI compatibility guide](https://ai.google.dev/gemini-api/docs/openai).

The compatibility API is not a promise that every Gemini-native feature is
available. Discover Omnius' advertised provider capabilities before depending
on a provider-specific extension.

## Key Resolution

Omnius resolves upstream provider credentials in this exact order:

```text
OMNIUS_PROVIDER_API_KEY
  → OMNIUS_MODEL_API_KEY
  → OMNIUS_UPSTREAM_API_KEY
  → OMNIUS_API_KEY
  → VLLM_API_KEY
  → persisted endpoint configuration
```

The first defined credential wins. REST bearer keys are a separate boundary:
use `OMNIUS_REST_API_KEY` or `OMNIUS_REST_API_KEYS` for callers of the daemon.
Never put either provider or REST credentials in URLs, logs, discovery data,
or agent guidance files.

## Fail-Closed Custom Endpoints

Omnius may infer a protocol only for a registered provider or an unambiguous
legacy configuration:

- a known host selects its registered protocol;
- a legacy `ollama` backend selects `ollama`;
- a legacy `vllm` backend selects `openai-chat`;
- a custom or ambiguous endpoint must declare its protocol explicitly.

When the protocol is ambiguous, Omnius performs no model listing, health
probe, embeddings request, chat request, or generated-token request. It keeps
the saved endpoint disabled and returns an actionable configuration error.
It never falls back to Ollama, CPU, a different provider, or a guessed
OpenAI-compatible path.

## Hardware Gate

The repository's `AGENTS.md` requires a live hardware preflight before local
inference. Confirm the designated endpoint, exact model tag, actual GPU
placement with `nvidia-smi`, and absence of the test runner on an unintended
device before generating tokens. If the endpoint-to-GPU mapping cannot be
verified, stop instead of falling back.

## Verify Without Generating Tokens

Use discovery and configuration inspection first. Endpoint/model probes may
contact a provider and therefore still require the correct protocol and
credential. A local model-load probe must obey the hardware gate.

For a service integration, also require the minimum Omnius version described
in [Service Version Compatibility](../operations/version-compatibility.md).