# Models

The models API exposes the local model index built from OpenRouter's public model catalogue. It is used internally to populate `context_size_limit` on new sessions and to fall back to a calculated cost when the LLM provider does not return a `usage.cost` field in its response.

The index is stored in `config/models.json` and loaded once at startup (cached in memory). Refresh it any time with `POST /models/refresh` or `npm run fetch-models`.

---

## GET /models

Returns the full model index.

**Response**
```json
{
  "updated_at": "2026-03-06T03:00:00.000Z",
  "models": [
    {
      "id": "anthropic/claude-sonnet-4.6",
      "name": "Anthropic: Claude Sonnet 4.6",
      "context_length": 1000000,
      "max_completion_tokens": 128000,
      "pricing": {
        "prompt": 0.000003,
        "completion": 0.000015,
        "cache_read": 0.0000003,
        "cache_write": 0.00000375
      },
      "input_modalities": ["text", "image"],
      "output_modalities": ["text"],
      "source": "builtin"
    }
  ]
}
```

**Model object fields**

| Field | Type | Description |
|-------|------|-------------|
| `id` | string | OpenRouter model identifier (e.g. `anthropic/claude-sonnet-4.6`) |
| `name` | string | Human-readable display name |
| `source` | string | Origin of the model entry: `"builtin"` (from OpenRouter fetch), `"custom"` (from `custom_models.json`), or `"claude-cli"` (discovered via Claude CLI / Agent SDK) |
| `context_length` | integer\|null | Context window size in tokens |
| `max_completion_tokens` | integer\|null | Maximum output tokens the provider allows |
| `pricing.prompt` | number | Price per input token in USD |
| `pricing.completion` | number | Price per output token in USD |
| `pricing.cache_read` | number | Price per cached input token (0 if not applicable) |
| `pricing.cache_write` | number | Price per cache-write token (0 if not applicable) |
| `input_modalities` | string[] | Supported input types (e.g. `["text", "image"]`) |
| `output_modalities` | string[] | Supported output types |

**Example**
```bash
curl http://localhost:5050/models
```

---

## GET /models/:provider/:name

Get a single model by its two-part ID.

**Path parameters**

| Param | Description |
|-------|-------------|
| `provider` | Provider slug, e.g. `anthropic` |
| `name` | Model slug, e.g. `claude-sonnet-4.6` |

**Response** — same shape as a single entry from `GET /models`

**Error responses**

| Code | Condition |
|------|-----------|
| `404 MODEL_NOT_FOUND` | Model ID not in the local index |

**Example**
```bash
curl http://localhost:5050/models/anthropic/claude-sonnet-4.6
```

---

## POST /models/refresh

Re-fetch the full model list from OpenRouter and reload the in-memory cache. Also fetches models available through the Claude CLI by invoking the Agent SDK with zero tokens (`maxTurns: 0`). Models discovered from the Claude CLI are tagged with `source: "claude-cli"` and merged into the index alongside OpenRouter models.

Internally runs `scripts/fetch-models.js`, which writes the result to `config/models.json`.

**Response**
```json
{
  "refreshed": true,
  "updated_at": "2026-03-06T03:15:22.000Z",
  "models": [ ... ]
}
```

**Example**
```bash
curl -X POST http://localhost:5050/models/refresh
```

---

## GET /models/custom

Read the `custom_models.json` file for a given level.

**Query parameters**

| Param | Type | Description |
|-------|------|-------------|
| `level` | string | `global` or `project` — which custom models file to read |

**Response**
```json
{
  "level": "project",
  "path": "/home/user/workspace/.veil/custom_models.json",
  "exists": true,
  "data": {
    "models": {
      "my-org/custom-llm": {
        "name": "My Custom LLM",
        "context_length": 128000,
        "pricing": {
          "prompt": 0.000002,
          "completion": 0.000010
        }
      }
    }
  }
}
```

**Example**
```bash
curl "http://localhost:5050/models/custom?level=project"
```

---

## PUT /models/custom

Write to the `custom_models.json` file for a given level. Custom models are merged into the main model index with `source: "custom"`.

**Query parameters**

| Param | Type | Description |
|-------|------|-------------|
| `level` | string | `global` or `project` — which custom models file to write |

**Request body**
```json
{
  "models": {
    "my-org/custom-llm": {
      "name": "My Custom LLM",
      "context_length": 128000,
      "pricing": {
        "prompt": 0.000002,
        "completion": 0.000010,
        "cache_read": 0,
        "cache_write": 0
      }
    }
  }
}
```

**Response**
```json
{
  "written": true,
  "level": "project",
  "path": "/home/user/workspace/.veil/custom_models.json"
}
```

**Example**
```bash
curl -X PUT "http://localhost:5050/models/custom?level=global" \
  -H "Content-Type: application/json" \
  -d '{"models":{"my-org/custom-llm":{"name":"My Custom LLM","context_length":128000,"pricing":{"prompt":0.000002,"completion":0.00001}}}}'
```

---

## CLI Shortcut

```bash
npm run fetch-models
```

Runs the same fetch script directly, without starting the server.

---

## Internal Usage

- **`context_size_limit` on sessions** — when a new session is created (`POST /sessions` or first chat turn), `createSession` looks up the model in the index and stores its `context_length` as `context_size_limit` on the session row.
- **Cost fallback in the loop** — after each LLM turn, if the API response does not include a `usage.cost` field (or returns `0`), the loop calculates an estimated cost using `pricing.prompt`, `pricing.completion`, and `pricing.cache_read` from the index.