# Model Capability Awareness And Multimodal Identity Memory Root Fix

Date: 2026-05-15

Status: planning and integration tracker

Latest integration pass: 2026-05-15

- Implemented the Telegram/TUI visual identity memory root path first.
- Added `identity_memory` in `packages/cli/src/tui/identity-memory-tool.ts`.
- Wired it into TUI sub-agent tools and Telegram admin/public action tools.
- Repaired CLIP-only zettelkasten neighbor linking in
  `packages/memory/src/zettelkasten.ts`.
- Added regression tests:
  `packages/memory/tests/zettelkasten.test.ts` and
  `packages/cli/tests/identity-memory-tool.test.ts`.

## Purpose

Omnius already has substantial capability metadata for creative models, voice,
image/audio generation, Telegram scoped media, CLIP embeddings, face memory,
and graph-backed multimodal identity. The current weakness is that this data is
not consistently exposed to the model as runtime-aware, inspectable context.

The root fix is to make the agent loop aware of:

- which voice, image, music, and sound models are available;
- which of those models are currently selected for this invocation;
- model details such as backend, size class, quality notes, hardware fit,
  cached/downloaded status, setup/prewarm commands, and runtime limitations;
- which tools are actually available in the current invocation after policy
  filtering;
- how to inspect deeper structures behind those tools without relying on
  brittle keyword heuristics;
- how to write natural multimodal identity memories through model-decided tool
  calls, not regex parsing.

This document is the tracking artifact for that integration.

## Hard Constraints

- Do not solve natural memory requests with regex or keyword aliases.
- The model must decide to call identity/capability tools from full context.
- Tool implementations may validate inputs and evidence, but must not silently
  infer identity without explicit user-provided evidence.
- Public Telegram scope must remain per-chat and policy filtered.
- Private/admin/TUI/global context must not leak into public group context.
- Capability awareness must describe the current invocation, not a stale static
  list.
- Shared services should feed TUI, Telegram, GUI/API, and future voice sessions.

## Existing Anchors

### Creative Model Registries

Image model metadata already exists in:

- `packages/execution/src/tools/image-generate.ts:24`
  - `ImageGenerationPreset`
  - fields: `id`, `label`, `backend`, `install`, `category`, `sizeClass`,
    `quality`, `minVramGB`, `recommendedVramGB`, `deployment`, `steps`,
    `guidance`, `width`, `height`, `fallbackFor`, `note`
- `packages/execution/src/tools/image-generate.ts:59`
  - `DEFAULT_DIFFUSERS_IMAGE_MODEL`
- `packages/execution/src/tools/image-generate.ts:120`
  - `IMAGE_GENERATION_MODEL_PRESETS`
- `packages/execution/src/tools/image-generate.ts:727`
  - `getImageGenerationPreset`
- `packages/execution/src/tools/image-generate.ts:733`
  - `imageGenerationQualityLadder`
- `packages/execution/src/tools/image-generate.ts:739`
  - `imageGenerationFallbackAlternates`
- `packages/execution/src/tools/image-generate.ts:1145`
  - `ImageGenerateTool`
- `packages/execution/src/tools/image-generate.ts:1248`
  - `ImageGenerateTool` supports `action=list_models`, but the schema still
    marks `prompt` as required and output is shallow.

Sound/music model metadata already exists in:

- `packages/execution/src/tools/audio-generate.ts:27`
  - `AudioGenerationPreset`
  - fields: `id`, `label`, `kind`, `backend`, `install`, `category`,
    `sizeClass`, `quality`, `output`, `bestUse`, `minVramGB`,
    `recommendedVramGB`, `deployment`, `defaultDurationSec`, `defaultSteps`,
    `note`
- `packages/execution/src/tools/audio-generate.ts:79`
  - `DEFAULT_SOUND_MODEL`
- `packages/execution/src/tools/audio-generate.ts:80`
  - `DEFAULT_MUSIC_MODEL`
- `packages/execution/src/tools/audio-generate.ts:135`
  - `AUDIO_GENERATION_MODEL_PRESETS`
- `packages/execution/src/tools/audio-generate.ts:1420`
  - `getAudioGenerationPreset`
- `packages/execution/src/tools/audio-generate.ts:1426`
  - `audioGenerationQualityLadder`
- `packages/execution/src/tools/audio-generate.ts:1648`
  - `AudioGenerateTool`
- `packages/execution/src/tools/audio-generate.ts:1790`
  - `AudioGenerateTool` supports `action=list_models`, but the schema does not
    expose `action`.

Voice model metadata exists in:

- `packages/cli/src/tui/voice.ts:171`
  - `VOICE_MODELS`
- `packages/cli/src/tui/voice.ts:247`
  - `listVoiceModels`
- `packages/cli/src/tui/voice.ts:271`
  - `getSupertonicVoiceOptions`
- `packages/cli/src/api/serve.ts:5355`
  - `GET /v1/voice/models`
- `packages/cli/src/api/serve.ts:5359`
  - `GET /v1/voice/supertonic-settings`
- `packages/cli/src/api/serve.ts:5390`
  - `POST /v1/voice/models/switch`
- `packages/execution/src/tools/audio-playback.ts:610`
  - `AudioPlaybackTool`
- `packages/execution/src/tools/audio-playback.ts:797`
  - `audio_playback action=list_voices`
- `packages/execution/src/tools/audio-playback.ts:1096`
  - `TtsGenerateTool`

### Current Model Selections

Settings schema:

- `packages/cli/src/tui/omnius-directory.ts:171`
  - `OmniusSettings`
- `packages/cli/src/tui/omnius-directory.ts:182`
  - `voiceModel`
- `packages/cli/src/tui/omnius-directory.ts:216`
  - `imageModel`
- `packages/cli/src/tui/omnius-directory.ts:218`
  - `imageBackend`
- `packages/cli/src/tui/omnius-directory.ts:220`
  - `soundModel`
- `packages/cli/src/tui/omnius-directory.ts:222`
  - `soundBackend`
- `packages/cli/src/tui/omnius-directory.ts:224`
  - `musicModel`
- `packages/cli/src/tui/omnius-directory.ts:226`
  - `musicBackend`
- `packages/cli/src/tui/omnius-directory.ts:282`
  - `resolveSettings`

TUI configured defaults:

- `packages/cli/src/tui/interactive.ts:982`
  - `imageGenerationDefaultsForRepo`
- `packages/cli/src/tui/interactive.ts:992`
  - `createConfiguredImageGenerateTool`
- `packages/cli/src/tui/interactive.ts:996`
  - `audioGenerationDefaultsForRepo`
- `packages/cli/src/tui/interactive.ts:1010`
  - `createConfiguredAudioGenerateTool`

Telegram configured defaults:

- `packages/cli/src/tui/telegram-bridge.ts:5106`
  - `imageGenerationDefaultsForRepo`
- `packages/cli/src/tui/telegram-bridge.ts:5116`
  - `audioGenerationDefaultsForRepo`
- `packages/cli/src/tui/telegram-bridge.ts:4939`
  - admin Telegram tools instantiate `ImageGenerateTool`
- `packages/cli/src/tui/telegram-bridge.ts:4940`
  - admin Telegram tools instantiate `AudioGenerateTool`
- `packages/cli/src/tui/telegram-bridge.ts:4975`
  - public Telegram appends scoped creative tools

### Hardware Rating And Disk Status

The richer model scoring is trapped inside TUI command rendering:

- `packages/cli/src/tui/commands.ts:8689`
  - `rateImagePresetForHardware`
- `packages/cli/src/tui/commands.ts:8772`
  - `renderImageModelList`
- `packages/cli/src/tui/commands.ts:8876`
  - `fetchOllamaModelSizes`
- `packages/cli/src/tui/commands.ts:8904`
  - `imageModelDiskStats`
- `packages/cli/src/tui/commands.ts:8914`
  - `audioModelDiskStats`
- `packages/cli/src/tui/commands.ts:9193`
  - `rateAudioPresetForHardware`
- `packages/cli/src/tui/commands.ts:9240`
  - `renderAudioModelList`

This logic needs to move into a shared service that can be used by:

- TUI slash commands;
- Telegram context injection;
- GUI/API endpoints;
- agent-facing capability inspection tool.

### Agent Tool Awareness

Static discovery:

- `packages/execution/src/tools/explore-tools.ts:20`
  - static `TOOL_CATALOG`
- `packages/execution/src/tools/explore-tools.ts:82`
  - `ExploreToolsTool`

Runtime-aware discovery:

- `packages/orchestrator/src/agenticRunner.ts:1372`
  - promoted/deferred tool tracking
- `packages/orchestrator/src/agenticRunner.ts:17376`
  - `buildToolDefinitions`
- `packages/orchestrator/src/agenticRunner.ts:17630`
  - `tool_search` pseudo-tool
- `packages/orchestrator/src/agenticRunner.ts:17659`
  - runtime handler for `tool_search`

Post-compaction awareness:

- `packages/orchestrator/src/agenticRunner.ts:15506`
  - post-compaction skill and MCP tool re-injection

Current gap:

- `tool_search` reflects runtime registered tools, but only as deferred tool
  one-liners.
- `ExploreToolsTool` is static and can drift from the actual invocation.
- Neither path exposes selected models, model detail, embedding status, scoped
  media affordances, or deep inspection hooks.

### Invocation Context Injection

TUI dynamic context:

- `packages/cli/src/tui/interactive.ts:2163`
  - project context loaded
- `packages/cli/src/tui/interactive.ts:2165`
  - `dynamicContext`
- `packages/cli/src/tui/interactive.ts:2188`
  - `environmentProvider`
- `packages/cli/src/tui/interactive.ts:2277`
  - self-modify settings loaded
- `packages/cli/src/tui/interactive.ts:2312`
  - vision capability guidance injection
- `packages/cli/src/tui/interactive.ts:2585`
  - passed into `AgenticRunner`

Telegram dynamic context:

- `packages/cli/src/tui/telegram-bridge.ts:4123`
  - quick-chat message construction
- `packages/cli/src/tui/telegram-bridge.ts:4145`
  - runtime context in quick-chat system prompt
- `packages/cli/src/tui/telegram-bridge.ts:4248`
  - action/chat sub-agent session context
- `packages/cli/src/tui/telegram-bridge.ts:4266`
  - `AgenticRunner` creation
- `packages/cli/src/tui/telegram-bridge.ts:4276`
  - Telegram `dynamicContext`
- `packages/cli/src/tui/telegram-bridge.ts:3278`
  - `buildTelegramSessionContext`
- `packages/cli/src/tui/telegram-bridge.ts:3297`
  - Telegram runtime context section
- `packages/cli/src/tui/telegram-bridge.ts:3315`
  - admin group safety/context section
- `packages/cli/src/tui/telegram-bridge.ts:3317`
  - public group safety/context section

Telegram reply/media context anchors:

- `packages/cli/src/tui/telegram-bridge.ts:204`
  - `TelegramReplyContext`
- `packages/cli/src/tui/telegram-bridge.ts:2110`
  - `resolveTelegramReplyContext`
- `packages/cli/src/tui/telegram-bridge.ts:2160`
  - `buildTelegramCurrentReplyContext`
- `packages/cli/src/tui/telegram-bridge.ts:2203`
  - `formatTelegramCurrentMessageForPrompt`
- `packages/cli/src/tui/telegram-bridge.ts:2793`
  - `buildTelegramConversationContextStream`
- `packages/cli/src/tui/telegram-bridge.ts:2853`
  - recent media context section
- `packages/cli/src/tui/telegram-bridge.ts:5048`
  - `buildTelegramMediaRecentTool`
- `packages/cli/src/tui/telegram-bridge.ts:5527`
  - `processMediaContextForMessage`

### Multimodal Memory And Identity

Graph-backed multimodal identity:

- `packages/memory/src/multimodalIdentity.ts:6`
  - source/scope types
- `packages/memory/src/multimodalIdentity.ts:35`
  - media refs
- `packages/memory/src/multimodalIdentity.ts:45`
  - `MultimodalIdentityAssertion`
- `packages/memory/src/multimodalIdentity.ts:53`
  - `MultimodalEmbeddingBundle`
- `packages/memory/src/multimodalIdentity.ts:167`
  - `MultimodalIdentityService`
- `packages/memory/src/multimodalIdentity.ts:178`
  - `ingest`
- `packages/memory/src/multimodalIdentity.ts:269`
  - identity assertion application
- `packages/memory/src/multimodalIdentity.ts:298`
  - `applyIdentityAssertions`
- `packages/memory/src/multimodalIdentity.ts:330`
  - embedding persistence

Knowledge graph relations:

- `packages/memory/src/temporalGraph.ts:24`
  - `KGRelation`
- `packages/memory/src/temporalGraph.ts:27`
  - reply/media identity relations: `authored_by`, `uploaded_by`,
    `replied_to`, `depicts`, `named_as`
- `packages/memory/src/temporalGraph.ts:28`
  - `face_match`, `voice_sample_of`, `speaker_candidate`,
    `same_person_candidate`

Episode store:

- `packages/memory/src/episodeStore.ts:69`
  - `Episode`
- `packages/memory/src/episodeStore.ts:80`
  - native `embedding`
- `packages/memory/src/episodeStore.ts:83`
  - `clipEmbedding`
- `packages/memory/src/episodeStore.ts:337`
  - `clip_embedding` column
- `packages/memory/src/episodeStore.ts:397`
  - automatic zettelkasten linking on insert
- `packages/memory/src/episodeStore.ts:625`
  - `setEmbedding`
- `packages/memory/src/episodeStore.ts:631`
  - `setClipEmbedding`

Zettelkasten:

- `packages/memory/src/zettelkasten.ts:72`
  - `findNeighbors`
- `packages/memory/src/zettelkasten.ts:78`
  - currently returns empty if the source episode has no native embedding
- `packages/memory/src/zettelkasten.ts:90`
  - CLIP fallback only runs when native embedding lengths mismatch
- `packages/memory/src/zettelkasten.ts:114`
  - `linkEpisode`
- `packages/memory/src/zettelkasten.ts:177`
  - `batchLink`

Embedding workers:

- `packages/cli/src/api/embedding-workers.ts:41`
  - `startEmbeddingWorkers`
- `packages/cli/src/api/embedding-workers.ts:64`
  - scans visual/audio episodes missing native embeddings
- `packages/cli/src/api/embedding-workers.ts:107`
  - stores native embedding
- `packages/cli/src/api/embedding-workers.ts:121`
  - creates `EmbeddingAligner`
- `packages/cli/src/api/embedding-workers.ts:143`
  - stores aligned embedding as `clipEmbedding`
- `packages/cli/src/api/embedding-workers.ts:148`
  - coarse KG node insertion currently labels visual episodes as `person`
- `packages/cli/src/api/embedding-workers.ts:165`
  - stale `person_node_id` co-occurrence metadata path

Embedding scripts:

- `scripts/embed-image.py:23`
  - OpenCLIP ViT-B-32 image embedding
- `scripts/embed-text.py:14`
  - OpenCLIP ViT-B-32 text embedding
- `scripts/embed-audio.py:45`
  - SpeechBrain ECAPA speaker embedding
- `packages/cli/src/api/py-embed.ts:16`
  - `ensureEmbedDeps`
- `packages/cli/src/api/py-embed.ts:41`
  - full embedding deps require `OMNIUS_INSTALL_FULL_EMBED_DEPS=1`
- `packages/cli/src/api/py-embed.ts:55`
  - `runEmbedImage`
- `packages/cli/src/api/py-embed.ts:66`
  - `runEmbedAudio`
- `packages/cli/src/api/py-embed.ts:77`
  - `runEmbedText`

Older visual/multimodal memory tools:

- `packages/execution/src/tools/visual-memory.ts:44`
  - `VisualMemoryTool`
- `packages/execution/src/tools/visual-memory.ts:58`
  - actions: `detect`, `enroll`, `identify`, `teach`, `recognize`, `list`,
    `forget`
- `packages/execution/src/tools/visual-memory.ts:161`
  - face enrollment
- `packages/execution/src/tools/multimodal-memory.ts:74`
  - `MultimodalMemoryTool`
- `packages/execution/src/tools/multimodal-memory.ts:89`
  - actions: `capture`, `meet`, `recall`, `timeline`
- `packages/execution/src/tools/multimodal-memory.ts:639`
  - stores visual CLIP into `EpisodeStore.setClipEmbedding`

Current gap:

- The new graph-backed identity path is central, but older face and
  multimodal-memory stores are still separate silos.
- Telegram media ingest writes scoped episodes, but there is no model-facing
  identity tool for explicit user assertions such as "this person is Amy".
- Zettelkasten linking does not fully exploit CLIP-only visual episodes.

### Telegram Memory Ingest

- `packages/cli/src/tui/telegram-bridge.ts:2227`
  - sender normalization
- `packages/cli/src/tui/telegram-bridge.ts:2247`
  - memory scope
- `packages/cli/src/tui/telegram-bridge.ts:2255`
  - reply ref
- `packages/cli/src/tui/telegram-bridge.ts:2272`
  - `telegramMemoryIngestPayload`
- `packages/cli/src/tui/telegram-bridge.ts:5477`
  - image ingest POST to `/v1/memory/ingest`
- `packages/cli/src/tui/telegram-bridge.ts:5506`
  - audio/voice ingest POST to `/v1/memory/ingest`
- `packages/cli/src/api/serve.ts:8324`
  - `/v1/memory/ingest`
- `packages/cli/src/api/serve.ts:8907`
  - `handleMemoryIngest`
- `packages/cli/src/api/serve.ts:8947`
  - identity assertion extraction from API payload
- `packages/cli/src/api/serve.ts:8997`
  - `MultimodalIdentityService`

## Root Fix Architecture

### 1. Shared Runtime Capability Snapshot

Create a shared capability module, likely under one of:

- `packages/execution/src/model-capabilities.ts`
- `packages/cli/src/tui/model-capabilities.ts`

Preferred direction:

- put pure registry/scoring logic in `@omnius/execution` if it does not need
  TUI renderer state;
- keep CLI-only disk probes and settings loaders in CLI helpers if necessary;
- expose plain JSON structures, not formatted TUI lines.

Proposed shape:

```ts
export type CapabilityModelKind = "voice" | "image" | "sound" | "music";

export interface RuntimeModelSelection {
  kind: CapabilityModelKind;
  selectedModel?: string;
  selectedBackend?: string;
  source: "project" | "global" | "default" | "runtime";
}

export interface RuntimeModelFit {
  score?: number;
  label?: string;
  note?: string;
  minVramGB?: number;
  recommendedVramGB?: number;
}

export interface RuntimeModelDiskState {
  downloaded: boolean;
  bytes: number;
  paths: string[];
}

export interface RuntimeModelCapability {
  kind: CapabilityModelKind;
  id: string;
  label: string;
  backend: string;
  category?: string;
  sizeClass?: string;
  quality?: string;
  output?: string;
  bestUse?: string;
  deployment?: string;
  install?: string;
  defaults?: Record<string, unknown>;
  fit?: RuntimeModelFit;
  disk?: RuntimeModelDiskState;
  selected?: boolean;
  notes: string[];
}

export interface RuntimeCapabilitySnapshot {
  generatedAt: string;
  repoRoot: string;
  hardware: {
    totalRamGB: number;
    availableRamGB: number;
    gpuVramGB: number;
    gpuName?: string;
  };
  selections: RuntimeModelSelection[];
  models: RuntimeModelCapability[];
  embedding: RuntimeEmbeddingStatus;
  tools?: RuntimeToolAwareness;
}
```

### 2. Agent-Facing Capability Tool

Add a tool such as `model_capabilities` or `capability_inspect`.

Actions:

- `summary`
  - current selected voice/image/sound/music plus compact fit.
- `list`
  - filter by `kind=voice|image|sound|music|embedding|tools`.
- `inspect`
  - inspect one model/tool/embedding provider in depth.
- `current`
  - exact current selections for this invocation.
- `embedding_status`
  - OpenCLIP, text CLIP, speaker embedding, transcript embedding, installed
    status, vector spaces, counts.
- `tool_context`
  - actual registered tools in this invocation after policy filtering.

This tool must be registered in:

- TUI `buildTools` / sub-agent tool set.
- Telegram admin and public contexts, with public output scoped and redacted.
- GUI/API agent runs.

### 3. Compact Capability Context Injection

Every invocation should receive a compact, non-overwhelming block:

```text
<runtime-capabilities>
Selected models:
- voice: supertonic (backend supertonic)
- image: stabilityai/sdxl-turbo (diffusers, fit 88/100 excellent)
- sound: cvssp/audioldm-s-full-v2 (diffusers, fit 74/100 comfortable)
- music: facebook/musicgen-small (transformers, fit 70/100 comfortable)

Use model_capabilities(action="inspect", kind="image", id="...") for details.
Use model_capabilities(action="list", kind="music") before switching models.
</runtime-capabilities>
```

Injection anchors:

- TUI: `packages/cli/src/tui/interactive.ts:2165`
- TUI environment refresh: `packages/cli/src/tui/interactive.ts:2188`
- Telegram quick chat: `packages/cli/src/tui/telegram-bridge.ts:4149`
- Telegram action context: `packages/cli/src/tui/telegram-bridge.ts:3297`
- API/GUI agent subprocess: agent launch path under `packages/cli/src/api/serve.ts`

The context should remain compact. Deep detail belongs in the tool.

### 4. Runtime Tool Awareness

Improve agent self-awareness with the actual invocation tool set.

Targets:

- keep `tool_search` as the schema promotion mechanism;
- replace or augment static `ExploreToolsTool` with a runtime-provided tool
  index where possible;
- expose policy-filtered tool names/categories in the capability snapshot;
- after compaction, re-inject the compact tool/category summary alongside MCP
  names.

Important anchors:

- `packages/execution/src/tools/explore-tools.ts:20`
- `packages/orchestrator/src/agenticRunner.ts:17376`
- `packages/orchestrator/src/agenticRunner.ts:17630`
- `packages/orchestrator/src/agenticRunner.ts:15506`

### 5. Natural Identity Memory Tool

Add an agent-facing tool, likely `identity_memory`.

Actions:

- `assert_identity`
  - user has explicitly provided identity information.
- `enroll_face`
  - enroll a face embedding using image evidence.
- `identify`
  - identify known faces/voices in supplied media.
- `recall`
  - retrieve identity evidence by name/person/media.
- `inspect`
  - show identity node, evidence, modalities, confidence, scope.

Inputs:

```ts
interface IdentityMemoryArgs {
  action: "assert_identity" | "enroll_face" | "identify" | "recall" | "inspect";
  name?: string;
  relation?: "depicts" | "speaker" | "named_as" | "same_person_candidate";
  media?: "reply" | "latest" | string;
  message_id?: string | number;
  scope?: "current" | "private" | "group" | "global";
  confidence?: number;
  evidence_note?: string;
}
```

Validation rules:

- The model may call this naturally when the user asks to remember a face,
  states a person name, identifies a speaker, or asks who/what is remembered.
- The tool must require explicit `name` for identity assertion writes.
- The tool must resolve media aliases through scoped media providers, not local
  filesystem guessing.
- If there is no usable image/voice evidence, store the textual identity
  assertion but mark it as lacking biometric embedding evidence.
- If a face is present, write through `MultimodalIdentityService` and mirror
  enrollment into `VisualMemoryTool` or a shared face embedding helper.
- In public Telegram, writes are scoped to the current chat.

### 6. CLIP And Associative Memory Repair

Required fixes:

- Allow zettelkasten linking when an episode has `clipEmbedding` but no native
  `embedding`.
- Make `findNeighbors` consider candidates with compatible CLIP embeddings even
  when native embeddings are absent.
- Avoid labeling every visual episode node as `person` in embedding workers.
- Replace stale `person_node_id` co-occurrence assumptions with graph relations
  created by `MultimodalIdentityService`.
- Add status/introspection for embedding providers and vector counts.

Important anchors:

- `packages/memory/src/zettelkasten.ts:78`
- `packages/memory/src/zettelkasten.ts:90`
- `packages/cli/src/api/embedding-workers.ts:148`
- `packages/cli/src/api/embedding-workers.ts:165`
- `packages/memory/src/multimodalIdentity.ts:330`

## Implementation Checklist

### Phase 0 - Tracking Document

- [x] Create root-fix document with anchors and artifacts.
- [x] Update this checklist as each implementation step lands.
- [ ] Add links from related docs if the implementation spans multiple passes.

### Phase 1 - Shared Model Capability Snapshot

- [ ] Extract image hardware fit logic from TUI command rendering into a shared
      pure function.
- [ ] Extract sound/music hardware fit logic from TUI command rendering into a
      shared pure function.
- [ ] Add shared disk/cache probe helpers or a CLI wrapper around shared model
      metadata.
- [ ] Add `buildRuntimeCapabilitySnapshot(repoRoot, options)`.
- [ ] Include current selected voice/image/sound/music settings.
- [ ] Include defaults when no explicit setting is present.
- [ ] Include voice model metadata from `listVoiceModels`.
- [ ] Include image/music/sound model metadata and fit scores.
- [ ] Include downloaded/cache size where available.
- [ ] Add unit tests for snapshot shape and selected/default resolution.

### Phase 2 - Agent-Facing Capability Tool

- [ ] Add `ModelCapabilitiesTool` or `CapabilityInspectTool`.
- [ ] Support `summary`, `current`, `list`, `inspect`, `embedding_status`,
      and `tool_context`.
- [ ] Register the tool in TUI main tool set.
- [ ] Register the tool in TUI sub-agent tool set.
- [ ] Register the tool in Telegram admin DM tools.
- [ ] Register a scoped/redacted version in Telegram public tools.
- [ ] Register the tool for GUI/API agent runs.
- [ ] Add tests for public redaction and full admin output.

### Phase 3 - Capability Context Injection

- [ ] Add compact `<runtime-capabilities>` renderer.
- [ ] Inject into TUI `dynamicContext`.
- [ ] Inject into TUI `environmentProvider` if selections can change mid-run.
- [ ] Inject into Telegram quick-chat system context.
- [ ] Inject into Telegram action sub-agent dynamic context.
- [ ] Inject into GUI/API agent subprocess context.
- [ ] Add post-compaction reinjection for compact capability awareness.
- [ ] Verify context remains compact for small/medium models.

### Phase 4 - Existing Tool Schema Cleanup

- [ ] Fix `ImageGenerateTool` schema so `action=list_models` does not require
      `prompt`.
- [ ] Expand `ImageGenerateTool` `list_models` output or delegate to the new
      capability snapshot.
- [ ] Add `action` to `AudioGenerateTool` schema.
- [ ] Ensure `AudioGenerateTool action=list_models kind=music|sound` works
      without `prompt`.
- [ ] Ensure `generate_tts`/`audio_playback list_voices` points to the same
      voice capability metadata.
- [ ] Update `/help` and README model/tool descriptions after implementation.

### Phase 5 - Natural Identity Memory Tool

- [x] Add `IdentityMemoryTool`.
- [x] Integrate with `MultimodalIdentityService`.
- [x] Resolve scoped media aliases: `reply`, `latest`, and explicit
      Telegram/media paths.
- [x] Validate explicit name/evidence on identity assertion writes.
- [x] Store scoped graph evidence for `depicts`, `named_as`,
      `speaker_candidate`, and `same_person_candidate`.
- [ ] Add voice embedding extraction so explicit speaker assertions can also
      create `voice_sample_of` evidence.
- [x] Mirror face enrollment into visual memory or a shared face embedding
      backend.
- [ ] Support recall/inspect by sender, media, voice, and scope. Person-name
      recall/inspect is implemented.
- [x] Add Telegram public scoped version.
- [x] Add tests for no-regex behavior: tool only writes when called with
      explicit structured args.

Implemented anchors:

- `packages/cli/src/tui/identity-memory-tool.ts`
  - `IdentityMemoryTool`
  - `formatIdentityMemoryContext`
  - media resolver interface for Telegram, TUI, GUI, and future voice/session
    surfaces.
- `packages/cli/src/tui/telegram-bridge.ts`
  - `buildTelegramIdentityMemoryTool`
  - `resolveTelegramIdentityMedia`
  - `formatIdentityMemoryContext` injection into Telegram action context.
- `packages/cli/src/tui/interactive.ts`
  - TUI main-agent and sub-agent registration of `IdentityMemoryTool`
  - compact identity memory contract injection.

### Phase 6 - CLIP/Zettelkasten Repair

- [x] Update `findNeighbors` to handle CLIP-only source episodes.
- [x] Update candidate filtering to include CLIP-compatible candidates.
- [x] Ensure CLIP image/text embeddings use clearly named vector spaces.
- [x] Add regression tests for visual episode with only `clipEmbedding`.
- [ ] Remove or replace stale `person_node_id` coupling in embedding workers.
- [ ] Stop labeling all visual embedding worker nodes as `person`.
- [x] Link identity assertions to media/message/person nodes through the graph.

### Phase 7 - GUI/API And Documentation

- [ ] Add `/v1/capabilities` or `/v1/model-capabilities`.
- [ ] Add `/v1/capabilities/current`.
- [ ] Add `/v1/capabilities/models?kind=image|sound|music|voice`.
- [ ] Add GUI display for selected/current models using the same endpoint.
- [ ] Add README section for model capability awareness.
- [ ] Add `/help` entries for the new capability and identity memory tools.
- [ ] Update existing docs that mention embedding auto-installs to reflect
      `OMNIUS_INSTALL_FULL_EMBED_DEPS=1`.

### Phase 8 - Verification

- [ ] `pnpm -r build`
- [x] targeted memory package tests
- [x] targeted CLI identity tool tests
- [x] targeted CLI and memory builds
- [ ] Telegram public scoped smoke test
- [ ] Telegram admin DM smoke test
- [ ] TUI smoke test for capability summary
- [ ] GUI/API smoke test for capability endpoint
- [ ] Manual test: ask "what image model are you using?"
- [ ] Manual test: ask "list available music models and pick the best fit"
- [ ] Manual test: reply to an image with "this is <name>" and confirm graph,
      visual memory, and recall evidence

## Expected End State

The agent should be able to answer, without guessing:

- "What voice model are you using right now?"
- "Which image models can you use, and which fits this GPU best?"
- "List music models by quality and size."
- "Can you inspect your sound model setup?"
- "Do you remember this face?"
- "This person is Amy. Remember that."
- "Who is in this image based on prior memory?"

For those questions, the model should either:

- answer from compact runtime capability context;
- call the capability inspection tool for details;
- call the identity memory tool with structured evidence;
- ask a clarifying question if the identity evidence is ambiguous.

No part of this should depend on regex phrase triggers.