# Media Generation

Omnius exposes media generation through TUI commands, tools, Telegram creative workflows, and sponsor media endpoints.

## Commands And Tools

```text
/image
/video
/sound
/music
/voice
/listen
/call
```

Tool names include:

- `generate_image`
- `generate_video`
- `generate_audio`
- `generate_tts`

## Video

Video generation supports setup/list/prewarm/delete flows, thumbnails, sidecars, audio muxing, broker preflight, and model presets such as SANA/Wan paths where installed.

## Backend Matrix

| Modality | Typical Backends | Hardware Notes | Output |
| --- | --- | --- | --- |
| Image | diffusers, SD.cpp, ComfyUI, Ollama-compatible image routes where available | CPU works slowly; CUDA/ROCm/Metal preferred for larger diffusion models | PNG/JPEG plus prompt sidecar |
| Video | diffusers video pipelines, ComfyUI, SANA/Wan-style presets | high VRAM pressure; preflight broker should reject unsafe loads | MP4/WebM plus thumbnail and sidecar |
| Sound | AudioCraft, Stable Audio, TangoFlux-style pipelines | GPU preferred for longer clips; duration caps protect providers | WAV/MP3 plus prompt sidecar |
| Music | transformers, AudioCraft, Stable Audio, diffusion audio | long generations should be queued and capped | WAV/MP3 plus prompt sidecar |
| Voice | LuxTTS, Kokoro, Supertonic, ASR backends | CPU can handle small TTS/ASR; clone models need setup validation | WAV/PCM/transcript |

## Setup And Preflight

Each modality should expose:

- `setup` for dependency guidance and install triage
- `list` for known models and hardware fit
- `prewarm` where model load time is large
- broker checks for RAM, VRAM, disk, and existing model pressure
- a sidecar file containing prompt, model, seed when available, duration/steps, and source chat/session

When a backend install needs elevation, the TUI should suspend, expose the terminal password prompt directly, then restore TUI state after the installer exits.

## Hugging Face Model Intake

Omnius can intake Hugging Face media models into a global catalog at `~/.omnius/media-models`.
Built-in presets remain the trusted defaults; catalog entries are merged into model lists when they map to a compatible runner.

```text
/models hf search <query> [--modality image|video|sound|music|tts|asr|voice] [--limit N]
/models hf intake <namespace/model> [--modality ...] [--backend ...] [--no-test]
/models hf list [--modality ...]
/models hf validate <namespace/model>
/models hf delete <namespace/model>
```

Agent tools:

- `hf_model_discover`
- `hf_model_intake`

The intake flow reads HF API metadata, model cards, sibling sizes, and `model_index.json` when present. Specs are saved only after schema validation. Runtime activation is limited to models that match a known safe runner or have a generated adapter recorded in the catalog; large downloads still go through disk/resource gates and setup/prewarm paths.

## Audio And Voice

Voice surfaces include:

- TTS synthesis
- ASR transcription
- voice clone references
- realtime voicechat WebSocket
- `/v1/audio/speech`
- `/v1/audio/transcriptions`

## Sponsor Media

Sponsors can expose selected media modalities to consumers. Provider-side controls should enforce:

- enabled modality list
- model allowlist
- max image steps
- max video/audio duration
- daily request limits
- output storage and download paths

Consumers receive generated artifacts under `.omnius/remote-media` where remote media download is enabled.

## Sponsor Media Contract

Remote media requests should carry modality, model, prompt, safety options, max duration or steps, requested format, and caller peer ID. Provider responses should include artifact metadata and a download handle, not arbitrary provider filesystem paths.

| Provider Control | Why It Exists |
| --- | --- |
| modality allowlist | prevents accidental exposure of expensive backends |
| model allowlist | prevents hidden/private models from being advertised |
| max steps/duration | bounds GPU time |
| output retention | prevents unbounded disk growth |
| per-peer daily jobs | prevents one peer from monopolizing media capacity |

## Telegram Media

Telegram public creative workflows use chat-scoped directories and return generated artifacts to the originating chat without exposing arbitrary paths.