---
name: comfyui
description: >
  ComfyUI as a GPU runtime: known-good base images per provider, install and
  version pinning, model weights placement, the HTTP API (queue prompt, poll
  history), and health checks. Use whenever a deployment's runtime is comfyui —
  the default for video models — or when installing custom nodes or weights.
---

# ComfyUI Runtime

ComfyUI is the workhorse runtime for open video models (Wan, HunyuanVideo,
H3 workflows). Treat it as a service: install once, talk HTTP afterwards.

## Base images (known-good, prefer these over hand-building)

- **RunPod:** template `comfyui` (official, preloaded). RunPod also has
  `runpod/stable-diffusion:comfyui-...` images on Docker Hub.
- **Vast.ai:** search offers with the image you want baked in
  (`ai-dock/comfyui` images are solid: `aicompanion` tag family). Otherwise
  `vastai create instance <bundle> --image nvidia/cuda:12.4.1-cudnn-devel-ubuntu22.04` and install ComfyUI by hand (below).
- **Modal:** run ComfyUI in a Modal app via `modal.com` container image
  (`modal` pip package, serve on port 8188, `modal deploy`). Serverless — no
  manual provisioning.

If installing by hand on a CUDA image:

```bash
git clone https://github.com/comfyanonymous/ComfyUI /opt/ComfyUI
cd /opt/ComfyUI && pip install -r requirements.txt
# torch must match the image's CUDA — on 12.x images:
pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu124
python main.py --listen 0.0.0.0 --port 8188 --disable-auto-launch
```

## Weights placement

ComfyUI loads weights from `/opt/ComfyUI/models/` subdirectories. Video models:

- Diffusion/checkpoint → `models/checkpoints/` (Wan, HunyuanVideo full)
- fp8/split variants → `models/diffusion_models/` (Wan fp8, Hunyuan fp8)
- VAE → `models/vae/`
- Text encoders (CLIP, T5, LLM) → `models/text_encoders/`
- Custom node model files → the node's own folder (e.g. `ComfyUI-WanVideoWrapper` may expect specific paths)

Verify weights with their published sha256 before trusting a download — corrupt
weights fail late and waste a retry cycle. Use `huggingface-cli download <repo>`
or `wget` with `--checksum` where possible.

## Custom nodes (the #1 failure source)

Video models need wrapper nodes. Known-good pins (as of this writing — re-verify at install):

- Wan: `ComfyUI-WanVideoWrapper` (kijai) — needs `comfyui_wan_video` deps
- HunyuanVideo: `ComfyUI-HunyuanVideoWrapper` (kijai)
- H3: check the model repo / community nodes at deploy time — this moves fast

Pin every custom node to a commit that works with the pinned ComfyUI version,
and record it in the deployment notes. `git -C <node> log --oneline -1` gives
you the pin. When a node breaks after an update, the fix is almost always:
revert to the previous known-good commit, not "upgrade everything".

## The HTTP API

ComfyUI exposes a JSON API on `:8188`:

1. **Queue a prompt:**
   ```bash
   curl -s -X POST http://<host>:8188/prompt -H "Content-Type: application/json" -d @workflow.json
   # workflow.json: {"prompt": <workflow_graph>, "client_id": "video-factory"}
   # → {"prompt_id": "..."}
   ```
2. **Poll the job:**
   ```bash
   curl -s http://<host>:8188/history/<prompt_id>
   # → status.outputs[].images[] / gifs[] / videos[] with filename
   ```
3. **Health check:** `curl -s http://<host>:8188/system_stats` → 200 with
   `system.device` reporting the GPU. Run this during health-check before
   registering a deployment — a registered deployment must respond.
4. **Outputs** land in `/opt/ComfyUI/output/` on the instance — always sync
   them off (see gpu-ops: retrieve artifacts) before shutdown.

## Workflow graph conventions

A workflow graph is a JSON dict of node id → `{class_type, inputs}`. For video
generation the chain is usually: `CheckpointLoaderSimple` (or `UNETLoader` +
`CLIPLoader` + `VAELoader`) → sampler nodes (e.g. `KSampler`, Wan-specific
samplers) → `VAEDecode` → video output node (e.g. `VHS_VideoCombine` for
video helper suite). Load the saved workflow JSON from the model's repo or
community, adapt the input node names to what's actually installed (check
`/object_info` on the instance: `curl -s http://<host>:8188/object_info` lists
every available node type — use it to validate a workflow before queueing).

Register any graph you get working as a workflow via `gpu_workflow_add` so
`gpu_run` can target it by id, and bump `workflow_version` when you change it —
the ledger then answers "which videos came from workflow v3".