# Provider Platform

This doc describes `videoclaw`'s provider/transport architecture as of
2026-05-29, after the Phase 1c schema upgrade, the Phase 5b Runway port,
and the Dreamina (Seedance 2.0 via useapi.net) route.

## Routes at a glance

| Route | Maturity | Native transport | Built-in adapter | Required env |
|---|---|---|---|---|
| `veo-useapi` | production | `native-veo.ts` (drives the local `vclaw-cli` Bun package) | `vclaw-provider-adapter --route veo-useapi` | `USEAPI_API_TOKEN`, `USEAPI_ACCOUNT_EMAIL` |
| `seedance-direct` | production | `native-seedance.ts` (uses `SUTUI_API_KEY`) | `vclaw-provider-adapter --route seedance-direct` | `SUTUI_API_KEY` |
| `runway-useapi` | production | `native-runway.ts` (pure Node fetch+fs) | `vclaw-provider-adapter --route runway-useapi` | `USEAPI_API_TOKEN`, `USEAPI_ACCOUNT_EMAIL` |
| `dreamina-useapi` | production | `native-dreamina.ts` (pure Node fetch+fs) | `vclaw-provider-adapter --route dreamina-useapi` | `USEAPI_API_TOKEN`, `VCLAW_DREAMINA_ACCOUNT` (optional `VCLAW_DREAMINA_REGION` default `CA`, `VCLAW_DREAMINA_MODEL` default `seedance-2.0`) |
| `magnific-rest` | production | `native-magnific.ts` (pure Node fetch, Magnific/Freepik REST) | `vclaw-provider-adapter --route magnific-rest` | `MAGNIFIC_API_KEY` (optional `VCLAW_MAGNIFIC_API_URL`, `VCLAW_MAGNIFIC_MODEL`) |

`magnific-rest` is an **aggregator** route — it proxies a catalog of video models (Seedance Pro, Kling, Hailuo, WAN, Runway, LTX, PixVerse, OmniHuman, VFX) through the Magnific/Freepik REST API and is cost-aware (defaults to the cheapest catalog model that fits the operation; premium models — incl. Seedance Pro 1080p — are opt-in via `VCLAW_MAGNIFIC_MODEL`). The same `native-magnific.ts` client also backs `vclaw video image-ops` (still-image upscale) and `finish --backend magnific-precision`.

`dreamina-useapi` reuses the **same** `USEAPI_API_TOKEN` as `runway-useapi`
(no new token). Like Runway, Seedance content moderation rejects real human
faces — describe stylized/illustrated characters or seed from a generated
start frame.

## Descriptor schema

`src/video/provider-platform/registry.ts` defines `DEFAULT_PROVIDER_REGISTRY`
as an array of `VideoProviderDescriptor` (from `./types.ts`). Each route
descriptor has:

```typescript
interface VideoProviderDescriptor {
  id: ProviderRouteId;                     // 'veo-useapi' | 'seedance-direct' | ...
  provider: VideoProvider;                 // 'veo' | 'runway' | 'seedance'
  displayName: string;
  path: ProviderPath;                      // 'direct' | 'useapi'
  summary: string;
  controls: ProviderControl[];             // 'audio', 'first-frame', 'last-frame',
                                           // 'reference-images', 'camera-grammar', ...
  operationSupport: Array<{
    operation: VideoOperationKind;         // 'text-to-video', 'image-to-video', ...
    aspectRatios: NormalizedAspectRatio[];  // 'landscape' | 'portrait'
    notes?: string[];                      // per-operation gotchas
    maxReferenceImages?: number;
  }>;
  routingHints: {
    latencyClass: 'low' | 'medium' | 'high';
    costClass: 'free' | 'paid' | 'premium';
    trustClass: 'direct' | 'aggregated';
    preferredWorkflows: VideoWorkflowKind[];
  };
  escapeHatches?: Array<{
    name: string;
    description: string;
    options: Array<{ name: string; description: string }>;
  }>;
  notes?: string[];                        // free-form per-route notes
}
```

This rich schema came from videoclaw during the Phase 1c merge. It replaced
the flat `supportedOperations[]` shape that `vclaw-video-core` had, and
lets the router make capability-aware decisions per operation × aspect ratio
rather than per route as a whole.

## Routing

`src/video/provider-platform/router.ts` exposes
`chooseVideoProviderRoute(request, policy)`. Given a routing request with
operation kind + aspect ratio + capability requirements, it filters routes
that satisfy the operation × aspectRatio combination, ranks the remainders
by the policy's preference (`trust-first`, `capability-first`, or
`balanced`), and returns a `VideoProviderRouteDecision` with the chosen
route + rationale.

Routes marked `scaffold` in `provider-status.ts:ROUTE_MATURITY` are
labeled `availability: 'degraded'` in the status report and the router
will skip them under default policy unless explicitly requested.

## Adapter contract

Live execution goes through `src/video/execution-runtime.ts`, which calls
`resolveAdapterCommand(routeId, env)`:

1. Look for the user override env var (`VCLAW_<ROUTE>_ADAPTER`). If set,
   that command runs as the adapter — `vclaw` invokes it with JSON on
   stdin and expects JSON on stdout.
2. Otherwise check `builtinAdapterCommandForRoute(routeId)`. Currently
   returns a built-in adapter command for `seedance-direct`, `veo-useapi`,
   `runway-useapi`, and `dreamina-useapi` (the bundled
   `dist/cli/provider-adapter.js` binary invoked with `--route <id>`).
3. Otherwise throw — the route doesn't have a usable adapter. (All four
   live routes ship a built-in adapter; the removed `veo-direct` route was
   the lone exception and no longer exists.)

The adapter protocol:

| Stage | Input (stdin) | Output (stdout) |
|---|---|---|
| submit | `{ scenes: [{ sceneIndex, prompt, ... }], outputDir, ... }` | `{ externalJobId, rawResult: {...} }` |
| poll | `{ action: 'poll', outputDir, externalJobId }` | `{ status: 'pending' \| 'completed' \| 'failed', outputs?: [...], issues?: [...] }` |
| cancel | `{ action: 'cancel', outputDir, externalJobId, workspaceRoot }` | `{ status: 'cancelled' \| 'unsupported', externalJobId?, issues?: [...] }` |

The runtime is strict about the returned `status`: `pollExecutionPayload`
throws unless poll status is exactly `pending` / `completed` / `failed`, and
`cancelExecutionPayload` throws unless cancel status is exactly `cancelled` /
`unsupported`. The cancel result is read into `VideoExecutionCancelResult`
`{ status, externalJobId, issues, rawResult }` — note the field is `issues`,
not `warnings`.

The built-in adapter (`src/cli/provider-adapter.ts`) also honors per-route
command shims for each of its four routes — `*_SUBMIT_CMD`, `*_POLL_CMD`,
`*_CANCEL_CMD` (e.g. `VCLAW_DREAMINA_USEAPI_SUBMIT_CMD` /
`VCLAW_DREAMINA_USEAPI_POLL_CMD` / `VCLAW_DREAMINA_USEAPI_CANCEL_CMD`). The
action is chosen from `input.action`: `poll` → the poll shim, `cancel` → the
cancel shim, anything else → the submit shim. Setting any shim (or the full
`*_ADAPTER` override) marks the route as an execution override, which
suppresses the runtime dependency probes for that route in the status report.

## Native in-process transports

Four routes have native TypeScript transports that bypass the adapter
subprocess hop:

- **`src/video/native-veo.ts`** — defaults to `<workspace>/vclaw-cli/flow.ts`
  via `bun`. Looks for `cookie.json` for Google Labs Flow auth.
  Customizable via `VCLAW_VEO_CLI_ROOT`, `VCLAW_VEO_OUTPUT_DIR`,
  `VCLAW_VEO_BUN_BIN`, `VCLAW_VEO_COMMAND_TIMEOUT_MS`. Forwards optional
  omni-flash fields to `flow.ts` when present (byte-identical when absent):
  `executionProfile.veoModel` → `-m` (`omni-flash` unlocks audio/V2V),
  per-scene `voicePreset` → `--voice` (omni-flash-only, gated in
  `route-capabilities.ts`), allowlisted `durationSeconds` (4/6/8/10) →
  `--duration`, and per-scene `referenceVideoMediaId` → `--ref-video`
  (omni-flash-only V2V edit, distinct from the scene-chaining seed).

- **`src/video/native-seedance.ts`** — direct Seedance API calls via
  `SUTUI_API_KEY`. No subprocess hop, no external CLI dependency.

- **`src/video/native-runway.ts`** — direct UseAPI REST calls via
  `USEAPI_API_TOKEN` + `USEAPI_ACCOUNT_EMAIL`. Pure Node `fetch` + `fs`.
  Supports both Gen-4.x (firstImageAssetId for i2v) and Seedance-2.0
  (startFrameAssetId for keyframe-driven) modes via the unified
  `/runwayml/videos/create` endpoint. Cancel marks scenes failed locally
  and warns about remote tasks UseAPI free-tier cannot cancel server-side.

- **`src/video/native-dreamina.ts`** — direct UseAPI Dreamina REST calls via
  `USEAPI_API_TOKEN` + `VCLAW_DREAMINA_ACCOUNT`. Pure Node `fetch` + `fs`.
  Reference routing: `referenceRole === 'character'`, OR more than one image,
  OR any video/audio reference → Omni Reference
  (`omni_N_imageRef`/`videoRef`/`audioRef`, multi-character lock); exactly one
  image keyframe → `firstFrameRef` (first_frame image-to-video mode); else
  text-to-video with the requested ratio. References are capped at 9 image /
  3 video / 3 audio (`assertDreaminaReferenceBudget`), preflighted across ALL
  tasks fail-fast before any upload. `Asset://` URIs are skipped with a warning
  (those are ARK avatars, not Dreamina assetRefs). It reuses
  `seedance-content-filter`: on a content-violation submit error it retries
  with a level-1 then level-2 sanitized prompt. Cancel has no UseAPI verb, so
  it marks local scenes failed, warns that the remote job keeps running and
  consuming credits, and returns `status: 'cancelled'`.

The pure-fetch native transports (`native-runway`, `native-dreamina`, `native-seedance`, `native-magnific`) accept an optional `fetchImpl` parameter for test injection. `native-veo` is the exception — it drives the `vclaw-cli` Bun subprocess rather than HTTP fetch, so it has no `fetchImpl`.
The provider-level adapter functions in `src/video/providers/runway-useapi.ts`
and `src/video/providers/dreamina-useapi.ts` also accept `fetchImpl`, so tests
can mock the entire HTTP layer end-to-end.

## Adding a new route

When you want to add a working new provider route (for example a
future `kling-useapi`):

1. **Descriptor.** Add a `VideoProviderDescriptor` entry to
   `DEFAULT_PROVIDER_REGISTRY` in `src/video/provider-platform/registry.ts`.
   Use videoclaw's rich schema (controls, operationSupport, routingHints,
   escapeHatches).
2. **Provider HTTP code.** Add `src/video/providers/<route>.ts` with the
   submit/poll/fetchResult functions. Accept `fetchImpl?: FetchLike` in
   each input interface for test injection.
3. **Native transport.** Add `src/video/native-<route>.ts` mirroring
   `native-runway.ts`: own workspace/env/job-state, call into providers/
   for HTTP. Accept `fetchImpl?: FetchLike` in the options.
4. **Wire the dispatcher.** In `src/video/execution-runtime.ts`, add a
   case for the new route in `builtinAdapterCommandForRoute()` and
   `adapterEnvVarForRoute()`. In
   `src/video/provider-adapter-runner.ts`, route the submit/poll/cancel
   payloads to your native module.
5. **Status.** In `src/video/provider-status.ts`, update
   `ROUTE_MATURITY[<route>]` to `'production'`,
   `ROUTE_REQUIRED_ENV_VARS[<route>]`, `ROUTE_REQUIRED_DEPENDENCIES[<route>]`,
   and `ROUTE_ADAPTER_ENV_VAR[<route>]`.
6. **Tests.** Add `src/tests/<route>.test.ts` (provider) and
   `src/tests/native-<route>.test.ts` (native wrapper with scripted
   fetch mock). The 4 runway tests are the reference pattern.
7. **Docs.** Update the route table at the top of this file.

`dreamina-useapi` is the most recent worked example — its
`src/video/providers/dreamina-useapi.ts` + `src/video/native-dreamina.ts`
landed after Runway and are the freshest reference for this checklist.
`runway-useapi` (Phase 5b, `6e99443`) remains a good canonical reference but
is no longer the newest. Read those files for the canonical structure.