# dsh-tongflow

**TongFlow as a [DeepSeek Harness](https://github.com/deepseek-ai/deepseek-harness) (dsh) plugin — a media studio inside your agent.**

Three layers, never mixed up:

| Layer | Owns |
|---|---|
| **dsh** | the harness: sessions, model routing, tools, jobs, web UI |
| **the agent** | creativity: the plan, the folder structure, briefs, scripts, prompts, review notes — plain files |
| **TongFlow** | deterministic generation: every image / voice / music / video / 3D asset is produced by **running a saved workflow file** (`<name>.tongflow.json`) through the TongFlow engine and its plugins |

There is deliberately no "generate an image" tool and no project template. The agent studies what the user wants to make (with web research when useful), proposes a folder structure, writes it down, and then — for every asset — creates a workflow file where that asset belongs, runs it, reviews the result, and builds the next stage on it. Users open the same `.tongflow.json` on the embedded canvas, tweak it, and re-run; they can also reorganize the folders by hand at any time.

## Install

```sh
npx @deepseek-ai/dsh@next plugin --profile web add dsh-tongflow      # from npm
# or from a tarball:  pnpm --filter dsh-tongflow pack  →  dsh plugin --profile web add ./dsh-tongflow-x.y.z.tgz
npx @deepseek-ai/dsh@next web
```

Requirements: dsh ≥ 0.1.2-rc.1 — the 0.1.2-rc, 0.1.3-alpha and 0.1.5 lines are supported; 0.1.1-rc and older are not (the client `Runtime` package they build on was removed upstream, so upgrade dsh rather than pinning an old plugin). Because npm only matches a prerelease when a range clause carries its exact `major.minor.patch`, each new dsh prerelease line needs its own clause in `peerDependencies` — a host on a newer line reports a compatibility warning until then. Also Node ≥ 22.19, **Python ≥ 3.10** on `PATH` (or `pythonPath` in the plugin config), `git`, and `ffmpeg` for video contact sheets. On first use the plugin creates `~/.dsh/tongflow/venv` with the `tongflow` SDK and shallow-clones every official TongFlow plugin into `~/.dsh/tongflow/plugins` (the live list from `config/official-plugins.json`; set `autoInstallOfficial: false` to install by hand), so the canvas offers the same node/plugin catalog as the hosted app. Keys and Modal deploys are only needed when something runs.

Start a session whose **first message begins with `@tongflow`** — that session becomes a studio session: the conversation view turns into the Studio (chat column · the project's folder tree · preview / editor / canvas · a runs drawer, all in the UI language of your browser), and the agent gets the `tongflow_*` tools and skill. Any other session is untouched dsh. In the Studio: create a project (a title and a brief — what you want to make), install TongFlow plugins and paste API keys under **Plugins & keys**, then talk to the agent — or click any file to preview / edit it, click a workflow to open it on the canvas.

### Chat model

Any model dsh can route works. For the agent to *see* generated images (`tongflow_look`) use a vision-capable route, e.g. in `$DSH_HOME/settings.yaml`:

```yaml
llm-pi-ai:
  providers:
    google:
      apiKeyEnv: GEMINI_API_KEY
    my-qwen:                          # a self-hosted Qwen3.8-27B behind vLLM
      apiKeyEnv: QWEN_API_KEY
      api: openai-completions
      baseURL: http://127.0.0.1:8000/v1
      models:
        - id: Qwen/Qwen3.8-27B
          input: [text, image]
```

Video and audio are reviewed through TongFlow's own describe / transcribe slots (`tongflow_perceive`), so they work with any chat model.

## The project (a plain folder)

```
~/.dsh/tongflow/projects/<id>/
  project.json                     title, brief, locale — the only fixed file
  README.md                        the agent's plan: what the folders are, in what order things get made
  …whatever the work needs…        e.g. characters/, ep01/sh010/, music/, export/ — designed per project
```

**The one rule:** every AI-generated asset comes from a workflow file that sits next to its outputs.

```
characters/mei/
  mei.md                    what the agent wrote about her
  mei_ref.tongflow.json     the workflow that renders her reference sheet
  mei_ref.01.png            run 1
  mei_ref.02.png            run 2  (a run never overwrites — fix the workflow, run again)
  mei_ref.runs.json         provenance of every run: inputs, plugins, note, timing
```

- Multi-output runs keep the workflow's output names: `mei_ref.03.image.png` + `mei_ref.03.caption.txt`; text outputs are written as `.txt` too.
- Workflows reference project files by **path**: `./mei_ref.02.png` / `../style/palette.png` (relative to the workflow file) or `characters/mei/mei_ref.02.png` (relative to the project root); URLs pass through.
- Text files can be **included** in prompts: `{{../style.md}} {{./mei.md}} full-body sheet` — expanded at run time, so a shared style note is written once.
- **Compose**: `tongflow_workflow_compose({ folder })` merges the small workflows of a folder (or an explicit list) into one `<folder>_all.tongflow.json` — a data node that references another part's output file (`./ref.01.png`) becomes an edge from that part's producing node, parts are ordered by those dependencies, every stage stays an output labelled after its part (`shot_all.01.i2v.mp4` via `meta.outputLabels`), the parts are untouched.
- The Studio tree nests a workflow's outputs under it; the user may rename / move / delete anything by hand and **upload files** (header button → the selected folder, or drag & drop onto a folder view; default `uploads/`) — the agent re-reads the tree (`tongflow_project_status`) before acting.

### Billing checkpoint

A run that uses a paid plugin spends the user's money — a paid API key, or GPU seconds on their Modal account (a Modal plugin also deploys on first use). So `tongflow_workflow_run` without `user_confirmed: true` does not run: it returns `needs_confirmation` with the plugins involved, how each is billed (`api` / `modal`), whether its API keys are set, the models it offers and installed alternatives. The agent puts that to the user and calls again with `user_confirmed` only after an explicit yes — **for every paid run**; nothing is remembered. Runs that use only local plugins are free and start directly. The Studio's own **Run** drawer shows the same notice and a **Confirm & run** button.

### Workflows follow TongFlow's grammar

`tongflow_node_catalog` opens with the node grammar — `add/` widgets (canvas only), `modality/` data nodes, and the four executable categories `transfer/` (1 → 1), `compose/` (N → 1), `decompose/` (1 → N), `batch/` (N → 1) — then lists every node type by category with its ABI slot, wires (`batch` / `collect` flags), config fields, outputs and installed plugins, all read from the ABI registry. The patch tool (`apply_graph_patch` from the `tongflow` package) validates each step against the same registry, so a workflow the agent saves is one the exporter and the canvas accept. The category table lives in [`src/engine/node-categories.ts`](src/engine/node-categories.ts) and a test keeps it in step with `packages/tongflow/src/canvas/node-types.tsx`.

## Agent tools

`tongflow_project_create / _open / _list / _status` · `tongflow_workflow_new / _patch / _read / _list / _validate / _run / _compose` · `tongflow_node_catalog / _describe` · `tongflow_look` (images / video contact sheets, returned as an image block — or described through a slot when the session's model takes no images) · `tongflow_perceive` (video/audio/image understanding via TongFlow slots; billing plugins need `user_confirmed`) · `tongflow_plugins_list / _install / _uninstall` · `tongflow_run_status`. Folder structure and text files are made with dsh's ordinary file tools. Long runs go through dsh background jobs (`run_in_background`).

Skill shipped: `tongflow-studio` (the working method: research → propose a structure → one workflow per asset next to its outputs → run → review → next stage), with four method references under [`skills/references/`](skills/references/) that the agent loads only when the step needs them:

| Reference | Read before |
|---|---|
| `prompt-layers.md` | writing any non-trivial prompt — the seven layers, and what belongs in the prompt text vs. node config vs. a wired reference file |
| `shot-contract.md` | a video shot — open/close state, beat timeline, camera start-path-end, audio, continuity across shots |
| `failure-codes.md` | a result came back wrong — locate the responsible layer, make the smallest fix |
| `iteration.md` | running the same asset again — one variable at a time, and when to stop rewriting the prompt |

Genre knowledge is not packaged; the agent researches or the user installs a skill of their own.

## HTTP (same origin as dsh)

`/tongflow/projects`, `/tongflow/p/:pid/{tree,status,workflows,workflow[/summary|/outputs|/describe|/patch],runs,files/*}`, `/tongflow/runs/:id[/events|/cancel]`, `/tongflow/plugins`, `/tongflow/env`, `/tongflow/health`, plus the canvas-compat API under `/tongflow/p/:pid/api/*` that `tongflow/canvas` talks to.

## Configuration (cordis row `tongflow`)

| key | default | |
|---|---|---|
| `studioRoot` | `<DSH_HOME>/tongflow` | projects, venv, plugins, data |
| `pythonPath` | auto-detect | Python ≥ 3.10 used to create the venv |
| `sdkSpec` | `tongflow==0.3.0` | pip spec installed into the venv (`-e /path/to/sdk` for development) |
| `pluginOrg` | `https://github.com/tong-io` | where official plugins are cloned from |
| `pluginGitUrls` | `{}` | plugin id → git URL overrides |
| `env` | `{}` | environment for plugin processes (API keys); the Studio's key store (`env.json`) is merged over it |
| `maxConcurrentRuns` | `2` | |
| `httpPrefix` | `/tongflow` | |
| `locale` | `en` | canvas UI locale (`en` / `zh` / `ja` / `ko`) |
| `autoInstallOfficial` | `true` | at start, shallow-clone every official plugin that is missing (a few hundred KB each) so the canvas offers the full catalog; API keys / Modal deploys are only needed when a workflow runs |

## Development

```sh
pnpm install
pnpm --filter dsh-tongflow build          # host lib/index.js + browser lib/client.js
pnpm --filter dsh-tongflow test
npx @deepseek-ai/dsh@next plugin --profile web add ./packages/dsh-tongflow   # link: install for hacking
```

The browser half is a single CJS bundle in dsh's `window.__ModuleLoader__` shape: only dsh's platform modules (react, cordis, slot kits) stay external; `tongflow/canvas`, @xyflow/react, zustand and use-intl are inlined (and deduplicated so React contexts match). See [`docs/design.md`](docs/design.md).

License: AGPL-3.0-only (same as TongFlow).