# dsh-persist

[简体中文](README.md) | **English**

> Persistent memory for DeepSeek Harness agents — agents stop forgetting.

## What problem does it solve?

By default, DSH agents **forget everything when you switch conversations** — close the context, and nothing is remembered. dsh-persist turns memory into a layered, filesystem-based structure with **on-demand injection** per conversation:

| Layer | Storage | Analogy | How it's read |
|---|---|---|---|
| User profile + long-term memory | `USER.md` / `MEMORY.md` | LTM (long-term memory) | Optional injection per conversation |
| Key-value memory | `memory.json` | Quick-reference cards | Optional injection; read/write via tool |
| Conversation memory | `sessions/<id>/memory.md` | Working memory | Injected into its own conversation only |
| Project memory | `projects/<key>/memory.md` | PARA Project layer | Opt-in injection per project |
| Vault semantic memory | `vault.md` + `vault.db` | Zettelkasten card box | On-demand semantic recall (tool or the "auto retrieval" tick; never injected verbatim) |

## Features

- **Per-conversation memory** — each conversation gets its own "notebook" that never gets lost; other conversations can't see or inject it
- **Selective injection** — inject only what you tick; new conversations default to zero injection, saving context
- **Shared project memory** — conversations in the same working directory share project experience; named projects split multi-project directories
- **Vault semantic search** — pair with a free key to find memories "by meaning"; falls back to keyword search without a key
- **Automatic recall (three-state mode)** — pick the auto-retrieval mode in the Memory tab: `Off` (default) / `Smart` (pure-local rules filter chatter — only history/project/topic-related messages retrieve; zero network) / `LLM judge` (a model call decides; more accurate but sends the message to the model provider; timeout/failure degrades to Smart)
- **Memory tab UI** — visual editing + injection config + live preview at the top of a conversation (WYSIWYG); the "Open memory admin" link in the tab's top-right corner opens the full management page in a new tab (all sessions/projects/Vault/global files)
- **Plain text everywhere** — every file under `~/.dsh-memory/` is human-readable and editable; easy to back up, migrate, and export

## Quick start

```sh
dsh plugin --profile web add dsh-persist   # 1. install
# 2. restart DSH (dsh --profile web)
# 3. open a conversation → "Memory" tab at the top → tick the blocks to inject
```

Tell the agent "remember this" and it writes into the current conversation's memory automatically.
For semantic search: get a free key from [SiliconFlow](https://siliconflow.cn/models?q=bge-m3), set `SILICONFLOW_API_KEY=...`, and restart (works without it too — falls back to keyword search).

## Installation

> Formerly `dsh-memory` (npm name taken, renamed to `dsh-persist`). Storage path `~/.dsh-memory/` and route `/dsh-memory/` are unchanged — no migration needed.

**Requirements**: Node.js **>= 22.6** (24+ recommended; `--experimental-strip-types` for tests is available since 22.6). The Vault layer uses built-in `node:sqlite`: available by default since 23.4, needs `--experimental-sqlite` on 22.6–23.3; on older versions the plugin still loads but Vault tools are disabled (memory/injection/UI unaffected). The host is DeepSeek Harness (provides `tools` / `systemPrompt` / `webServer` / `sessions` / `agents` services and the `conversation.view` UI slot).

`dsh-persist` is a standard DSH **bundle**: it declares `dsh.bundle` and installs into any profile via `dsh plugin`. `@deepseek-ai/*` are **optional peer dependencies provided by the DSH host at runtime** — npm doesn't (and doesn't need to) install them.

From **npm** (ships with prebuilt `lib/`, no build needed):

```sh
# install into your web profile (recommended)
dsh plugin --profile web add dsh-persist
# or any other profile
dsh plugin --profile demo add dsh-persist
```

Restart (`dsh --profile web`) after installing and the plugin takes effect.

From **GitHub** (pulls source and runs the `prepare` build, see below):

```sh
dsh plugin --profile web add github:tluoluo/dsh-persist
```

> When installing from git, pnpm may ask you to authorize the `prepare` build script on first `add`: copy the package key `dsh` prints into `allowBuilds` in your profile's `pnpm-workspace.yaml`, then re-run `add` (see the official DSH docs [publish.md](https://github.com/deepseek-ai/deepseek-harness/blob/main/docs/user/develop/basic/publish.md)).

Optional environment variables (**all optional — it works right after install**):

- `SILICONFLOW_API_KEY=...` → **optional**, enables "semantic search". bge-m3 embeds memories into vectors and finds them by meaning (smarter than plain keywords). **Works without it too**: Vault memory falls back to keyword search. [Get a free key at SiliconFlow](https://siliconflow.cn/models?q=bge-m3). It's the only optional "external brain" — makes search smarter, not required.
- `DSH_MEMORY_INJECT=0` → disable automatic injection (enabled by default)
- `DSH_MEMORY_ALLOW_REMOTE=1` → allow non-loopback access to the memory API (default: 403 for everything remote; only needed when the DSH web server binds 0.0.0.0 — assess the privacy risk yourself)
- `DSH_MEMORY_AUTO_VAULT=0` → **force off** automatic semantic retrieval for every conversation; `=1` → force on (unset = follow each conversation's "Auto retrieval" mode in the Memory tab, default off. Note: with `=1` and no explicit GATE, the smart gate always applies — the tab's LLM choice is ignored)
- `DSH_MEMORY_AUTO_VAULT_GATE=heuristic|llm|off` → globally override every conversation's auto-retrieval mode (`heuristic` smart gate: pure-local rules filter chatter — zero cost, zero network, zero latency; `llm` calls a model judge (see `DSH_MEMORY_JUDGE_MODEL`), any failure degrades to heuristic; `off` disables the gate — retrieve on every turn). Unset = follow each conversation's Memory-tab selection — note the tab's `Off` means "no retrieval", which differs from this env's `off`: the env value only takes effect when set explicitly. The word lists are heuristic with known edge cases (short messages without trigger words aren't retrieved; long chatter can slip through) — a deliberate trade-off
- `DSH_MEMORY_JUDGE_MODEL=provider/model` → model used by the LLM-judge mode (default `deepseek-official/deepseek-chat`, cheap and fast; format `provider/model`)
- `DSH_MEMORY_JUDGE_TIMEOUT_MS=5000` → LLM-judge timeout (default 5000ms; on timeout the mode degrades to Smart)
- `DSH_MEMORY_AUTO_VAULT_NAMESPACES=user,dsh-persist` → restrict automatic retrieval to these namespaces (default: all, including project archives)

> **Privacy**: with semantic search enabled (`SILICONFLOW_API_KEY`), every user message and memory entry is sent to SiliconFlow for embedding; automatic retrieval does the same. **LLM-judge mode** additionally sends the current message to the model provider configured by `DSH_MEMORY_JUDGE_MODEL` for a "needs retrieval?" decision. If that's a concern, don't configure a key, or set `DSH_MEMORY_AUTO_VAULT=0` to disable automatic retrieval (manual `vault search` still works).

**In one sentence: `dsh plugin add` + restart = memory. Want smarter search? Grab a free SiliconFlow key.**

### First steps after installing

1. **Restart DSH** (`dsh --profile web` — don't reuse the old running process).
2. Open the web UI, enter any conversation — a **"Memory" tab** appears at the top (next to the trace tab). That's your per-conversation memory and injection switches.
3. Tell the agent what to remember — it writes into the conversation memory via the `memory` tool automatically; or edit manually in the Memory tab. By default **nothing is injected** (no context cost) until you tick blocks in the tab.

> Optional: for semantic search, grab a free SiliconFlow key, set `SILICONFLOW_API_KEY=...`, restart, and the Vault layer upgrades from keyword to semantic search.

## Security

`/dsh-memory/api/*` is loopback-only (non-local requests get 403 unless you explicitly set `DSH_MEMORY_ALLOW_REMOTE=1`). Memory contains personal information — don't expose it on shared networks.

## Building from source

**Users don't need to build** — published packages ship with `lib/`, `dsh plugin add` just works.

**External developers**: after cloning, `npm install` runs `prepare` automatically (tsdown transpile-only, no `@deepseek-ai` types needed to produce `lib/`), so the package builds self-containedly:

```bash
npm install        # runs prepare → lib/index.js + lib/client.js
npm run prepare    # explicit rebuild of self-contained artifacts (host: tsdown.host.config.ts, client: tsdown.config.ts)
npm test           # smoke tests (node --experimental-strip-types src/smoke.ts)
```

`prepare` is **transpile-only** (no type-check): it strips `import type` of `@deepseek-ai/*` from source, so the runtime artifact keeps just two host-provided imports (`@deepseek-ai/dsh-tools.defineTool` and `@deepseek-ai/dsh-llm.createUserMessage`) — buildable without host types, loaded by the DSH host.

**Maintainers** (inside a DSH host tree, junctioned to host deps for `@deepseek-ai` types) can run the full build for `.d.ts` + full type-checking:

```bash
npm run build      # typecheck + typecheck:client + build:host + bundle + dts
```

> End-to-end semantic tests live in `src/smoke-semantic.ts` (need a real `SILICONFLOW_API_KEY`) and are **not** part of `npm test` — run manually:
> - PowerShell: `$env:SILICONFLOW_API_KEY=...; node --experimental-strip-types src/smoke-semantic.ts`
> - bash: `SILICONFLOW_API_KEY=... node --experimental-strip-types src/smoke-semantic.ts`

> Note: `lib/` is gitignored; a running DSH needs a restart to load new host code (client bundle just needs a page refresh).

## Usage

### Tool actions (model-called)

The `memory` tool's `scope` parameter decides where things are written/read:

- `scope=conversation` (**default**) → current conversation memory: `add` appends a note, `list`/`search` read it all
- `scope=project` → project memory of the current working directory (shared across conversations in the same directory)
- `scope=global` → global keyed memory: `add`/`get`/`search`/`delete` (by key)

| Action | Args | Effect |
|---|---|---|
| `add` | `scope`, `content` (plus `key` for global) | store a memory |
| `get` / `search` / `delete` | `scope`, `key` | global keyed operations |
| `list` | `scope` | read all conversation/project memory |
| `profile` | `profileKind`, `profileOp`, `content` | read/write global user profile (USER.md/MEMORY.md) |
| `vault` | `vaultOp`, `content/query`, `namespace` | semantic storage: `add` / `search` / `list` / `export` / `import` (namespace-isolated) |

### Memory files (all human-readable/editable)

| File | Content | How to edit |
|---|---|---|
| `~/.dsh-memory/sessions/<id>/memory.md` | this conversation's memory | Memory tab, or `memory add` |
| `~/.dsh-memory/sessions/<id>/inject.json` | per-conversation injection config | tick boxes in the Memory tab |
| All historical sessions | per-conversation memory + inject config | "Session memories" tab on `/dsh-memory/`: browse / edit / delete (the Memory tab only edits the current conversation) |
| `~/.dsh-memory/projects/<key>/memory.md` | project experience | `memory add` (scope=project) or direct edit |
| `~/.dsh-memory/USER.md` / `MEMORY.md` | global pool | `/dsh-memory/` page or direct edit |
| `~/.dsh-memory/memory.json` | global keyed | edit JSON directly |
| `~/.dsh-memory/vault.md` | Vault semantic memory | Tool add/delete (`memory(vaultOp="add"/"delete")`) **auto-syncs this file**; after hand-editing, `memory(vaultOp="import")` (or "Vault sync" in the admin page) writes back to the DB |

### Examples

```
Conversation A (cwd /work/projA):
- Memory tab ticks: user profile + conversation memory + project memory(projA)
- each turn injects those three blocks; conversation B never sees A's memory

agent: user prefers spaces over tabs
agent: memory(action="add", content="user prefers spaces")   # writes to conversation A's memory

agent: user asks "do you remember my indentation preference?" (same conversation)
agent: memory(action="list") → reads conversation A's memory back

agent: record project experience
agent: memory(action="add", scope="project", content="build script is build.ps1")
       # writes to /work/projA's project memory, shareable with other conversations there

agent: global keyed memory (shared across conversations)
agent: memory(action="add", scope="global", key="user-name", content="Xiaoming")
```

## Technical design (maps to the DSH course)

| Part | Implementation | Course module |
|---|---|---|
| Storage | `MemoryStore` / `ProfileStore` / `SessionMemoryStore`, files + atomic writes | module ⑤ option A |
| Vault | `VaultStore` + `embedder` (SQLite + bge-m3) | module ⑤ option C |
| Tools | `ctx.tools.register(defineTool({...}))`, three scopes | modules ③④ |
| Selective injection | `ctx.systemPrompt.context()` renders inject.json per session (sub-agents inherit via parentSession) | module ⑥ |
| Multi-agent isolation | namespace + conversation/project isolation | module ⑦ |
| Client UI | `conversation.view` slot id `memory` order 20 (right of trace), self-contained tsdown bundle | module ⑧ |
| Hybrid retrieval | semantic when vectors exist, keyword otherwise, merged & ranked | module ⑤ |
| Plugin anatomy | `name` / `inject` / `apply` | module ② |

> After code changes you need to **restart DSH** to load new host code (the running process holds the old modules; client bundle just needs a page refresh).

## Troubleshooting

| Symptom | Fix |
|---|---|
| Corrupted memory files (memory.json / inject.json won't parse) | The plugin backs up the broken file to `*.corrupt-<timestamp>` in the same directory and resets to empty; the console prints the backup path — recover from the backup |
| Memory API returns 403 | Loopback guard is active (localhost only by default). Confirm DSH binds 127.0.0.1; for remote access set `DSH_MEMORY_ALLOW_REMOTE=1` (evaluate privacy risk yourself) |
| Want automatic injection fully off | `DSH_MEMORY_INJECT=0`, restart DSH |
| Host code changes don't take effect | The DSH process holds old modules — restart DSH (client bundle just needs a page refresh) |
| Injected content too long | Static blocks (USER/MEMORY/conversation/project) have a 100-line cap; overflow isn't injected — trim the corresponding file on the `/dsh-memory/` page. The Vault has no static injection: it is recalled on demand only (auto retrieval topK=3 or the tool), so no line cap applies |
| Conversation/project memory files growing | Injection truncates at 100 lines, but the on-disk memory.md keeps growing — trim regularly (e.g. per milestone) with the `memory` tool or by editing; move stale entries into the Vault archive |
| Hand-edited vault.md content disappears | Tool add/delete (`memory(vaultOp="add"/"delete")`) **rewrites vault.md wholesale** (auto-synced mirror). Hand-edit during a quiet moment, then immediately `memory(vaultOp="import")` or use the management page's "Vault sync" to write back to the DB |

## Contributing

- **Code layout**: top-level `src/` = host logic + shared pure functions, `src/client/` = browser side; the two artifacts build independently (host via `tsdown.host.config.ts`, client via `tsdown.config.ts`), type declarations (`.d.ts`) generated by tsc
- **Dev flow**: edit → `npm run build` (full: typecheck + host + client + d.ts) → `npm test` all green; for client-only changes `npm run bundle` alone iterates faster
- Tests always use temp directories (as `src/smoke.ts` does) — never touch real `~/.dsh-memory/` data
- Before committing, run `npm pack --dry-run` to check publish contents (`prepack` hook builds automatically)

## Roadmap

- [x] v1 base: `memory` tool + file storage
- [x] Builtin layer: user profile + automatic injection (`context()`)
- [x] Vault layer: vector semantic retrieval (bge-m3, keyword fallback)
- [x] Conversation layer: per-conversation memory + injection config + project (cwd) memory
- [x] Client UI: Memory tab (right of trace) + edit/injection-config panel
- [x] True automatic vector injection: `agent/pre-step` retrieves topK=3 per message (tick "Auto retrieval" in the memory tab; off by default; `DSH_MEMORY_AUTO_VAULT=0/1` forces globally)

## License

MIT