# Telegram Reflection Corpus Integration Plan

Date: 2026-05-15

Status: planning and implementation tracker

Goal: make Telegram reflection cycles operate over a durable scoped corpus, not just the retained in-memory chat window. Each cycle must efficiently perform tagging, summation, titling, extraction, and linking over Telegram-scoped conversation data, then commit the resulting artifacts back into memory and graph stores so future reflection, live replies, and multimodal identity recall can use them.

## Hard Requirements

- No implementation stubs, placeholder functions, dead exports, or unused planning-only branches.
- No hidden fallback to workspace filesystem dream mode from Telegram chat contexts.
- Every artifact must be scoped by Telegram session key, chat id, chat type, and message anchors.
- Public group reflection must stay private unless a later model-gated outreach pass chooses a safe same-group reply.
- Private DM outreach remains disabled unless an explicit consent and delivery policy is added.
- Graph traversal must start from scoped Telegram corpus candidates, not repo files.
- Vector search must be used first when embeddings are available; lexical/recent fallback is allowed only when embedding generation is unavailable and must be recorded in artifact metadata.
- Candidate selection must prefer inner graph nodes with meaningful degree and evidence, then randomly choose one candidate from the eligible set for reflection.
- The graph walk must expand from the selected node across valid scoped edges until depth, candidate, and token budgets are met.
- The model extraction pass must return structured data for all five required operations: tagging, summation, titling, extraction, and linking.
- All committed memory must preserve source message ids and reply relationships.

## Current State

Telegram already has a scoped reflection command path:

- `packages/cli/src/tui/telegram-bridge.ts:2152` has `handleTelegramReflectionSlash`.
- `packages/cli/src/tui/telegram-bridge.ts:2710` has `runTelegramChannelDmnForSession`.
- `packages/cli/src/tui/telegram-bridge.ts:2746` has `runTelegramChannelDmnSweep`.
- `packages/cli/src/tui/telegram-bridge.ts:2767` has `maybeSendTelegramReflectionFollowup`.
- `packages/cli/src/tui/telegram-bridge.ts:2877` injects the latest reflection context into live routing.

The existing artifact builder is still mostly a retained-history summarizer:

- `packages/cli/src/tui/telegram-channel-dmn.ts:44` defines `TelegramChannelDaydreamInput`.
- `packages/cli/src/tui/telegram-channel-dmn.ts:57` defines `TelegramChannelDaydreamArtifact`.
- `packages/cli/src/tui/telegram-channel-dmn.ts:516` builds the artifact.
- `packages/cli/src/tui/telegram-channel-dmn.ts:579` formats markdown.
- `packages/cli/src/tui/telegram-channel-dmn.ts:641` writes JSON and markdown.
- `packages/cli/src/tui/telegram-channel-dmn.ts:666` formats injected runtime context.

Telegram conversation retention exists but does not yet commit text turns into the shared graph/vector corpus:

- `packages/cli/src/tui/telegram-bridge.ts:2539` computes scoped conversation file paths.
- `packages/cli/src/tui/telegram-bridge.ts:2556` loads persisted chat history.
- `packages/cli/src/tui/telegram-bridge.ts:2623` saves chat history, participants, memory cards, stimulation, and reflection state.
- `packages/cli/src/tui/telegram-bridge.ts:2901` records user messages in scoped history.
- `packages/cli/src/tui/telegram-bridge.ts:2949` records assistant messages in scoped history.
- `packages/cli/src/tui/telegram-bridge.ts:3216` updates JSON-only memory cards.
- `packages/cli/src/tui/telegram-bridge.ts:3322` builds the live Telegram context stream.

Shared memory primitives already exist:

- `packages/memory/src/episodeStore.ts:66` includes `social`, `text`, and `reflection` modalities.
- `packages/memory/src/episodeStore.ts:355` inserts episodes and auto-links them when a graph is attached.
- `packages/memory/src/episodeStore.ts:412` performs lexical plus vector search.
- `packages/memory/src/episodeStore.ts:545` merges PPR graph retrieval with normal search.
- `packages/memory/src/episodeStore.ts:625` stores native embeddings.
- `packages/memory/src/episodeStore.ts:631` stores CLIP-compatible embeddings.
- `packages/memory/src/temporalGraph.ts:24` includes Telegram-relevant relations such as `contains`, `authored_by`, `said_by`, `replied_to`, `depicts`, `named_as`, `voice_sample_of`, and `same_person_candidate`.
- `packages/memory/src/temporalGraph.ts:121` creates graph nodes.
- `packages/memory/src/temporalGraph.ts:186` creates graph edges.
- `packages/memory/src/temporalGraph.ts:253` returns valid one-hop neighbors.
- `packages/memory/src/zettelkasten.ts:94` finds vector/CLIP neighbors.
- `packages/memory/src/zettelkasten.ts:127` links related episodes into the graph.
- `packages/memory/src/pprRetrieval.ts:161` runs personalized PageRank.
- `packages/memory/src/pprRetrieval.ts:252` maps PPR graph scores back to episodes.
- `packages/memory/src/multimodalIdentity.ts:178` ingests multimodal evidence into episodes and graph nodes.

## Target Architecture

### Corpus Layer

Add `packages/cli/src/tui/telegram-reflection-corpus.ts`.

Responsibilities:

- Convert scoped Telegram history entries into durable `EpisodeStore` episodes.
- Use `MultimodalIdentityService` for message, sender, reply, and media graph atoms where the message has Telegram evidence.
- Generate text embeddings for new or missing scoped Telegram text/reflection episodes when an embedding backend is configured.
- Add graph nodes and edges that make text-only Telegram turns traversable even without media:
  - scope node contains message node
  - message node authored_by sender node
  - assistant/user message node replied_to target message node when present
  - message node said_by sender node for natural-language content
  - message node related_to derived tag/title/summary nodes after extraction
- Expose a single integrated function for the reflection runner:
  - `buildTelegramReflectionCorpus(options): Promise<TelegramReflectionCorpusResult>`

Required result shape:

- `stats`: counts for retained messages, inserted episodes, reused episodes, embedded episodes, candidate episodes, graph nodes, graph edges, selected nodes, walked episodes.
- `seed`: selected episode id, node id, node text, selection method, vector search limit used, graph depth used.
- `walk`: ordered nodes, edges, episodes, source message ids, reply message ids.
- `fallbacks`: explicit list of degraded paths such as embedding unavailable or no inner graph candidates found.

### Graph/Vector Selection Layer

Add `packages/memory/src/graphWalk.ts` and export it from `packages/memory/src/index.ts`.

Responsibilities:

- Accept a list of vector/lexical episode candidates plus a `TemporalGraph`.
- Resolve candidate episodes to graph nodes through current edges carrying `sourceEpisodeId`.
- Score inner node candidates by:
  - valid degree;
  - distinct source episode count;
  - relation diversity;
  - scoped Telegram metadata match;
  - recency of source evidence.
- Expand candidate search limits in order: 48, 96, 192, 384, 500 until at least one eligible inner node exists or the corpus is exhausted.
- Randomly choose one eligible inner node using a deterministic seeded RNG derived from session key plus artifact run id.
- Walk outward breadth-first from the selected node across valid graph edges.
- Return the selected seed, visited nodes, traversed edges, source episode ids, and degree/evidence diagnostics.

Exact memory anchors:

- `packages/memory/src/temporalGraph.ts:253` currently provides only one-hop neighbors; `graphWalk.ts` will compose this without changing graph storage.
- `packages/memory/src/pprRetrieval.ts:161` remains available for PPR scoring, but this reflection cycle needs explicit node candidate selection, seeded random choice, and bounded walk diagnostics.
- `packages/memory/src/index.ts:74` currently exports PPR helpers; add graph walk exports adjacent to this section.

### Extraction Layer

Add `packages/cli/src/tui/telegram-reflection-extraction.ts`.

Responsibilities:

- Build a compact corpus packet from the graph walk and source episodes.
- Ask the configured backend for strict JSON output.
- Validate and normalize the parsed structure without regex-based semantic decisions.
- Return all five required operation groups:
  - `tagging`: scoped tags with confidence, source message ids, and target node/episode ids.
  - `summation`: concise channel/thread/user/message-window summaries with evidence anchors.
  - `titling`: artifact title, thread titles, memory card titles, and graph node labels.
  - `extraction`: facts, preferences, open questions, decisions, identity assertion candidates, media references, and follow-up opportunities.
  - `linking`: proposed graph links between messages, users, topics, media, summaries, and prior memory cards.
- Never claim a face, voice, or person identity unless the source corpus contains explicit identity evidence or the `identity_memory` tool later verifies it.

Required schema:

- `artifact_title: string`
- `tags: Array<{ label, kind, confidence, source_message_ids, target_episode_ids, target_node_ids }>`
- `summaries: Array<{ title, scope, text, confidence, source_message_ids, target_episode_ids }>`
- `extractions: Array<{ kind, text, confidence, source_message_ids, target_episode_ids, target_node_ids }>`
- `links: Array<{ relation, src_node_text, dst_node_text, confidence, fact, source_message_ids, target_episode_ids }>`
- `followups: Array<{ target, text, reply_to_message_id, rationale, confidence }>`

### Artifact Layer

Extend `packages/cli/src/tui/telegram-channel-dmn.ts`.

Required changes:

- Bump artifact `version` from `2` to `3`.
- Add `corpus`, `selectedSeed`, `graphWalk`, `tagging`, `summation`, `titling`, `extraction`, and `linking` fields to `TelegramChannelDaydreamArtifact`.
- Keep existing fields only when they are either directly useful to live routing or backed by corpus extraction.
- Update `buildTelegramChannelDaydream` to accept an optional corpus/extraction result and merge it into the artifact.
- Update markdown formatting at `packages/cli/src/tui/telegram-channel-dmn.ts:579` to show source anchors, graph seed, walk stats, tags, summaries, titles, extractions, and links.
- Update injected context at `packages/cli/src/tui/telegram-channel-dmn.ts:666` to prioritize:
  - selected seed;
  - graph walk summary;
  - top tags;
  - strongest summaries;
  - unresolved extraction items;
  - reply-linked source message ids.

### Telegram Bridge Wiring

Modify `packages/cli/src/tui/telegram-bridge.ts`.

Required changes:

- At `recordTelegramUserMessage` (`packages/cli/src/tui/telegram-bridge.ts:2901`), enqueue or immediately upsert the text turn into the scoped reflection corpus after local history is recorded.
- At `recordTelegramAssistantMessage` (`packages/cli/src/tui/telegram-bridge.ts:2949`), upsert assistant outputs into the same corpus with assistant sender metadata and reply anchors.
- At `runTelegramChannelDmnForSession` (`packages/cli/src/tui/telegram-bridge.ts:2710`), build the corpus first, run graph/vector selection, run extraction, then build and write the artifact.
- At `maybeSendTelegramReflectionFollowup` (`packages/cli/src/tui/telegram-bridge.ts:2767`), source `candidateMessageIds` from artifact extraction/linking/source anchors, not only curiosity threads and memory proposals.
- At `handleTelegramReflectionSlash`, expose status lines for corpus episode count, selected seed, walk size, tags, summaries, extraction count, and link count.
- Preserve current `/reflect`, `/reflection`, `/daydream`, and Telegram-scoped `/dream` behavior; do not route these into TUI DreamEngine.

### Persistence And Indexing

Use existing shared stores:

- `episodes.db` for text, social, and reflection episodes.
- `knowledge.db` for graph nodes and edges.
- `.omnius/telegram-daydreams/<session-hash>/` for JSON and markdown reflection artifacts.
- `.omnius/telegram-conversations/<session-hash>.json` remains the lightweight retained context cache.

Required data model conventions:

- Episode `sessionId` is the Telegram `sessionKey`.
- Episode `metadata.sourceSurface` is `telegram`.
- Episode `metadata.scope` contains `{ kind: "group" | "private", id: chatId, title }`.
- Episode `metadata.telegram` contains chat id, chat type, chat title, message id, thread id, sender id, username, display name, reply target, and media summary.
- Graph scope node text matches `scope:telegram:<chatType>:<chatId>`.
- Graph message node text uses a stable Telegram message key.
- Summary/tag/title/extraction nodes must carry `sourceEpisodeId` on their creating edges.

### Zettelkasten And Multimodal Memory Integration

Required changes:

- Text Telegram episodes with embeddings must use existing zettelkasten linking through `EpisodeStore.insert` plus explicit post-embedding link refresh.
- CLIP-only visual/audio episodes already link through `packages/memory/src/zettelkasten.ts:94`; reflection cycles must include related visual/audio episodes when graph walk reaches media nodes.
- `packages/memory/src/multimodalIdentity.ts:178` remains the central ingest service for media and explicit identity evidence.
- Reflection extraction can propose identity assertion candidates, but must not commit `named_as`, `depicts`, or `voice_sample_of` unless the underlying message already contains explicit user-supplied identity evidence or a later tool call verifies it.

### Efficient Small-Context Handling

The reflection cycle must not dump entire chat history into the model.

Budgeting rules:

- Vector/lexical search candidate budget starts at 48 and expands only if no inner graph candidates are found.
- Graph walk depth defaults to 2, expands to 3 only if fewer than 8 source episodes are recovered.
- Corpus packet passed to extraction includes compact episode snippets, source anchors, and graph relation facts, not raw full logs.
- Summaries and tags are committed as artifacts so future cycles can read compressed memories instead of rereading old turns.
- Live Telegram context injection displays only top-scoring structured outputs and source anchors.

## Implementation Checklist

### Planning

- [x] Create this tracking document before code changes.
- [x] Confirm all existing reflection/dream tests still represent Telegram-scoped behavior.

### Shared Graph Walk

- [x] Add `packages/memory/src/graphWalk.ts`.
- [x] Export graph walk helpers from `packages/memory/src/index.ts`.
- [x] Add tests for vector candidate expansion, inner node selection, seeded random choice, scoped filtering, and bounded graph walking.

### Telegram Corpus

- [x] Add `packages/cli/src/tui/telegram-reflection-corpus.ts`.
- [x] Upsert scoped Telegram user messages into `EpisodeStore`.
- [x] Upsert scoped Telegram assistant messages into `EpisodeStore`.
- [x] Preserve reply relationships in graph edges.
- [x] Generate or fill text embeddings when configured.
- [x] Refresh zettelkasten links after embeddings are stored.
- [x] Include scoped visual/audio/media graph neighbors in walks.

### Extraction

- [x] Add `packages/cli/src/tui/telegram-reflection-extraction.ts`.
- [x] Build strict JSON extraction prompt with source anchors.
- [x] Validate tagging, summation, titling, extraction, and linking groups.
- [x] Commit summary/tag/title/extraction episodes and graph nodes.
- [x] Add tests for malformed JSON, empty corpus, embedding-disabled fallback, and complete five-operation output.

### Artifact

- [x] Bump Telegram daydream artifact to version 3.
- [x] Add corpus, seed, walk, tags, summaries, titles, extractions, and links to the artifact.
- [x] Update markdown output to make source anchors readable.
- [x] Update runtime context injection to use structured reflection data.

### Bridge

- [x] Wire corpus upsert into `recordTelegramUserMessage`.
- [x] Wire corpus upsert into `recordTelegramAssistantMessage`.
- [x] Wire corpus build, graph/vector seed selection, graph walk, extraction, and artifact commit into `runTelegramChannelDmnForSession`.
- [x] Update reflection follow-up candidate message id selection to use extraction/link anchors.
- [x] Update `/reflect status` output with corpus and graph metrics.

### Validation

- [x] Run `pnpm --filter @omnius/memory test`.
- [x] Run `pnpm --filter omnius test -- tests/telegram-bot-api-10.test.ts`.
- [x] Add and run focused tests for the new corpus/extraction modules.
- [x] Run `pnpm --filter @omnius/memory build`.
- [x] Run `pnpm --filter omnius build`.
- [x] Run `git diff --check`.
- [x] Backward-pass each checklist item against the final diff before commit.

## Backward-Pass Verification

- `packages/memory/src/graphWalk.ts` implements inner-node selection from episode-backed graph evidence, seeded random choice, and bounded graph walking.
- `packages/memory/src/temporalGraph.ts` now exposes current edges by source episode id so vector candidates can be mapped into graph neighborhoods.
- `packages/memory/src/multimodalIdentity.ts` accepts `sessionId` and caller metadata so Telegram text/media evidence lands in the scoped episode corpus.
- `packages/cli/src/tui/telegram-reflection-corpus.ts` upserts Telegram user and assistant turns, fills embeddings when configured, refreshes zettelkasten links, expands candidate limits, and selects graph walks.
- `packages/cli/src/tui/telegram-reflection-extraction.ts` builds strict JSON prompts, validates all five operation groups, and commits reflection/gist episodes plus graph nodes and links.
- `packages/cli/src/tui/telegram-channel-dmn.ts` writes version 3 artifacts with corpus, selected seed, graph walk, tags, summaries, titles, extractions, links, and follow-up candidates.
- `packages/cli/src/tui/telegram-bridge.ts` records live Telegram turns into the corpus and routes `/reflect` through corpus build, graph walk, model extraction, artifact writing, and model-gated follow-up anchoring.
- Regression coverage:
  - `packages/memory/tests/graphWalk.test.ts`
  - `packages/cli/tests/telegram-reflection-corpus.test.ts`
  - `packages/cli/tests/telegram-reflection-extraction.test.ts`
  - `packages/cli/tests/telegram-bot-api-10.test.ts`

## Completion Definition

This integration is complete only when a Telegram `/reflect` run can:

1. Read the scoped chat corpus.
2. Persist new text turns into `episodes.db`.
3. Preserve sender, message, reply, scope, and media relationships in `knowledge.db`.
4. Search scoped episodes with embeddings where available.
5. Expand candidate search depth when there are not enough graph candidates.
6. Select an inner graph node from eligible vector-search candidates.
7. Walk the graph outward from that node within depth and token budgets.
8. Produce structured tagging, summation, titling, extraction, and linking data.
9. Write JSON and markdown artifacts with source message anchors.
10. Commit reflection outputs back into episodes and graph links.
11. Inject compact structured context into future Telegram agent turns.
12. Let the model-gated follow-up path use the same source anchors when deciding whether to reply.