# Session Diary: LLM Training Alignment Analysis

## Question

Is the session diary format what LLMs are trained on, such that it helps system handling?

## Answer: Partially

### What LLMs ARE Trained On (Matches)

| Session Diary Feature | LLM Training Data Match |
|-----------------------|-------------------------|
| Markdown structure | ✓ Common in docs, READMEs, wikis |
| Timestamp headers | ✓ Git commits, changelogs |
| Task descriptions | ✓ GitHub issues, Stack Overflow |
| File lists | ✓ PR descriptions, commit messages |
| Summary fields | ✓ Documentation patterns |

### What LLMs Are NOT Trained On (Gaps)

| Session Diary Feature | Why It's Novel |
|-----------------------|----------------|
| `toolCalls: number` | LLMs don't see tool call counts in training |
| `toolsUsed: string[]` | Tool names are model-specific, not universal |
| `completed: boolean` | Binary status is unusual in natural text |
| `source: "task_complete"` | Internal state tracking, not user-facing |
| `_dedupeHash` | Implementation detail, not semantic |

## Current Schema (from `omnius-directory.ts:537-564`)

```typescript
interface SessionContextEntry {
  savedAt: string;        // ISO timestamp
  task: string;           // What task was being done
  summary: string;        // Agent's summary
  filesModified: string[]; // Files touched
  toolCalls: number;      // Tool calls completed
  toolsUsed?: string[];   // Tool names used
  provenance?: string;    // Path to provenance file
  assistantResponse?: string; // Visible reply
  source?: "task_complete" | "manual" | "api";
  completed: boolean;     // Success status
  model: string;          // Model used
  sessionId?: string;     // Link related entries
  _dedupeHash?: string;   // Fast deduplication
}
```

## What Would Make It More LLM-Friendly

### 1. Add Explicit Outcome Field

Current: `completed: boolean` (binary)
Better: `outcome: "success" | "partial" | "failed" | "blocked"`

LLMs understand these natural language outcomes better than booleans.

### 2. Add Decision Log

```typescript
decisions?: Array<{
  decision: string;    // What was decided
  reason: string;      // Why
  alternatives?: string[]; // What was rejected
}>
```

This captures reasoning that LLMs can learn from.

### 3. Add Blockers Field

```typescript
blockers?: Array<{
  type: "dependency" | "error" | "ambiguity" | "resource";
  description: string;
  resolution?: string;
}>
```

LLMs benefit from seeing what blocked progress and how it was resolved.

### 4. Compress Summaries

Current: 280 char free-form text
Better: Structured one-liner

```typescript
summary: {
  outcome: string;      // One sentence result
  keyChange: string;    // Most important change
  nextStep?: string;    // What's left to do
}
```

### 5. Add Token Efficiency Metrics

```typescript
metrics?: {
  inputTokens: number;
  outputTokens: number;
  toolCallTokens: number;
  contextWindowUsed: number; // percentage
}
```

This helps LLMs understand their own efficiency patterns.

## Proposed Enhanced Schema

```typescript
interface SessionContextEntryV2 {
  // Core (unchanged)
  savedAt: string;
  task: string;
  model: string;
  
  // Enhanced outcome
  outcome: "success" | "partial" | "failed" | "blocked";
  summary: {
    result: string;      // ≤100 chars
    keyChange: string;   // ≤80 chars
    nextStep?: string;   // ≤80 chars
  };
  
  // Files (unchanged)
  filesModified: string[];
  
  // Tools (enhanced)
  tools: {
    names: string[];     // Tool names used
    calls: number;       // Total calls
    failed?: number;     // Failed calls
    retries?: number;    // Retry count
  };
  
  // NEW: Decision log
  decisions?: Array<{
    at: string;          // When decided
    choice: string;      // What was chosen
    reason: string;      // Why
    rejected?: string[]; // Alternatives not taken
  }>;
  
  // NEW: Blockers
  blockers?: Array<{
    type: "dependency" | "error" | "ambiguity" | "resource";
    what: string;        // What blocked
    resolved: boolean;   // Was it resolved?
    how?: string;        // How (if resolved)
  }>;
  
  // NEW: Efficiency metrics
  metrics?: {
    inputTokens: number;
    outputTokens: number;
    contextWindowPct: number;
    durationMs: number;
  };
  
  // Session linkage (unchanged)
  sessionId?: string;
  source?: "task_complete" | "manual" | "api";
}
```

## Rendered Example (Current vs Proposed)

### Current Format

```markdown
## 2026-04-22 07:44 ✓ what can this help us to do plantUML first, dont reinvent here

- **Prompt:** what can this help us to do plantUML first, dont reinvent here
- **Model:** zai-org/GLM-5-Turbo
- **Tools:** 5 calls
- **Summary:** PlantUML provides 40+ diagram types that can visualize CRL: (1) Math notation for logic symbols (⊤⊥⊢⊨∀∃), (2) Class/Object diagrams for concept nodes...
```

### Proposed Format (More LLM-parseable)

```markdown
## 2026-04-22 07:44 ✓ plantUML for CRL visualization

- **Task:** what can this help us to do plantUML first, dont reinvent here
- **Outcome:** success
- **Result:** PlantUML can visualize CRL with 40+ diagram types
- **KeyChange:** Identified math notation + class diagrams for concept nodes
- **Tools:** web_search, web_fetch, task_complete (5 calls)
- **Model:** zai-org/GLM-5-Turbo
- **Duration:** 55s, 5 turns
```

## Implementation Path

1. **Backward Compatible**: Add new fields as optional, keep old fields
2. **Gradual Migration**: New sessions use V2 format, old sessions stay V1
3. **Renderer Update**: `renderSessionDiaryV2()` with structured output
4. **Context Injection**: Use V2 format in system prompt context

## Research Basis

- **Chain-of-Thought Prompting** (Wei et al., 2022): Structured reasoning improves LLM performance
- **Self-Consistency** (Wang et al., 2022): Multiple reasoning paths help accuracy
- **Constitutional AI** (Anthropic, 2022): Explicit principles guide behavior
- **Tool Learning** (Qin et al., 2023): Tool-use patterns benefit from structured logging

## Conclusion

The session diary format is **partially aligned** with LLM training data. The markdown structure and task descriptions match what LLMs see. However, the internal state tracking (tool counts, dedupe hashes) is novel.

**Key improvements for LLM-friendliness:**
1. Replace `completed: boolean` with `outcome: "success" | "partial" | "failed" | "blocked"`
2. Add structured `decisions` log
3. Add `blockers` field
4. Compress summaries to structured one-liners
5. Add efficiency metrics

These changes would make the session diary more parseable by LLMs and more useful for cross-session learning.