# voice-guard training/eval dataset

Build-time only. **Never bundled into the public npm tarball** (same posture as
`../index/positives.json`: carries real customer and employee names). The publish
workflow must assert this whole `dataset/` dir is absent from the staged tarball,
exactly as it already does for `positives.json` / `negatives.json`.

## Layout

```
dataset/
  raw/*.jsonl     # hand/MCP-gathered + synthesized records, one object per line
  out/            # generated by build_eval.py: train.jsonl, eval.jsonl, report.txt
```

`out/` is derived and reproducible; only `raw/` is source of truth.

## Record schema

```json
{
  "text": "the prose chunk",
  "label": "positive | negative",
  "surface": "slack_dm | jira_standalone | thread_reply | longform",
  "topic": "on | off",
  "source": "slack | jira | granola | confluence | synth",
  "generator": "human | claude | gpt | gemini | ...",
  "mimicry": false
}
```

- **positive** must be `generator: human` and `mimicry: false` (no synthetic positives;
  "Forging the Forger" shows synthetic-author augmentation is net-negative).
- **topic** `on` = core Postman CS customer-engagement work (the subject the curated
  positives cluster on). `topic` `off` = operator's real voice on a different subject
  (internal IT, infra, CI/CD plumbing, tooling procurement). The on/off split is what
  makes the `off_topic_operator` quadrant able to catch a model that learned
  "talks about Postman CS = in voice."
- **mimicry** `true` = a negative the model was few-shot prompted to imitate the operator.
  These are the premium near-misses ("competent but not him").

## Quadrants (all four must be non-empty)

| quadrant | label | topic | role |
|----------|-------|-------|------|
| in_voice_on_topic  | positive | on  | easy positive |
| off_topic_operator | positive | off | catches topic-leakage |
| on_topic_slop      | negative | on  | the near-miss that matters |
| off_topic_slop     | negative | off | easy negative |

## Sourcing

- **Positives**: mined from the operator's real Slack/Jira/Granola/Confluence via MCP,
  and seeded from the already-curated `../index/positives.json`. Slack markup
  (`<@U..|handle>`, `<url|text>`) is reduced to plain readable text; pure mention/emoji
  messages are dropped.
- **Negatives**: synthesized from the assistant families that are the actual runtime
  threat (Claude first, since that is the assistant in the loop; GPT/Gemini are a noted
  follow-on needing other API keys for multi-generator robustness). Generated in the
  operator's real surfaces/registers, on the same topics, at natural register, including
  operator-mimicry near-misses. This makes the negative distribution *be* the threat
  distribution.

## This is a representative proving slice, not the full corpus

The first `raw/` set proves the harness end to end with all four quadrants populated and
generator diversity. Scaling to thousands of mined positives and multi-generator negatives
is the follow-on data run; the schema and `build_eval.py` do not change.