# fff-subconscious-pi

A background memory layer for the [pi](https://pi.dev) coding agent. Before
each LLM call it greps your local text corpora with
[fff](https://pypi.org/project/fff-search/), a fast local search engine,
makes one cheap `deepseek-v4-flash` call to synthesize the hits into a short
note, and injects that note into the prompt as a `<fff_subconscious>` block.
The agent gets a reminder of what you worked on before, without being asked.

## Install

```bash
pi install npm:fff-subconscious-pi
```

or straight from the repo:

```bash
pi install git:github.com/esinecan/fff-subconscious-pi
```

Requirements: `python3` >= 3.10 on PATH and `DEEPSEEK_API_KEY` in the
environment. On the first few context events after install, the extension
bootstraps its own python venv in the background (`.venv/` inside the
package directory, with `fff-search` and `openai`, takes about a minute) and
injects nothing until the `.venv/.ready` sentinel appears. From then on it
is live. Unix-only for now, since the pipeline script uses `fcntl` and
`/tmp`.

## Point it at your own corpora

`config.json` in the package directory decides what gets searched:

- `fff.transcripts_root` defaults to `~/.claude/projects`, which is where
  the Claude Code CLI keeps past session transcripts. If you use Claude
  Code, this works as is. If not, point it at any directory of text you
  want recalled.
- `fff.canon_roots` ships pointing at the author's private notes
  directories, which do not exist on your machine. Replace the entries with
  your own documentation or notes directories, or delete them.

A configured root that does not exist is named in `/tmp/subconscious.log`
on every run and skipped, so a fresh install runs on whatever corpora it
can find rather than failing.

## What happens per prompt

`pi.on("context", handler)` fires before every LLM call with the full
message history. The handler:

1. Guards: venv ready, mode, minimum messages, minimum input length, dedup.
2. Serializes the conversation into a temp transcript file.
3. Runs the vendored `observe.py` with the package venv's python. That
   script extracts search terms from the recent conversation, greps every
   configured corpus with fff, and makes one `deepseek-v4-flash` call to
   turn the hits into a short note, or `NO_UPDATE` when they are not worth
   injecting. The model may also escalate once, asking for a deep read of
   one promising file or a widened search, at the cost of one more flash
   call.
4. Reads the result from `cache/` and injects it into the last user
   message as `<fff_subconscious>...</fff_subconscious>`.

A typical run takes about six seconds. Tool-call continuation turns
re-inject the cached note without re-running the pipeline, and different
user messages are spaced out by a cooldown. Any failure injects nothing.

## Design notes

The pipeline script comes from a Claude Code hook setup by the same author
and is vendored verbatim, so this package is self-contained. Claude Code
runs it across two hooks: generation happens after each assistant reply,
injection happens on the next prompt, and the injected note therefore
always trails the conversation by one turn. Pi has no such split, only a
single `context` event before each LLM call, so this extension runs the
pipeline synchronously and injects a note built from the very prompt being
answered. No lag, and a hard subprocess timeout
(`FFF_SUBCONSCIOUS_TIMEOUT_MS`, default 60s) guarantees the LLM call never
blocks on it.

One consequence of firing before the LLM call: on the first turn of a
fresh session the transcript holds only the user's message, so the
vendored config sets `min_messages` to 1. The note can then arrive on the
opening question instead of waiting for turn two.

## Configuration (env vars)

| Variable | Default | Purpose |
|---|---|---|
| `FFF_SUBCONSCIOUS_MODE` | `every` | `every`, `first`, or `off`. |
| `FFF_SUBCONSCIOUS_DEBUG` | `0` | `1` for stderr logs prefixed `[fff-sub]`. |
| `FFF_SUBCONSCIOUS_COOLDOWN_MS` | `120000` | Min gap between pipeline runs across user messages. |
| `FFF_SUBCONSCIOUS_MIN_MESSAGES` | `1` | Skip if fewer messages. |
| `FFF_SUBCONSCIOUS_MIN_CHARS` | `12` | Skip injection for inputs shorter than this. |
| `FFF_SUBCONSCIOUS_MAX_CHARS` | `4000` | Hard cap on injected note length. |
| `FFF_SUBCONSCIOUS_TIMEOUT_MS` | `60000` | Hard cap on the pipeline subprocess. |
| `DEEPSEEK_API_KEY` | (required) | DeepSeek key for the flash calls. |

Tuning of the retrieval itself (corpora roots, max search terms, escalation
budgets) lives in `config.json`.

## Cache and logs

Notes are cached in `cache/` inside the package directory, one file per
working directory, keyed by `sha256(cwd)`. Temp transcripts are cleaned up
after each run. Delete the cache directory to force fresh queries. The
pipeline logs to `/tmp/subconscious.log`, with a structured version in
`/tmp/subconscious-diag.log`.