# CapsLockX – Claude Instructions

## Project Overview
CapsLockX 2.0 — keyboard productivity tool + LLM agent, rewritten in Rust.
Uses CapsLock and Space as trigger keys for hotkey combos (hold trigger + press action key).
macOS adapter uses CGEventTap for keyboard hook + CGEventPost for input injection.

## Architecture
```
rs/
  core/                    — platform-agnostic engine + modules
    src/engine.rs          — main key event router (trigger detection, bypass logic)
    src/modules/
      edit.rs              — Space+HJKL cursor, Space+YUIO page nav, Space+G enter, Space+T delete
      mouse.rs             — Space+WASD mouse movement (AccModel physics)
      media.rs             — media keys
      voice.rs             — Space+V voice input (STT via SenseVoice/Whisper)
      brainstorm.rs        — Space+B LLM chat (streams to overlay)
      agent.rs             — Space+M LLM agent (controls computer via CLX commands)
      window_manager.rs    — Space+Z window cycle, Space+X close, Space+C tile
      virtual_desktop.rs   — Space+1-9 virtual desktops
    src/platform.rs        — Platform trait (key_tap, mouse_move, etc.)
    src/llm_client.rs      — multi-provider LLM streaming (Gemini, OpenAI, Anthropic, Ollama)
  adapters/macos/
    src/main.rs            — entry point, subcommand routing (clx, clx agent, clx dino)
    src/hook.rs            — CGEventTap keyboard hook
    src/output.rs          — CGEventPost input injection + AX window management
    src/bin/clx-agent.rs   — standalone agent binary (AX tree, LLM loop, scan reflex)
    src/bin/clx-prompt.rs  — subprocess prompt dialog (no CGEventTap interference)
    src/agent_cmd.rs       — bridges `clx agent` subcommand to clx-agent.rs
    src/brainstorm_overlay.rs — floating overlay panel for brainstorm/agent output
    src/voice_overlay.rs   — voice STT overlay
    src/tray.rs            — menu bar tray icon
    src/config_store.rs    — preferences persistence
    src/prefs.rs           — preferences GUI
skills/clx-agent/SKILL.md — system prompt for LLM agent (editable without rebuild)
docs/agent/               — comprehensive design docs for the agent system
```

## Hotkey Map (hold CapsLock or Space as trigger)
| Key | Action | Module |
|-----|--------|--------|
| HJKL | Cursor movement (vim-style) | edit.rs |
| YUIO | Page nav (Home/End/PgUp/PgDn) | edit.rs |
| G | Enter | edit.rs |
| T | Delete | edit.rs |
| N/P | Tab / Shift+Tab | edit.rs |
| WASD | Mouse movement | mouse.rs |
| V | Voice input (hold=listen, tap=toggle) | voice.rs |
| B | Brainstorm AI chat | brainstorm.rs |
| M | Agent (LLM controls computer) | agent.rs |
| Z | Window cycle (Shift+Z = reverse) | window_manager.rs |
| X | Close tab (Shift=window, Ctrl+Alt=kill) | window_manager.rs |
| C | Tile windows (Shift=side-by-side) | window_manager.rs |
| 1-9 | Switch virtual desktop | virtual_desktop.rs |
| ESC | Dismiss overlays / kill agent (bare, no trigger needed) | engine.rs |
| Ctrl+Cmd+Shift+T | Toggle system Dark Mode (global, no trigger needed) | engine.rs → dark_mode.rs |

**Chord layer** — hold *both* triggers (CapsLock **and** Space) and the number
row becomes function keys: `1`-`0`, `-`, `=` emit F1-F12 (`fn_row.rs`). Held
modifiers decorate them, so CapsLock+Space+Alt+`4` is a real Alt+F4. A single
trigger keeps the virtual-desktop bindings above. Tapping the chord with no
other key still toggles CLX lock, and the lock now latches when both triggers
come *up*, not on key-down.

## Build (macOS)
**Always use `./build.sh`** — it builds, copies, AND code-signs the binary.
Code signing with stable identifier `com.snomiao.capslockx` makes
Accessibility + Screen Recording permissions persist across rebuilds.
```bash
./build.sh
# Equivalent to:
# cd rs && cargo build -p capslockx-macos --release
# cp target/release/capslockx ../clx
# codesign -s - --force --identifier "com.snomiao.capslockx" ../clx
```

After building, clx auto-deduplicates (kills old instance on startup).
Launch: `clx` (forks to background) or `clx -f` (foreground).
The `bin/clx` wrapper script sets `DYLD_LIBRARY_PATH` for onnxruntime.

## Build (Windows)
```
cargo build -p capslockx-windows --release
```
Binary: `rs/target/release/clx.exe`

## After fixing bugs / building
**Always rebuild and relaunch after code changes — without being asked.**
```bash
./build.sh && pkill -f "CapsLockX/clx" || true; clx
```

**Do NOT commit or push without explicit user approval.**
After building, test the feature locally, report findings, and wait.
Only commit when the user says "commit". Only push when the user says "push".
Never chain implement → build → commit → push in one shot.

## Voice Module Rules
- **NEVER suppress or gate mic STT** — both mic and sys tracks must always run and produce output, even if system audio is playing. The user wants to see both transcriptions simultaneously.
- Echo/bleed in the mic track is acceptable; do NOT add energy gates, mic suppression, or silence-feeding to VAD when sys audio is active.
- **The microphone belongs to otoji, not clx.** `otoji listen` opens the input device itself (`rs/core/src/modules/voice_otoji.rs`), so the Microphone TCC grant is otoji's. clx must NOT request mic permission — do not re-add a `check_and_request_mic_permission`-style call in `main.rs`. clx only *reads* the mic mode (query-only `active_microphone_mode()` in `mic_mode.rs`) for the tray label / Voice Isolation tip; that opens nothing and needs no grant.

## STT polish chain
Default since 2026-05-14: `min-chars:15,min-duration:5s,mlx:qwen2.5-3b,llm-corrector,raw`. The two leading pseudo-stages short-circuit the chain (return raw text immediately) on short voice commands, where LLM polish was empirically shown to degrade CER ~6× — see [`lib/otoji/docs/2026-05-14-polish-bench.md`](lib/otoji/docs/2026-05-14-polish-bench.md). Long dictation still flows through MLX/llm-corrector. Stages are parsed in `rs/core/src/modules/voice.rs::polish_stt_with_chain`; callers can override via the otoji settings panel. Existing user configs are auto-migrated on load (`lib/otoji/src/config.rs::apply_migrations`).

## Privacy: never use user's real prompts as examples
When writing few-shot examples or documentation for LLM prompts (polish, translate, agent system prompts, `lib/otoji/src/polish.rs`, `skills/clx-agent/SKILL.md`, etc.), **NEVER copy phrases from the user's voice logs, chat messages, transcripts, or `tmp/clx-watchdog.log`**. Always synthesize generic, unrelated examples (e.g. `kafka`, `npm run`, `下雨了`, `water bottle`).

Why: the user speaks personal, work-in-progress content through the mic. Anything embedded in a committed system prompt (a) ships to the cloud LLM on every subsequent call, and (b) lives in git history forever. This rule is a hard requirement — violating it is a privacy regression even when the example "seems harmless."

Rule of thumb: if you can find the candidate example string by grepping `tmp/` or scrolling the conversation, it is forbidden. Invent a new one.

## Key Design Decisions

### macOS Permission Hazard
**NEVER call AX or CG APIs directly** without confirming permission first.
- `AXUIElementCopyAttributeValue` → hangs in kernel (UE) without Accessibility permission
- `CGWindowListCreateImage` → hangs without Screen Recording permission
- UE processes cannot be killed (even kill -9), only cleared by reboot
- **Solution**: Use osascript for AX tree, fork() child for screen capture test

### Agent Architecture
- `clx agent --tree` — dumps AX tree via osascript (safe, no UE risk)
- `clx agent --prompt "task"` — full LLM loop (AX tree + screenshot → Gemini → CLX commands)
- `clx agent --exec` — executes CLX commands from stdin
- `clx dino` — auto-plays Chrome Dino game with 60fps pixel-scan reflex
- Agent logs to `./tmp/agent.log` and `./tmp/agent-live.log` (overlay watches live log)
- All temp files in `./tmp/` (never /tmp/ or ~/Library/)

### CLX Agent Language
```
k a          tap key 'a'
k c-c        Ctrl+C (c=ctrl, s=shift, a=alt, w=cmd)
k w-p        Cmd+P (macOS)
k "text"     type string (\n \t \" \\ escapes)
m 400 300    move mouse absolute
m 400 300 c  move + click
w 200ms      wait
wf "text" 3s wait for text in AX tree (polls every 200ms)
scan ID x y w h dark>N { k key }   pixel-scan reflex at 60fps
scan_stop ID / scan_stop all
S screen region x y w h   set capture region
? screen                  one-shot screenshot
```

### Bypass Logic (engine.rs)
These modifier+Space combos bypass CLX mode (pass through to OS):
- Shift+Space → IME switching
- Ctrl+Space → IME switching (like AHK)
- Cmd+Space → Spotlight
```