# pi-automation

Automation toolkit for the [pi coding agent](https://pi.dev). Aggregates web access, browser control, screen/window tools, and native Windows UI automation into a single extension.

## Tools

| Tool | Source | Description |
|------|--------|-------------|
| `web_search` | pi-web-access | Web search (multiple providers) |
| `source_check` | pi-web-access | Verify a claim with source citations |
| `fetch_content` | pi-web-access | Fetch URLs (GitHub / PDF / video / YouTube) |
| `get_search_content` | pi-web-access | Retrieve stored search/fetch content |
| `browser` | local | Drive a real browser via CDP (launch/list/navigate/click/type/js/read/wait/screenshot/close) |
| `take_screenshot` | local | Capture screen (single/stitched/region/scale) |
| `hide_window` | local | Hide / show a window |
| `ui_automation` | local | Read & operate native Windows UI controls |

Commands (from pi-web-access): `/websearch`, `/curator`, `/google-account`, `/search`.
Shortcuts: `Ctrl+Shift+S` (curate), `Ctrl+Shift+W` (activity).

## `ui_automation` actions

Target a window via `process` (name/PID), `window_title`, or `window_handle` (hex). Match controls by `name`, `automation_id`, or `control_type`.

**Control actions**
- `list_windows` — list top-level windows
- `list_controls` — enumerate a window's control tree
- `find` — find controls by name / controlType / automationId
- `click` — invoke a control (InvokePattern, or focus+click fallback)
- `click_coords` — mouse click at `x`, `y`. Supports `button` (left/right/middle) and `click_count`. With `relative: true`, `x`/`y` are offsets from the `relative_to` origin instead of absolute screen coords (default `relative_to: "desktop"` = virtual desktop top-left, or `"window"` = target window top-left).
- `set_value` / `get_value` — ValuePattern (edit boxes)
- `get_text` — TextPattern (read text content)
- `focus` — move focus to a control
- `toggle` — TogglePattern (checkbox / switch)
- `set_range` / `get_range` — RangeValuePattern (slider / numeric)
- `select` — SelectionItemPattern (list item)
- `scroll` — ScrollPattern (`amount`, `scroll_axis`)
- `expand` / `collapse` — ExpandCollapsePattern

**Window actions**
- `window_minimize` / `window_maximize` / `window_restore` / `window_close`
- `window_move` — move window to (`x`, `y`)

`find` / `list_controls` report each control's supported patterns.

## Architecture

- Reuses [pi-web-access](https://github.com/nicobailon/pi-web-access) as an npm dependency (`bundledDependencies`), calling its default extension export.
- `index.ts` wires `webAccess(pi)` plus `registerBrowser`, `registerScreen`, and `registerUiAutomation`.
- Local modules:
  - `browser.ts` → `registerBrowser` (`browser`)
  - `screen.ts` → `registerScreen` (`take_screenshot`, `hide_window`)
  - `ui-automation.ts` → `registerUiAutomation` (`ui_automation`)

## Install

### As a pi package

Add the repo to pi's `settings.json`:

```json
{
  "extensions": ["D:/pi-automation"],
  "packages": []
}
```

### Copy to the auto-discovery folder

Copy this package to pi's global extensions dir and install deps:

```
C:\Users\Admin\.pi\agent\extensions\pi-automation\
cd pi-automation && npm install --omit=dev
```

## Avoid tool name collisions

This package registers `browser`, `take_screenshot`, `hide_window`, and the web tools. Disable old sources before enabling it:

- `~/.pi/agent/extensions/browser-cdp.ts` (`browser`)
- `~/.pi/agent/extensions/screen-tools.ts` (`take_screenshot`, `hide_window`)
- `settings.json` `packages`: remove `"npm:pi-web-access"`

Then run `/reload`.

## Development

```bash
cd D:/pi-automation
npm install --omit=dev      # install runtime deps
npm run typecheck           # optional TS check
git add -A && git commit -m "..."   # commit
```

To test loading in pi: `pi -e D:/pi-automation`, or `/reload`.

## Notes

- `ui_automation`, `take_screenshot`, `hide_window` are Windows-only; other platforms register no-op/`not available` tools.
- Reusing `pi-web-access` also brings its commands and shortcuts.