# pi-perf

> Pi-native performance benchmarking toolkit for pi.dev extensions — profiling, benchmarking, and CI reporting.

## Installation

```bash
pi install npm:pi-perf
```

## What It Does

pi-perf provides deterministic performance measurement tools for pi.dev extensions. No LLM calls — all measurements are pure timing, memory, and CPU instrumentation.

**Three tools:**
- **perf_profile** — Run a tool N times, collect timing/memory/CPU stats, detect outliers via IQR
- **perf_bench** — Run a benchmark suite (JSON-defined test cases), compare against baselines for regression detection
- **perf_report** — Generate CI-friendly reports (JSON/Markdown/HTML) with configurable pass/fail thresholds

## Tools

### `perf_profile`

Profile the execution of a pi.dev extension tool.

**Parameters:**
- `toolName` (string, required) — Name of the tool to profile
- `args` (object, optional) — Arguments to pass to the tool
- `iterations` (number, optional, default: 10) — Number of measured iterations (1-1000)
- `warmup` (boolean, optional, default: true) — Run 3 warmup iterations before measuring

**Example:**
```
Use the perf_profile tool with toolName="my_tool", iterations=20, warmup=true
```

**Returns:** Duration stats (avg, min, max, P50/P95/P99), memory delta, CPU usage, outlier list.

### `perf_bench`

Run a benchmark suite against pi.dev extensions.

**Parameters:**
- `suitePath` (string, optional) — Path to a benchmark suite JSON file (default: discovers in `./perf-suites/`)
- `compare` (string, optional) — Path to a baseline result JSON for comparison
- `output` (string, optional, default: "markdown") — Output format: `json`, `markdown`, or `table`

**Example:**
```
Use the perf_bench tool with suitePath="./perf-suites/ci.json", compare="./baselines/v1.json"
```

**Suite JSON format:**
```json
{
  "name": "my-suite",
  "description": "Regression tests",
  "cases": [
    { "name": "fast-tool", "toolName": "tool_a", "iterations": 10 },
    { "name": "slow-tool", "toolName": "tool_b", "iterations": 5, "warmup": false }
  ]
}
```

**Comparison thresholds:**
- Regressions: >15% slower → FAIL
- Warnings: >5% slower → WARN
- Improvements: >10% faster → noted

### `perf_report`

Generate a performance report with CI-friendly output.

**Parameters:**
- `format` (string, required) — Output format: `json`, `markdown`, or `html`
- `outputPath` (string, optional) — Write report to file (default: return inline)
- `thresholds` (object, optional) — Pass/fail thresholds as `metric:value` pairs

**Available threshold metrics:**
- `avg_duration_ms` — Average execution time
- `p95_duration_ms` — 95th percentile latency
- `p99_duration_ms` — 99th percentile latency
- `memory_delta_mb` — Memory usage change
- `outliers_count` — Number of statistical outliers

**Example:**
```
Use the perf_report tool with format="markdown", thresholds={"avg_duration_ms": 50, "p95_duration_ms": 100}
```

## CI Integration

Use with pi-ci to add performance gates to your CI pipeline:

```yaml
- name: Performance check
  run: |
    pi --provider xiaomi --model mimo-v2.5-pro --api-key "$API_KEY" \
      -e . -p "Run perf_bench with suitePath='./perf-suites/ci.json' then perf_report with format='json' and thresholds={\"avg_duration_ms\":50}" \
      --no-session
```

## Resources

- [npm](https://www.npmjs.com/package/pi-perf)
- [GitHub](https://github.com/ZachDreamZ/pi-perf)
- [pi.dev](https://pi.dev/packages/pi-perf)

## License

MIT