# Test Plan: `pi-c-review`

> **Comprehensive Quality Assurance & Verification Plan**  
> Document Version: 1.0.0  
> Status: Approved & Executed

---

## 1. Scope & Objectives

The primary goal of this test plan is to verify that `pi-c-review` accurately loads custom and preset C coding rules, generates structured review prompts, parses LLM output payloads reliably, computes accurate line diffs, handles file system mutations safely (with backup creation), and integrates smoothly with the Pi Coding Agent (`pi.dev`) runtime.

### In-Scope
- Configuration discovery, JSON validation, and hierarchical rules merging.
- Built-in rule presets (`memory-safety`, `buffer-security`, `embedded-misra`).
- Prompt generation and defensive JSON unmarshaling (handling markdown code fences and whitespace).
- LCS dynamic programming diff algorithm and terminal ANSI color formatting.
- Pi extension lifecycle registration (slash commands, LLM tools, event hooks).
- Safety guardrails (file existence checks, confirmation dialogs, `.bak` backup creation).

### Out-of-Scope
- Third-party LLM inference latency or model-specific hallucinations (mitigated via prompt constraints).
- Testing host operating system file system drivers.

---

## 2. Test Strategy & Levels

```
                     ┌───────────────────────┐
                     │   End-to-End (E2E)    │  Simulate full /c-review & /c-fix
                     │   Scenario Testing    │  cycles on unsafe_sample.c
                     └───────────┬───────────┘
                                 │
                     ┌───────────▼───────────┐
                     │  Integration Testing  │  Extension registration, Pi API
                     │                       │  mocks, tool invocation
                     └───────────┬───────────┘
                                 │
                     ┌───────────▼───────────┐
                     │     Unit Testing      │  Config parser, Diff engine,
                     │                       │  Prompt builder, Presets
                     └───────────────────────┘
```

---

## 3. Test Matrix & Detailed Test Cases

| Test ID | Test Category | Description | Input / Setup | Expected Outcome | Status |
| :--- | :--- | :--- | :--- | :--- | :--- |
| **TC-CFG-01** | Unit | Verify existence and integrity of built-in presets | Query `BUILTIN_PRESETS` map | All 3 presets (`memory-safety`, `buffer-security`, `embedded-misra`) are defined and contain non-empty rule arrays. | **PASS** |
| **TC-CFG-02** | Unit | Fallback to default presets when no config exists | Call `resolveAllRules(DEFAULT_CONFIG)` | Returns rules containing malloc NULL checks and string boundary restrictions. | **PASS** |
| **TC-CFG-03** | Unit | Hierarchical rule merging with ad-hoc CLI constraints | Pass custom config + ad-hoc array `["Rule X"]` | Merged array contains preset rules, custom config rules, and the ad-hoc rule in order. | **PASS** |
| **TC-ENG-01** | Unit | Review prompt assembly | Call `buildReviewPrompt` with C code and rules | System prompt defines expert persona & JSON schema; User prompt contains C code in fences and numbered rules. | **PASS** |
| **TC-ENG-02** | Unit | Defensive parsing of markdown-fenced JSON | Pass response wrapped in ` ```json ... ``` ` | Code fences are stripped, valid JSON is parsed, returning typed `ReviewResult`. | **PASS** |
| **TC-ENG-03** | Unit | Robustness against non-JSON text surrounding payload | Pass text with introductory notes before `{` and after `}` | First `{` and last `}` are isolated and successfully unmarshaled. | **PASS** |
| **TC-DIF-01** | Unit | Accurate LCS line diff computation | Compare 3-line string with modified 4-line string | Accurately identifies `added`, `removed`, and `unchanged` lines without off-by-one errors. | **PASS** |
| **TC-DIF-02** | Unit | Terminal diff formatting with context | Call `formatTerminalDiff` on sample modification | Output contains `--- Original`, `+++ Corrected`, cyan hunk header, and ANSI-colored diff lines. | **PASS** |
| **TC-SCN-01** | Unit | Banned string functions detection | Scan code with `gets`, `strcpy`, `strcat`, `sprintf` | Reports `SEC-01..04` with critical severity and replacement suggestions. | **PASS** |
| **TC-SCN-02** | Unit | Contextual allocation NULL check | Scan `malloc` followed by `strcpy` vs `malloc` followed by `if (!ptr)` | Only flags `MEM-01` when NULL check is absent; zero false positives on compliant code. | **PASS** |
| **TC-SCN-03** | Unit | Pointer nullification after `free()` | Scan `free(ptr)` followed by `return` vs `free(ptr); ptr = NULL;` | Only flags `MEM-02` when pointer is not nullified. | **PASS** |
| **TC-SCN-04** | Unit | Inline `// NOLINT` suppression | Scan line with `// NOLINT` and `// NOLINT(RULE)` | Suppressed rules are excluded from findings. | **PASS** |
| **TC-SCR-01** | Unit | Score deduction & capping | Compute score for 10 critical issues | Deductions capped at 60 points; score floors at 40/D instead of negative. | **PASS** |
| **TC-SCR-02** | Unit | Grade boundaries | Compute scores for 100, 89, 74, 59, 39 | Returns exact grades A, B, C, D, F respectively. | **PASS** |
| **TC-ALL-01** | Integration | Multi-file directory scan | Run `reviewDirectory("tests/samples")` | Discovers all sample files, computes per-file metrics and aggregated overall score. | **PASS** |
| **TC-ALL-02** | Integration | Project table formatting | Call `formatProjectTablePlain` on summary | Outputs formatted ASCII table with columns File, Score, Grade, Crit, Warn, Style. | **PASS** |
| **TC-EXT-01** | Integration | Registration of commands, tools, and hooks with Pi | Invoke `cReviewExtension(mockPi)` | Slash commands (`c-review`, `c-fix`, `c-rules`, `c-review-all`), tool (`c_review_and_fix`), and `tool_result` event hook are registered. | **PASS** |
| **TC-EXT-02** | Integration | LLM tool schema and execution | Execute `c_review_and_fix.execute()` | Returns JSON content package containing instructions, system prompt, user prompt, and rules applied. | **PASS** |
| **TC-E2E-01** | E2E Scenario | Audit of unsafe C file | Execute review pipeline on `tests/samples/unsafe_sample.c` | Identifies all 5 deliberate violations (missing NULL check, unbounded `strcpy`, missing `fclose`, memory leak, dangling pointer). | **PASS** |
| **TC-E2E-02** | E2E Scenario | TypeScript compilation check | Run `npm run build` (`tsc`) | Clean compilation with zero TypeScript errors; outputs declaration files (`.d.ts`) and ESM JavaScript. | **PASS** |

---

## 4. Test Data & Fixtures

### Sample Unsafe C Source: `tests/samples/unsafe_sample.c`
This fixture includes 5 intentional C programming anti-patterns:
1. `char *buffer = (char *)malloc(32);` (No NULL check before use)
2. `strcpy(buffer, input_str);` (Unbounded string copy / buffer overflow risk)
3. `FILE *fp = fopen("output.log", "w");` (Failure to close on error exit)
4. Early `return -1;` before `free(buffer)` (Memory leak)
5. `free(buffer);` without setting `buffer = NULL;` (Dangling pointer)

---

## 5. Execution Procedures

### 5.1. Running Unit & Integration Tests
Execute the automated test suite using Vitest:

```bash
# Single execution run
npm test

# Continuous watch mode during development
npm run test:watch
```

### 5.2. Verification of TypeScript Compilation
Verify type safety and bundling:

```bash
npm run build
```

---

## 6. Acceptance Criteria

The extension meets release standards when:
1. 100% of automated unit and integration tests pass without warnings or unhandled rejections.
2. TypeScript compilation completes with code `0` under `strict: true`.
3. All registered slash commands gracefully handle missing arguments or non-existent files with informative TUI notifications.
4. File write actions in `/c-fix` guarantee backup creation prior to modifying user files.