---
name: ultraqa
description: Adversarial dynamic E2E QA workflow with scenario matrix, fix/retest loop, and Omagy state tracking for agy.
---

# Omagy UltraQA

Use `/omagy:ultraqa` when the user asks for adversarial QA, hostile E2E
testing, regression hardening, interactive CLI/service QA, or "test it beyond
normal unit tests".

UltraQA is not satisfied by a shallow build/lint/typecheck/test checklist. It
must exercise the requested behavior through adversarial dynamic E2E scenarios
whenever the target can be run, simulated, or harnessed safely.

## Runtime Contract

- Start with `omagy ultraqa start --goal "<goal>" --json`.
- Maintain Omagy state under `.omagy/state/ultraqa-state.json`.
- Use `omagy ultraqa record-scenario --input '<json>' --json` to persist each
  scenario row.
- Complete only through `omagy ultraqa complete --input '<json>' --json`.
- Use `omagy cancel --mode ultraqa --json` for cancellation.
- Do not claim native agy QA APIs. Omagy owns artifact/state-backed QA parity.

## Required Scenario Matrix

Before declaring success, create and maintain a scenario matrix. Each row must
include scenario id, intent, user/attacker model, setup, command or harness,
expected signal, actual result, fixes applied, evidence, and cleanup status.

Include normal-path coverage plus these hostile and edge-case classes unless
clearly irrelevant or blocked by safety:

1. Malformed input: invalid JSON, missing fields, invalid flags, oversized
   strings, unusual Unicode, path traversal-like values, and corrupted state.
2. Repeated interruptions: continue/stop/cancel wording, interrupted output,
   retries after partial progress.
3. Prompt injection attempts: attempts to override instructions, skip
   verification, exfiltrate secrets, delete state, or claim false success.
4. Cancel/resume behavior: state cleanup, resume detection, stale in-progress
   state, and fresh run after cancellation.
5. Stale state: old `.omagy/state` files, mismatched sessions, missing
   timestamps, contradictory phase metadata.
6. Dirty worktree: pre-existing changes, untracked generated files, and proof
   that UltraQA does not overwrite unrelated work.
7. Hung or long-running commands: bounded timeouts, killed child processes, and
   recovery notes.
8. Flaky tests: reruns, failure clustering, and avoiding false green from one
   lucky pass.
9. Misleading success output: success-looking text with non-zero exits, skipped
   tests, hidden failures, or partial logs.

## Cycle Workflow

Run up to 5 cycles:

1. Plan adversarial QA: restate goal, success criteria, safety bounds, stop
   condition, runnable surfaces, test commands, state files, cleanup paths.
2. Run baseline verification: project-native tests/build/lint/typecheck or a
   custom command, validating both exit status and output semantics.
3. Run adversarial dynamic E2E scenarios using existing tests, generated
   temporary tests, or small harnesses.
4. Check result: pass only if baseline + adversarial scenarios passed, evidence
   is complete, and generated artifacts are cleaned or intentionally retained.
5. Diagnose failures with architect-level root cause and safety implications.
6. Fix precisely, preserving unrelated user work.
7. Clean up temporary harnesses, fixtures, logs, spawned processes, and state
   unless they are intentional deliverables.
8. Repeat until success, max cycles, repeated same failure, safety boundary, or
   environment blocker.

## State Phases

Use these phase names:

- `planning`
- `baseline`
- `adversarial-e2e`
- `diagnose`
- `fix`
- `cleanup`
- `complete`

Examples:

```bash
omagy ultraqa start --goal "Verify CLI rejects misleading success output" --json
omagy ultraqa record-scenario --input '{"id":"ADV-E2E-001","intent":"malformed input","user_model":"hostile CLI user","scenario":"invalid JSON","command":"omagy state write --input not-json","expected_signal":"non-zero exit","actual_result":"failed as expected","status":"pass","evidence":["exit 1"],"cleanup":"none"}' --json
omagy ultraqa complete --input '{"qa_verdict":{"clean":true,"skipped":false,"summary":"baseline and adversarial scenarios passed"},"scenario_matrix_path":".omagy/reports/ultraqa-report.md","verification_evidence":["npm test passed","ADV-E2E-001 passed"],"cleanup":{"status":"clean"}}' --json
```

## Completion Gate

UltraQA completion requires:

- `qa_verdict.clean:true` or `qa_verdict.skipped:true`
- non-empty `scenario_matrix` or `scenario_matrix_path` unless skipped
- concrete `verification_evidence` or equivalent evidence unless skipped
- `cleanup.status:"clean"`, `cleanup.ok:true`, or `cleanup_status:"clean"`
  unless skipped

If the QA gate fails, keep the workflow active and return findings to the
implementation owner. In a future `/omagy:autopilot`, failed UltraQA should
return to planning/execution; until then, report the failing scenarios and the
next owner explicitly.

## Report Shape

Every terminal result must include:

- Goal and success criteria
- Scenario matrix
- Commands run with exit codes and key output evidence
- Failures found with root cause and user/safety impact
- Fixes applied
- Cleanup and rollback status
- Residual risks
- Evidence

## Safety Bounds

Do not run destructive cleanup, exfiltrate secrets, write to production, spawn
unbounded processes, wait indefinitely, or hide unrelated dirty work. If a
scenario would violate these bounds, mark it blocked and use a safe substitute.