--- name: muggle-test-regenerate-missing model: sonnet description: "Bulk-regenerate test scripts for every test case in a Muggle AI project that has no active script. Scans the project, finds test cases stuck in DRAFT or GENERATION_PENDING, shows the user the list, and on approval kicks off bulk remote script generation via the Muggle Test cloud. Use when the user wants to regenerate missing scripts, fill in missing test scripts, generate scripts for test cases without one, rebuild scripts for stale test cases, or bulk regenerate across a project. Handles discovery, filtering, confirmation, and remote dispatch end-to-end." --- # Muggle Test — Regenerate Missing Test Scripts > Telemetry first step: see [`_shared/telemetry-emit.md`](../_shared/telemetry-emit.md). Use `skillName: "muggle-test-regenerate-missing"`. A bulk maintenance skill for Muggle AI projects. It finds every test case in a project that does **not** currently have an active (ready-to-run) test script, shows the list to the user, and on approval triggers a remote test script generation workflow for each one. Useful after creating a batch of new test cases or when cleaning up a project that has drifted. Execution is **remote only** — Muggle Test's cloud generates the scripts in parallel against the project URL. The user's machine is not involved beyond making API calls. ## Preferences Gates run per `preference-gates/README.md`. | Preference | Step | Decision it gates | |------------|------|-------------------| | `autoLogin` | 1 | Reuse saved credentials when auth is required | | `autoSelectProject` | 2 | Reuse last-used Muggle Test project for this repo | ## Concept: what counts as "no active script" In the Muggle Test data model, a test case carries a status that reflects whether it has a usable script attached: | Status | Meaning | Regenerate? | |:-------|:--------|:-----------:| | `ACTIVE` | Has a generated, ready-to-run script | No — already good | | `DRAFT` | Created but never generated | **Yes** | | `GENERATION_PENDING` | Queued but generation never started | **Yes** | | `GENERATING` | Currently generating | No — generation is in flight; don't double-dispatch | | `REPLAY_PENDING` / `REPLAYING` | A replay is in flight; script exists | No — busy with replay | | `DEPRECATED` | Marked stale on purpose | No — user decision | | `ARCHIVED` | Hidden from normal flows | No — user decision | The skill targets exactly **DRAFT** and **GENERATION_PENDING** by default. These are the two states that mean "no usable script attached and nothing running to produce one." `GENERATING` is deliberately excluded — a workflow is already in progress, and firing a second one races against the first and wastes budget. Treat this filter as a default, not a law. If the user explicitly says "include generating ones too, they're stuck" or "include deprecated", respect the override — but don't widen the filter on your own. ## UX Guidelines — Minimize Typing **Every selection-based question MUST use the `AskUserQuestion` tool** (or the platform's equivalent structured selection tool). Never ask the user to "reply with a number" — always present clickable options. - **Selections** (project, which test cases to include): Use `AskUserQuestion`, with `allow_multiple: true` for the test case picker. - **Free-text inputs** (project URL when creating, override filters): Only ask as plain text when the option set isn't finite. - **Batch related questions** when independent. Don't ask sequentially what could be one screen. ## Workflow ### Step 1 — Authenticate (gated by `autoLogin`) 1. Call `muggle-remote-auth-status`. 2. If **authenticated and not expired** → gate `autoLogin` (per `preference-gates/README.md`): - `always` → proceed with saved session. - `never` → `muggle-remote-auth-login` with `forceNewSession: true`, then `muggle-remote-auth-poll`. - `ask` → run Picker 1 from `preference-gates/autoLogin.md` via `AskUserQuestion`; map the answer back to one of the actions above. 3. If **not authenticated or expired** → call `muggle-remote-auth-login`, then poll with `muggle-remote-auth-poll`. 4. Do not skip auth and do not assume a stale token still works. If auth keeps failing, suggest the user run `muggle logout && muggle login` from a terminal. ### Step 2 — Select Project (gated by `autoSelectProject`) A **project** is the unit on the Muggle AI dashboard that groups test cases, scripts, and runs. The user must pick the one to scan — never auto-select from repo name, branch, or URL heuristics. The per-repo project cache lives in `~/.muggle-ai/last-project.json`, in the entry keyed on `` (via the `muggle-local-last-project-get` / `muggle-local-last-project-set` MCP tools). Look for `Muggle Test Last Project: id=… url=… name="…"` in session context. Gate `autoSelectProject` (per `preference-gates/README.md`). Cache: `Muggle Test Last Project` session line. - `always` + cache → use cached `projectId`, proceed to Step 3. No cache → fall through to `ask`. - `never` → full project list; skip Picker 2. - `ask` → project list picker (see gate file for spec + Picker 2 override). Skip Picker 2 if "Create new project". ### Logic 1. Call `muggle-remote-project-list` (only when not satisfied by the `always` cache). 2. Use `AskUserQuestion` to present projects as clickable options. Include the project URL in each label so the user can disambiguate. Always include "Create new project" as the last option. 3. Wait for explicit selection. 4. If the user picks "Create new project": collect `projectName`, `description`, and `url`, then call `muggle-remote-project-create`. Store `projectId` and `projectUrl` only after the user confirms — both are needed downstream. ### Step 3 — Scan Test Cases Pull the full set of test cases for the project. The list endpoint is paginated. 1. Call `muggle-remote-test-case-list` with `projectId`. Start at page 1; continue requesting pages until the response indicates no more items. Use a generous `pageSize` (e.g. 100) to keep the call count low. 2. Accumulate everything into a single in-memory array. 3. Tell the user the totals as you go if the project is large (e.g., "Found 247 test cases across the project"). If the call returns zero test cases, stop and tell the user — there is nothing to regenerate. Suggest they create test cases first (point them at `muggle-test` or `muggle-test-feature-local`). ### Step 4 — Filter to Missing Scripts Apply the default filter: keep only test cases where `status` ∈ `{DRAFT, GENERATION_PENDING}`. Then show a one-line summary. Also surface how many cases are currently `GENERATING` so the user knows about in-flight work (but don't include them in the candidate list unless they override the filter): ``` Project: () Total test cases: 247 With active script (skipped): 198 Currently generating (skipped, in-flight): 32 Needs regeneration: 17 • DRAFT: 12 • GENERATION_PENDING: 5 ``` If after filtering the list is empty, congratulate the user — every test case already has an active script — and stop. Do not invent work. ### Step 5 — Present and Confirm Selection Use `AskUserQuestion` with `allow_multiple: true` to present every candidate test case as a clickable option. The user must explicitly approve which ones to regenerate. For each option label, show enough context for the user to make a real decision: ``` [] — use case: <use case title> ``` For example: - `[DRAFT] Sign up with valid email — use case: User Registration` - `[GENERATION_PENDING] Add item to cart — use case: Checkout Flow` Default behavior: - If there are **≤ 25** candidates, present all of them in a single `AskUserQuestion` with everything pre-checked and let the user deselect anything they want to skip. - If there are **> 25** candidates, show the first 25 ranked by status priority (`DRAFT` → `GENERATION_PENDING`), plus a tail option **"Include all N — don't make me click each one"**. The user can also pick "Show next batch" to see more. After selection, call `AskUserQuestion` once more for a final confirmation: > "About to start remote test script generation for **N** test cases against `<projectUrl>`. This will consume Muggle Test workflow budget. Proceed?" > > - "Yes, start all N" > - "No, cancel" Only proceed after the user picks "Yes". ### Step 6 — Dispatch Remote Generations (Bulk) Send a single bulk request instead of dispatching one workflow per test case: 1. Call `muggle-remote-workflow-start-test-script-generation-bulk` with: - `projectId` — from Step 2 - `name` — `"muggle-test-regenerate-missing: bulk ({count} test cases)"` where `{count}` is the number of selected test cases - `testCaseIds` — array of all selected test case IDs from Step 5 2. The backend handles looking up full test case details (goal, precondition, instructions, expectedResult, url), so there is no need to call `muggle-remote-test-case-get` per test case. 3. Parse the response to get the `items` array with per-test-case status. Each item contains the test case ID, dispatch status, and (when successful) the workflow runtime ID. **Failure handling:** the bulk API returns per-item status in the response `items` array. Individual test cases may fail (validation error, missing field, etc.) while others succeed. Surface failures in the Step 7 report — partial progress beats no progress. ### Step 7 — Report After the bulk dispatch returns, build a summary table from the response `items` array. Each item contains a test case ID, dispatch status, and (when successful) a workflow runtime ID. Cross-reference with the test case list from Step 3 to fill in titles and use case names: ``` Test Case Use Case Prev Status Dispatch Runtime ─────────────────────────────────────────────────────────────────────────────────────────────────── Sign up with valid email User Registration DRAFT ✅ started rt-abc123 Sign up with disposable email User Registration DRAFT ✅ started rt-def456 Add item to cart Checkout Flow GENERATION_PEND. ✅ started rt-ghi789 Apply expired coupon Checkout Flow GENERATION_PEND. ❌ failed — ─────────────────────────────────────────────────────────────────────────────────────────────────── Total: 17 dispatched | 16 started | 1 failed ``` For failures: include a one-line error excerpt from the item's error field and (where possible) a hint at the cause (e.g., "missing instructions field — edit the test case in the dashboard, then re-run this skill"). ### Per-item failure routing (regen) For every item with a non-success dispatch status, follow [`_shared/failure-mode-handling.md`](../_shared/failure-mode-handling.md) section C (regen failure buckets: `transient` / `infra` / `agent-course` / `product-uxux`). Because this skill is bulk-dispatch (no live user attention per item), batch the routing — don't ask the user per item: 1. Group failed items by AI-classified bucket using the signal heuristics in the shared doc (most failures here will be `transient` or `infra` since dispatch failures rarely surface product-uxux signals). 2. Emit one `regen-failure-classified` event per failed item via `muggle-local-telemetry-event-emit`. 3. Present the buckets to the user via a single `AskUserQuestion` summarizing counts: e.g., "12 transient (recommend retry), 3 infra (recommend report-bug). What do you want to do?" Options: "Retry transient items", "Report all to Muggle AI", "muggle-feedback for selected items", "Skip — leave them as-is". 4. Emit one `regen-failure-resolved` event per item with the user's batch decision applied. Bulk regen does not run replays, so the section B (replay) router does not apply here. ### Step 8 — Open the Dashboard Open the Muggle AI dashboard so the user can watch progress visually. Point them at the **scripts** page (where DRAFT → GENERATING → ACTIVE transitions are visible), not `/runs` (which is for replay batches and shows nothing for a regen): ```bash open "https://www.muggle-ai.com/muggleTestV0/dashboard/projects/{projectId}/scripts" ``` Tell them: > "I've opened the project's scripts page. Generation jobs typically take a few minutes each — your test cases will move from `DRAFT` through `GENERATING` to `ACTIVE` as scripts complete." ### Step 9 (optional) — Poll Status Only if the user explicitly asks for a status check, call `muggle-remote-wf-get-ts-gen-latest-run` for each runtime ID and report progress. **Do not poll on a tight loop by default** — the dispatch step is the actual goal of this skill, and the dashboard already shows live status better than a CLI loop ever can. Polling exists as a courtesy for users who don't want to leave the terminal. If the user wants a one-shot snapshot, present a small table: ``` Test Case Runtime Status Steps so far ──────────────────────────────────────────────────────────── Sign up with valid email rt-abc123 RUNNING 6 Add item to cart rt-ghi789 COMPLETED 12 ``` ## Tool Reference | Phase | Tool | |:------|:-----| | Auth | `muggle-remote-auth-status`, `muggle-remote-auth-login`, `muggle-remote-auth-poll` | | Project | `muggle-remote-project-list`, `muggle-remote-project-create` | | Scan | `muggle-remote-test-case-list` (paginated) | | Dispatch | `muggle-remote-workflow-start-test-script-generation-bulk` | | Status (optional) | `muggle-remote-wf-get-ts-gen-latest-run`, `muggle-remote-wf-get-latest-ts-gen-by-tc` | | Browser | `open` (shell command) | ## Non-negotiables - **The user MUST select the project** — present projects via `AskUserQuestion`, never infer from cwd, repo name, or URL guesses. - **The user MUST approve which test cases to regenerate** — show the candidates via `AskUserQuestion`, let them deselect, then confirm again before any dispatch. Bulk-regenerating without approval can waste meaningful workflow budget. - **Default filter is `DRAFT` + `GENERATION_PENDING`** — never include `GENERATING`, `ACTIVE`, `DEPRECATED`, `ARCHIVED`, `REPLAYING`, or `REPLAY_PENDING` unless the user explicitly says so. `GENERATING` already has a workflow in flight and dispatching another races against it. `ACTIVE` test cases already have working scripts. The rest reflect deliberate user decisions or in-flight replays the skill should not interfere with. - **Use the bulk endpoint for dispatch** — call `muggle-remote-workflow-start-test-script-generation-bulk` once with all selected test case IDs rather than dispatching one-by-one. The backend resolves full test case details internally. - **Failures don't abort the batch** — the bulk API returns per-item status. Surface failures in the report. Partial progress beats no progress. - **Open the dashboard, don't poll by default** — the runs page is the canonical view of progress. Only poll if the user explicitly asks. - **Use `AskUserQuestion` for every selection** — never ask the user to type a number. - **Can be invoked at any state** — if the user already has a project chosen in conversation context, skip Step 2 and go straight to scanning.