# Build status

Current verification follow-up: [verify.md](verify.md). The table below is a historical build
log; later dated entries and that follow-up list supersede earlier "untested" notes.
The 2026-09-10 review fixes are recorded in [CODE_REVIEW.md](CODE_REVIEW.md).

Plan: `~/.claude/plans/twinkly-inventing-hopper.md` (approved 2026-09-05).

| Step | Status | Notes |
| --- | --- | --- |
| 1 Bootstrap (git, package, deps, pac install) | done | .NET 10 SDK user-scope in `~/.dotnet` (winget needed elevation and hung); pac 2.11.2 as dotnet global tool; `DOTNET_ROOT` user env var set; `runPac` defaults it |
| 2 Workspace oracle (`pac copilot init` / `pack`) | done | scaffolds committed as `test/fixtures/pac-default` and `pac-clicopilot`; `scripts/oracle-pack.mjs` re-runs init + authoring + pack |
| 3 Workspace + schema (`workspace.ts`, `schema.ts`) | done | validator ported from Microsoft's schema-lookup and extended; root unknown props are warnings, action-level are errors |
| 4 Sync layer (`pac.ts` + tools) | done (code) | init/clone/pull/push/pack/import/publish/status/list + `cs_pac`; only `init`, `pack`, `auth list` exercised (no environment) |
| 5 Authoring tools | done | topics, knowledge (3 kinds + files), tools (connector/mcp/flow), flows (experimental), triggers, variables, agent/settings edits; naming aligned to pac (`<PascalName>.mcs.yml`, `mcs.metadata`) |
| 6 Cloud: auth, environments, publish, chat | done (code) | MSAL device-code + interactive; BAP; Dataverse; DirectLine + SDK chat; untested live |
| 7 Evaluations | done (code) | PPAPI list/run/get/summarize; CSV builder; local conversation tests |
| 8 Packaging + registration | done | README, CLAUDE.md, `.vscode/mcp.json`; stdio smoke test; registered in Claude Code (user scope, connected) and in the `C:\Files\Apps` workspace `.vscode/mcp.json`; public repo https://github.com/jgt87/copilot-studio-mcp |
| 9 Solution ALM (added 2026-09-05 on request) | done (code) | `cs_list_solutions`, `cs_list_connections`, `cs_describe_solution`, `cs_pull_solution`, `cs_create_deployment_settings`, `cs_pack_solution`, `cs_deploy_solution`; inventory parser + settings file verified offline |
| 10 DTAP comparison (added 2026-09-05 on request) | done (code) | `cs_snapshot_environment`, `cs_compare_snapshots`, `cs_compare_environments`; comparison logic unit-tested on synthetic snapshots; capture path (clone per agent, Dataverse reads) untested live |
| 12 Getting started (added 2026-09-05 on request) | done (code) | `cs_create_solution`, `cs_create_agent solutionName/createSolution` (init, pack, import, clone), `cs_generate_instructions` via `pac copilot model predict`; empty-solution manifest packs offline; import/clone/predict unverified live |
| 13 Day-two authoring + review (added 2026-09-06 on request) | done (code) | `cs_edit_topic`, `cs_edit_tool`, `cs_edit_knowledge`, `cs_remove_component`, `cs_delete_agent`, `cs_delete_solution`, `cs_review_agent`; new topic nodes: adaptive card (display and input), transfer, end conversation, scoped generative answers; node shapes from the schema, unverified live |
| 14 Portal drift (added 2026-09-06 on request) | done (code) | `cs_check_drift` quick (Dataverse component stamps) and full (temporary clone, three-way file diff); sync stamp `.mcs/cs-sync.json` written by clone / pull / push / init; drift preflight in `cs_push`; stamp placement verified with `pac copilot pack`; `botcomponent` query and formatted-value annotations unverified live |
| 15 Remaining pac commands (added 2026-09-07 on request) | done (code) | 29 declarative wrappers in `pacCommands.ts` (copilot create / extract-template / translations / quarantine, solution project + checker + versioning + upgrade + publish + add-component, pipeline list / deploy, service-principal connections, auth profiles, env list / who / fetch / select) plus `CPS_TOOLS` filter; flags read from pac 2.11.2 help, no command exercised live |
| 16 In-product guidance (added 2026-09-07 on request) | done | `cs_guide` with 10 walkthroughs, handshake instructions, 6 MCP prompts, next steps in `cs_init` / `cs_describe_workspace`; guides are checked against the registered tool names by `test/guide.test.js` |
| 17 Write policy (added 2026-09-07 on request) | done | `CPS_READ_ONLY` hides every environment-changing tool; `cs_pac` refuses non-read-only commands; `cs_init` reports the policy; `test/policy.test.js` keeps the list in step with the tools that declare `confirm` |
| 18 Cloud flows (added 2026-09-07 on request) | done (code) | `cs_list_flows`, `cs_get_flow`, `cs_set_flow_state`, `cs_update_flow` over Dataverse `workflow` rows (category 5); state pairs statecode 1/statuscode 2 and 0/1 from the documented option sets, definition swapped inside `clientdata`; unit-tested against recorded responses, never run against a tenant |
| 19 Flow creation and runs (added 2026-09-07 on request) | done (code) | `cs_create_flow` (workflow row: category 5, type 1, primaryentity none, `MSCRM.SolutionUniqueName` header, created switched off) and `cs_list_flow_runs` / `cs_get_flow_run` / `cs_run_flow` over the Power Automate Process Simple API (`api.flow.microsoft.com`, api-version 2016-11-01, Flow service scope). Both the create body and the run API are unit-tested against recorded shapes only; the run endpoints and their scope come from the service the portal calls, not from documented API reference |
| 20 Tenant administration (added 2026-09-07 on request) | done (code) | 29 `pac admin` wrappers, a `profile` argument on every pac-backed tool with `withPacProfile` switching and restoring the active profile, `cs_list_auth_profiles`, and `cs_backup_tenant` writing the tenant configuration to files; the backup is unit-tested end to end with a fake pac, but no admin command has been run against a tenant |
| 21 Agent response settings (added 2026-09-08 on request) | done (code) | `cs_update_agent` now writes `responseInstructions`, `defaultResponseMode`, `historyType`, `gptCapabilities` and the `aISettings` block, validated against the schema enumerations; README documents both files field by field. The portal-control-to-field mapping is inferred from the schema, not from a live round trip |
| 22 Flow builder (added 2026-09-08 on request) | done (code) | `flowBuilder.ts` composes a Logic Apps definition from a step spec (6 trigger kinds, 11 step types, nested control flow, runAfter chaining, connection reference collection); `cs_build_flow_definition` plus spec support in `cs_create_flow` and `cs_update_flow`. Unit-tested on the emitted JSON; no generated flow has been imported into an environment |
| 23 Question-shaped results (added 2026-09-08 on request) | done | `needs.ts` plus `cs_add_tool` (connector, operation, flow, prompt, agent), `cs_add_knowledge_source` (site, files, environment variable) and `cs_clone_agent` (which agent); choices come from the connector catalog or the offline seed of 1546 connectors, ranked against what the user asked for |
| 24 Conversation transcripts (added 2026-09-08 on request) | done (code) | `dataverseTranscripts.ts` reads the `conversationtranscript` table; `cs_list_transcripts`, `cs_get_transcript`, `cs_summarize_transcripts`, `cs_test_set_from_transcripts`. 10 unit tests over recorded responses cover the two content shapes, the outcome heuristics, the lookup fallback and the aggregates. **Unverified live:** the agent lookup column (three shapes tried in order), the `content` activity fields for topic and tool attribution, and whether the environment stores transcripts at all |
| 25 Long-running tools block the client (found live 2026-09-09) | fixed | `cs_pull_solution` does export (twice with the default `Both`), unpack, create-settings and a `pac copilot clone` per agent inside one MCP call. The server allows an hour per export; the MCP client caps the call long before that and the work is orphaned. Found on a real tenant during Phase G. Fixed in `7a7c762`: `jobs.ts` runs the work detached, the tool returns a job id immediately, and `cs_job_status` reports progress and the outcome. `background: true` is accepted by the solution and sync tools, chat, and every pac wrapper; outcomes are mirrored to disk so a server restart does not lose them. 8 tests in `test/jobs.test.js` |
| 26 First live run, phases A-F (2026-09-08, real tenant) | verified | Connector registry endpoint returns `source: registry` (was the offline seed's only alternative); `parseConnectionList` column layout correct; `pac copilot clone` layout and sync metadata as documented; **a portal-made agent validates clean, so `cs_validate` has no false positives against real content**; drift detect + `cs_push` conflict refusal work; the whole evaluation path works end to end including the portal CSV import format and the metric statuses (`Pass`/`Fail`/`Error`), which `summarizeRun` buckets correctly. Clone contains **no** evaluation test sets as YAML, so the CSV route stays (closes checklist item 7) |
| 27 cs_publish reported success on a failed publish (found live 2026-09-08) | fixed | `pac copilot publish` prints "Failed to publish" and exits 0, so `ok: code === 0` reported a successful publish of an agent that had not published. `PacRunOptions.failOnOutput` lets a command declare patterns that mean failure on a zero exit; `cs_publish` opts in with `PUBLISH_FAILED` and adds a hint. Opt-in per command so an unrelated "failed" in a log line cannot misfire. 4 tests |
| 28 Still unverified after the first live run | open | The Dataverse `listBots` path (`cs_list_agents` defaults to pac when pac is installed, so A3 exercised the pac path only); `cs_check_drift` quick mode with its `botcomponent` query and formatted-value annotations (needs `cs_login`, which the first run did not complete); **all of the transcript module** (the client was running a stale build, so the tools were absent from its tool list); `cs_chat` (timed out); phase G |
| 11 Tool catalog (added 2026-09-05 on request) | done (code) | `cs_list_connectors`, `cs_describe_connector`, `cs_list_prompts`; `cs_add_tool` covers all 10 schema TaskAction kinds (6 typed, 4 raw) with catalog-backed operation checks and input filling; offline seed of public connectors; registry endpoint (`api.powerapps.com` apis + `$expand=swagger`) unverified live |
| 29 Microsoft 365 agent catalogue (added 2026-09-09 on request) | done (code) | `graphPackages.ts` wraps the Copilot Package Management API (Microsoft Graph): `cs_list_org_agents`, `cs_get_org_agent`, `cs_block_org_agent`, `cs_reassign_org_agent`. The tenant-wide view of agents that Power Platform cannot give: `availableTo`, `deployedTo` and `isBlocked` answer whether a published agent reached anyone. Graph is a fourth resource with its own consent (`cs_login scope='graph'` / `'graph_write'`); reads use `CopilotPackages.Read.All` on `/v1.0`, block and reassign use `CopilotPackages.ReadWrite.All` and exist only on `/beta`. 13 unit tests over recorded responses cover filter building and quote escaping, paging and its cap, the write verbs and bodies, and the 403 licence hint. **Unverified live, and untestable without a Microsoft Agent 365 licence:** every response shape comes from the published reference, not a recorded call. Global cloud only. |
| 30 Bulk role assignment (added 2026-09-09 on request) | done (code) | `adminAssign.ts` + `cs_admin_assign_users`: a CSV or inline roster is expanded to one `pac admin assign-user` call per user-and-role pair and run under a single approval, because pac takes one user and one role per invocation. The dry run lists every pair, which is what makes one approval defensible; `cs_deploy_solution` is the precedent. Rows are independent by default (`continueOnError`), and the run is sequential because pac's active profile is machine-wide. Includes an RFC4180 CSV reader (quoted cells, doubled quotes, CRLF). 17 unit tests drive the whole roster with a fake pac, covering the parser, alias headers, role accumulation across rows, dedupe, argv, the plan, partial failure and stop-on-first-failure. **Unverified live:** no `pac admin` command has been run against a tenant (row 20). Two things a first run must check: role names are per-environment, and a user must exist in the environment before they can hold a role. |

## Verified offline (2026-09-05)

- `npm test`: 63 unit tests over dist/ (parsers, CSV, schema validation, every authoring tool on fixtures, solution inventory and settings, comparison, catalog, bootstrap, validation, day-two edits and review, drift).
- `pac copilot init` (default, minimal, cli-copilot) and `pac copilot pack` on the classic scaffold.
- `pac copilot pack` accepts settings + agent + topics only on an init workspace; every other folder
  is "Unsupported directory". Authoring tools report this via `layoutNote`.
- pac assemblies (`Microsoft.CopilotStudio.McsCore.dll`, `Microsoft.CopilotStudio.Sync.dll`) contain the
  sync layout names: `topics/`, `actions/`, `tools/`, `trigger/`, `knowledge/files`, `behaviors/`,
  `variables/`, `workflows`, `connectionreferences.mcs.yml`, `.mcs/conn.json`, `agent.sync.yaml`.
- `pac copilot pack` accepts extra files inside `.mcs/` and rejects a dotfile at the workspace root
  (`Unsupported file: .cs-sync.json`), 2026-09-06.
- `pac solution unpack` of the packed oracle solution (fixture `test/fixtures/unpacked-solution`, with
  synthetic flow, connection reference, environment variable and connector entries), `pac solution pack`
  of that fixture, and `pac solution create-settings` on the result (shape includes `CopilotAgents.AadGroupId`).

## Open questions to verify against a real environment

- Whether `pac copilot push` from a cloned workspace accepts the generated `knowledge/`, `actions/`,
  `trigger/`, `variables/` and `workflows/` files as written.
- Whether `pac copilot clone` projects evaluation test sets as YAML (`TestCaseComponent` / `EvaluationSet`
  exist in the schema). If so, `cs_create_test_set` can write them instead of a CSV.
- Exact `workflows/<name>/` format after a real clone.
- Evaluation API response field names for the summary buckets.
- `pac connection list` column layout (parser anchors on the GUID and connector name only).
- Whether `pac solution import` proceeds with the all-zero `AadGroupId` placeholder (treated as informational, not blocking).
- Whether `createFlow` needs more than category 5 / type 1 / primaryentity "none" (an `ownerid` or a
  `workflowidunique`, for instance) and whether the Power Automate run endpoints and the
  `service.flow.microsoft.com` scope work with the first-party client id.
- Which component query an environment accepts (`bots({id})/bot_botcomponent` navigation or the
  `_parentbotid_value` filter), whether `schemaname` follows `<agent>.<kind>.<Stem>` for every
  component kind (the drift-to-file mapping relies on it), and whether `pac copilot push` refuses
  when the server changed since the last pull.