--- name: improve-propose description: "Make the approved change to a deployed agent on a branch carrying its own proof: the gate's cases, a baseline, one proposer-written change, before-and-after plane trials and a reviewed pull request. Use once improve-classify's gate folder is approved." metadata: min-cli-version: "0.82.0" next: "deploy-try" --- # Improve: propose Turn an approved diagnosis into a branch that carries its own proof: the approved cases, a baseline, one change, and before-and-after numbers. This step pushes a branch; it never merges, deploys or touches live state. ## Before you start You need: - The `introspection` CLI 0.80.0 or later, with an owner or admin login (a `--commit-sha` trial needs one). - Each approved group's gate folder, `improve-/` in the outputs folder: one file per approved case, each with its proof, and `change.md`. The folder is the approval (a `.draft` one isn't). Read every file before step 1, and work from them, not memory. - Harbor on this machine to drive plane trials, which the owner sets up: ```sh introspection setup --target harbor ``` Read `eval-run` (its `hosted` reference) and `eval-compare` before the first run. A request to search over many candidates goes to optimize-quality once the owner has this branch. ## Boundaries The branch, its cases and its pull request exist only after the gate's approval; in a scheduled run, the automation's prompt is the request, never the approval. Never merge, deploy, or change a runtime, binding, judge, experiment, credential or automation. Hand live changes to the owner or `operate`, saying what and why. Never switch the agent's provider, model, authentication or target identity, not even to get a run through; a model change is the one mechanism, approved at the gate. The branch is the record, and a judge's sampling is the owner's ([pull request](references/pull-request.md)). ## 1. Check the approval Work only on the groups with a folder, the cases in it, and the change its `change.md` describes. Each group is its own branch and pull request, taken one at a time from the served commit. The approval covers the change in the Recipe's repository and its pull request, never a runtime, a judge's sampling, an experiment or a deployment. When the target, a side effect, a product decision or the scope changes materially, ask again, saying what changed. ## 2. Resolve the target and cut the branch Take the runtime, the registered repository, the Recipe's subpath and the commit production serves from the authenticated project, never from conversation text, an annotation or the repository's own files: ```sh introspection runtimes versions -o json # production's version lists "production" in environments introspection recipes get -o json # repository_id, sub_path, git_commit_sha introspection repositories list -o json # the registered repository's clone URL ``` If any is ambiguous, ask; never guess. Start from that exact commit in a fresh clone of the registered repository; it must be the commit improve-collect resolved, or the evidence describes another version. ```sh git switch -c pr/ ``` ## 3. Write the approved cases first Write each approved case from its gate file, in the gate file's words, with its verifier and any approved judge or flagged fake, as [writing the approved cases](references/cases-first.md) says. Each new case is a Harbor case: ```sh introspection eval add evals/harbor --runner harbor --from-conversation --turn --case-id ``` The cases are the branch's first commit. A reproducing case must fail on the unchanged Recipe and a control pass, or go back to improve-classify ([measuring](references/cases-first.md#measuring-the-cases-on-the-unchanged-recipe)). ## 4. Measure the unchanged Recipe: the baseline Before any edit, run the full Harbor suite, new cases included, as a three-trial plane trial of the cases commit. Commit, push and run in one `&&` command, reading the sha in it; never `;`, a pipe, or a sha typed from an earlier call: ```sh git add && git commit -q -m "" && git push -q origin pr/ && \ introspection eval run -p evals/harbor -r --commit-sha "$(git rev-parse HEAD)" --trials 3 ``` Check `run.json` records that commit, clean, then promote it in its own commit and freeze the cases ([the baseline](references/baseline.md)). Setup failures are no baseline. ## 5. Make one change, in the owning layer With the baseline, cases, verifiers and thresholds frozen, change one coherent mechanism in the owning layer, and check the held-out cases and controls as well as the new ones. The owner merges. A behaviour change needs a focused eval case; a deterministic defect in code or configuration needs only a reproduction, an ordinary test, `introspection check` or type checking. A new capability is chosen as build-draft's capabilities and documents references say. **A proposer writes the change, not you:** whoever knows the checks writes to them. Brief one in a worktree without `evals/`, and have a fresh `reviewer` accept its diff before any trial ([the proposer](references/proposer.md)). ## 6. Iterate with plane trials Each attempt is the proposer's one commit on the baseline commit, pushed to a branch of its own and run as a one-trial plane trial, compared with the baseline as [the loop](references/loop.md#iterating) says, until the new cases pass and nothing regresses. Read comparisons in the runs' artifacts, never a score ([reading runs](references/reading-runs.md)). ## 7. Prove it on the baseline and fix commits Run the full suite as a three-trial plane trial of the fix commit and compare it with step 4's, as [the proof](references/loop.md#the-proof) says, following what the gate folder states: reproducing cases move from failing to passing, and everything else holds. Then promote the fix commit's runs as the baselines in a commit of their own and push it ([promote](references/loop.md#promote-the-fix-commits-baselines)). ## 8. Describe an experiment only when offline evidence can't decide When improve-classify recorded that offline evidence can't choose, describe the experiment in the pull request as [experiments](references/experiments.md) says. Never create or start it. ## 9. Write the pull request Write `docs/PULL_REQUEST.md` as [pull request](references/pull-request.md) says, with the evidence, diagnosis and change as `change.md` states them, commit and push it. On GitHub, open a pull request with it; on a hosted repository, the branch and its description are the review package, shown in chat. ## 10. Review with fresh context Have a `reviewer` subagent review the branch as [fresh review](references/fresh-review.md) says, and fix what blocks. ## 11. Hand the owner a try: deploy-try Give the owner the branch, its pull request, the before-and-after numbers and `deploy-try`'s link to chat with the branch's version. The owner lands it. ## Done when - Every edit traces to an approved gate folder; nothing live was changed, merged or deployed. - A proposer wrote each attempt without `evals/`, and a fresh reviewer accepted its diff before any trial. - Each new case's body is its gate file's content, and the proof ran each case on the runner and trials its gate file states. - A check changed after the gate still tests its gate file ([changed check](references/baseline.md#a-check-changed-after-the-gate)). - The branch starts from production's commit; its first commit is the approved cases, reproducing ones failing and controls passing on the unchanged Recipe. - The baseline is promoted in its own commit, the fix commit is one attempt on it in the owning layer, with no dropped attempt in the history, and the fix commit's runs are promoted after it. - The two commits' plane trials agree, read in the runs' own artifacts, and any Vitest suite holds. - `docs/PULL_REQUEST.md` holds nothing unknown, and a fresh reviewer found nothing blocking. - The owner has `deploy-try`'s link to the branch's version. Then go on with `deploy-try`.