---
name: deploy-test
description: "The last step of deploy: run on the version only the cases the platform must prove, report what is proven, and hand the owner the chat link. Holds a conversation of its own only when asked."
metadata:
  min-cli-version: "0.91.0"
  next: ""
---

# Deploy: test

Hand the owner a chat with the deployed version and tell them what is proven: before the pull request lands after a build, or before real users rely on a Recipe deployed from `main`. Run on the version only what build couldn't prove on this machine; the owner's own chat is the first real conversation.

## Before you start

deploy-connect has checked the CLI, the login and the scopes. Check the stage: the record names the version (runtime, version id, commit), its image is ready, and each server is recorded as connected, faked or missing. If not, go back.

```sh
introspection runtimes get <slug> --query image_build_status
```

<!-- rule: R-no-smoke-with-missing-binding -->

**Check the connections** for the version and environment the owner will use:

```sh
introspection bindings mcp list --runtime-id <version-id> --environment staging -o table
```

Never run a task, or hand over a chat, in an environment missing a connection the Recipe requires. A faked server reaches only a development task while `introspection dev --mcp` stays attached; a missing one means its cases are recorded as not run.

## 1. Decide what runs on the version

By default nothing does: build's local tasks already ran the agent, and improve writes its checks. Read whether anything changed since the commit build ran:

```sh
git diff --stat <the commit build ran> <the version's commit> -- . ':!docs'
```

Run cases only for what that leaves unproven:

- **Harbor cases,** and the `later` cases build left as needing the Runtime: as a cloud trial of the version.
- **Vitest cases,** when the Recipe or its evals changed since the baseline: once, locally at the version's commit.
- **Anything the owner asks** for, such as a conversation of your own with the agent.

<!-- rule: R-print-config-budget-approval -->

**Say what you'll run before the first paid run,** and get the owner's yes: the environment, the runtime and version, each agent and its model, which cases on which runner, and the number of trials and retries. More trials, reruns or a different model need a new yes.

## 2. Run the cases on the version

When step 1 leaves any, run them as [running cases on a version](references/cases-on-version.md) says: Harbor as a cloud trial (`eval-run`'s `cloud`), Vitest locally at the version's commit, reported as local and faked. Read results as `eval-run` says, and compare each with its baseline as `eval-compare` says.

## 3. A conversation, when the owner asks

Hold one only when the owner asks, as [the conversation](references/conversation.md) says: in staging when every server it needs is connected there, in development when one is faked. Give the owner its link from `introspection open --task "$T" --print`, and read it whole.

<!-- rule: R-verify-active-commit-fresh-task -->

**Once a change lands in production,** verify the active commit the same way: a fresh task in the change's situation, read whole, then the judgement trend against the one before. A drop is evidence for improve-select.

## 4. Turn the turns that matter into cases

When a conversation on the version (yours, or one the owner reports on) has a turn where the agent went wrong, or did something the owner wants kept, draft its case with `eval-case`, get the owner's approval (including that a real conversation may be committed), add it as a Harbor case, run it on the version and promote the new baseline, as [turning turns into cases](references/cases-from-turns.md) says. A new case that fails goes back to build-draft.

## 5. Report

In plain words: what ran where, what passed and failed and why, what's left for a person to judge, and every server that was faked. Until a conversation on the version shows the work done, it is deployed, not verified.

- **After a build:** write it into the pull request description and Part B ([the report](references/conversation.md#the-report)) and commit to `pr/<name>`. Push it with the branch the deploy's go-ahead covered; a change pinned in staging asks first, since a push builds a new version. Landing the pull request is the owner's decision.
- **A Recipe deployed from `main`** with no pull request: give the report in your message, and commit it only if the owner asks.

## 6. Close with the chat link

Check the project watches conversations for patterns, which improve works from:

```sh
introspection routines list
```

Then one short message, and stop: the agent is deployed, the report in a line, and the link to talk to it, from `introspection open --runtime <slug> --print` for a first deploy, or the version's link as `deploy-try`'s "Build the link" says for a change pinned in staging. Say whether conversations are watched; when not, that you can turn it on (`introspection routines update <id> --enable`) if they say so. Improving it starts once it has real conversations.

## Guardrails

<!-- rule: R-four-lifecycle-states -->

- **Keep the four claims apart.** A version is created when it exists, active when an environment serves it, deployed when it's the one selected there, and verified only when a conversation on it showed the work done from the commit you meant. A version existing, an environment serving it or a task's status alone is not verified.

<!-- rule: R-env-success-doesnt-promote -->

- **A pass in one place proves only that place.** A local run proves the package, a pull request version tested in staging proves that version in staging, and production serves what lands on `main`. Say which one you tested.

<!-- rule: R-dev-task-setup-failures-blockers -->

- **A failure before the agent runs is a blocker, not a failing case.** Setup, credentials, connections, the runner and the platform failing are reported as what blocked the run, never scored as the agent's.

<!-- rule: R-cloud-behavior-proven-in-cloud -->

- **Network behaviour is proven here, not locally.** A local run had open network; on the platform the agent reaches only the hosts its connections allow. A tool call that worked locally and fails here on network is a missing connection, not an agent fault.
- **Never change an approved case to make it pass,** and never tune the agent here: a fix goes back to build-draft on the same branch, and deploy runs again on its version.
- **No credentials, no connecting, no merging.** The owner signs in, connects and lands.

## Done when

- Every case build left for the platform ran on the version or has a written reason, and the Vitest cases ran at the version's commit when the Recipe or its evals changed since the baseline.
- Every result says where it ran and what was faked; the report calls the version verified only on a conversation that showed the work.
- Any turn that mattered from a conversation on the version is an approved Harbor case that passes or has a reason.
- The report is in the pull request description and Part B (or the message, for a Recipe deployed from `main`).
- The owner has the chat link and knows whether conversations are watched. Deploy ends here; improve starts from real conversations, best in a new session in the repository.
