--- name: optimize-latency description: "Find the fastest version of a deployed agent that holds the quality floor and cost ceiling, and review the whole branch with fresh context. Use when the owner wants an agent faster without losing quality, or after the cost search." metadata: min-cli-version: "0.73.0" next: "deploy-connect" --- # Optimize: latency Find the fastest version that keeps the quality floor and the cost ceiling the earlier steps set, then hand the branch, with every phase's measured trade-off, to deploy. The search is optimize-quality's, run with latency as the objective and the earlier results as hard gates. ## Before you start - What optimize-cost hands over: the contract, the ledger, `pr/` with the accepted changes and the promoted baseline, the quality floor and the cost ceiling. When the owner asked only for a faster agent, run optimize-quality's steps 1 to 5 first and take the reference's pass rate per case as the floor and its cost per completed task as the ceiling. - Read optimize-quality's references to the contract and the search before the first candidate, and `optimize-cost`'s `knobs` and `starting-points` references before writing a hypothesis: they list what a candidate may change and what has been measured about models and tool modes. - improve-collect's metrics reference has the query document for the production check. ## Not this step Quality and cost are the earlier steps'; the search's mechanics are optimize-quality's, and a run and a comparison are `eval-run`'s and `eval-compare`'s. Pushing, pinning staging and testing are deploy-create's and deploy-test's. ## Measuring latency A local run records its duration only for the whole run (`started_at`, `completed_at` in `run.json`), so a latency objective compares whole runs over the same case set and trials, with quality held as a gate. For per-conversation latency, use the metrics `conversations` view, one row per conversation: its `duration_ns` gives end-to-end percentiles, and its `total_cost_usd` the cost per conversation. The `spans` view is per model call, so its latency leaves out tool time and the turns between calls. Judge latency per completed task, as cost is: a candidate that fails a task fast isn't faster. Record quality, argument errors and cost beside latency for every candidate; a change that cuts argument errors usually cuts retried calls and turns, and so latency. ## 1. Add the latency objective to the contract The objective is the median run duration over the selection set across the contract's trials, with the slow tail (the slowest trial per case) reported beside it. The gates are every case of the quality floor and a cost per completed task no higher than the ceiling. Add them and this phase's budget in money to `/contract.md`. ## 2. Measure the reference: the cost winner Reuse its full selection run when the contract still matches. Read where the time goes in its trajectories: requests per case, sequential tool calls that don't depend on each other, tools declared but never called, and turns spent recovering from argument errors. ## 3. Choose the hypotheses From that evidence and `optimize-cost`'s `knobs`: fewer tools, fewer turns, parallel tool calls, a mode with fewer lookups or argument errors, a faster model where cost allows. One change per proposer. ## 4. Search, choose and accept Do this exactly as optimize-quality's steps 6 to 8 say, with this phase's objective and both gates. Apply the winner to `pr/`, promote its baseline in its own commit, and clean up. ## 5. Check recent production traffic Fewer tools can quietly break a rare case the suite doesn't hold. List the tools the served version actually called in recent production conversations (improve-collect's evidence reading, read-only). Replay each turn that called a tool the winner removed or hides against the winner (`eval-run`'s "Replay a production conversation"). A replay is exploration: when one breaks, drop the change that broke it, or keep it and say so for the owner to decide, and take the turn to improve as a case. ## 6. Review the whole branch with fresh context Start a `reviewer` subagent, as improve-propose does, given only the repository's path on `pr/` and these questions: - Does each change do what `docs/PULL_REQUEST.md` says and nothing more? - Is any change fitted to the cases rather than the users? - Does every number tie to a run id? Fix what blocks, rerun what the fix touches, and update the pull request. ## 7. Write the trade-off and hand the branch to deploy Finish `docs/PULL_REQUEST.md`: for each phase, the reference and the winner's quality, cost per completed task and latency, with run ids, trials and the held-out result, the production check, and every change that needs a person's review before it ships (capability changes, and rewrites of `SYSTEM.md` or a skill's text). Commit it. Tell the owner the measured trade-off in a few lines, with the search's results page built as optimize-cost's results-page reference says. Then go on with deploy's path for a change, starting at deploy-connect: push the branch, never `main`, pin staging to its version and test it there; the owner decides by landing the pull request. Once it runs, compare its cost and latency with the served version by `recipe_git_commit_sha`, as `optimize-cost` says. ## Done when - The latency objective and both gates were added to the contract, and every candidate was measured under it. - The winner held the quality floor and the cost ceiling, was accepted once on the held-out set, and the production traffic it could break was replayed; or the Recipe is unchanged and the owner was told why. - A fresh-context reviewer found nothing blocking on `pr/`, and `docs/PULL_REQUEST.md` states each phase's trade-off with run ids and the changes a person must review. Then go on with `deploy-connect`.