---
name: improve-algorithm
description: Systematically improve an existing model or algorithm — profile, hypothesize, ablate, and measure accuracy/latency/memory. No guessing.
keywords: optimization, algorithm, performance, latency, accuracy, ablation, profiling
---

# Improve Algorithm

> This skill inherits the discipline of `superpowers:systematic-debugging` — measure before changing, change one thing at a time, measure again. Guessing is not a method; every change must be motivated by evidence from profiling or error analysis.

---

## When to use this skill

- Gate 3, for tickets specifically about improving an existing model or algorithm (accuracy, latency, or memory targets)
- Any time a model already exists and the goal is to make it better rather than build from scratch
- When performance regressions are reported in production

---

## Steps

### 1. Establish the current baseline

Run the existing system through the eval harness and record the exact number — metric, split, and seed. This is the floor that every proposed change must be measured against. If no eval harness exists yet, invoke `train-model` first to establish one.

### 2. Profile before optimizing

Do not form a hypothesis before you know where the bottleneck is.

- **Accuracy bottleneck:** run error analysis by slice and by class. Plot learning curves to distinguish bias (underfitting — train error high) from variance (overfitting — large train/val gap). Identify the failure pattern first.
- **Latency/memory bottleneck:** profile the hot path end-to-end. Tools: `cProfile` / `line_profiler` for Python; `torch.profiler` or `torch.autograd.profiler` for PyTorch; `tf.profiler` for TF. Measure throughput (samples/sec) and peak memory before and after every change.

Never skip this step. Optimizing the wrong part wastes time and can introduce regressions.

### 3. Form ONE hypothesis

Based on the profiling/error-analysis evidence, form exactly one hypothesis tied to the identified bottleneck. Examples:

- "Error analysis shows the model fails on short-text inputs → hypothesis: adding n-gram features reduces error on that slice"
- "Profiler shows 60% of latency in the preprocessing step → hypothesis: vectorizing the transform halves latency"
- "Learning curves show high variance → hypothesis: stronger L2 regularization closes the train/val gap"

State the hypothesis explicitly before writing any code.

### 4. Run a single ablation

Implement only the change implied by the hypothesis. Run the eval harness with the same seed, same split, same data version. Track the run in MLflow/wandb with a descriptive name and note the hypothesis in the run description. Compare to the baseline number directly.

### 5. Decide: keep or revert

Keep the change only if it:

- Beats the baseline meaningfully (respects the success threshold from Gate 1)
- Respects all constraints: accuracy/latency/memory trade-off stated explicitly
- Does not introduce a new regression in a different slice or metric

Revert if it does not satisfy the above. Log the failure reason; failed experiments are also information.

### 6. Repeat and document

Repeat steps 3–5 for the next bottleneck. Log every attempt — including reversions — so the improvement story is fully reproducible. The final state must be explainable: "we tried X (failed because Y), then tried Z (improved metric by W%)."

Output language: auto-detect from the ticket/task input — see `custom/rules/output-language.md` (Vietnamese input → Vietnamese output; otherwise English).

---

## Completion Checklist

- [ ] Baseline number recorded on the eval harness before any change
- [ ] Bottleneck identified by profiling (latency/memory) or error analysis (accuracy) — not guessed
- [ ] Each change is a single isolated ablation motivated by a stated hypothesis
- [ ] Every attempt (including failures) measured on the same harness and tracked with params/metrics
- [ ] Trade-offs between accuracy, latency, and memory stated explicitly for kept changes
- [ ] Kept changes beat the baseline by the success threshold
- [ ] Full improvement log available (hypothesis → result → decision for each attempt)