---
description: Toil reduction—identify, measure, automate or eliminate. Manual, repetitive, automatable work with no enduring value; invest in reduction.
alwaysApply: false
---

# Toil Reduction

Guidelines for reducing operational toil.

## Core Principles

1. **Identify Systematically** - Measure toil (time, frequency) before reducing.
2. **Automate Ruthlessly** - If you do it more than twice, automate or eliminate.
3. **Eliminate First** - Sometimes the best fix is to stop doing the task (e.g. fix root cause).
4. **Invest Continuously** - Dedicate time (e.g. 20% or sprints) to toil reduction.

## What Is Toil?

- **Manual** - Requires human action (e.g. SSH to check, clickops).
- **Repetitive** - Same task again and again (e.g. weekly rotation, manual deploy).
- **Automatable** - A machine could do it (scripts, pipelines, self-service).
- **Tactical** - Reactive, not strategic; doesn’t improve the system long-term.
- **No enduring value** - Once done, no lasting benefit (e.g. one-off restarts that could be automated).
- **Scales with growth** - More services = more of the same work.

Not toil: writing automation, design, documentation, postmortems, on-call response (though reducing alert toil is toil reduction).

## Process

1. **Log** - Track recurring tasks; estimate time and frequency.
2. **Prioritize** - High time × frequency × pain; quick wins first.
3. **Automate or eliminate** - Script, pipeline, or remove (e.g. fix cause so task isn’t needed).
4. **Measure** - Time saved; track trend so toil doesn’t creep back.

## Examples

- **Manual restarts** → Health check + restart policy or runbook + one-click.
- **Repeated “why is X down?”** → Better alerting and runbook; or fix flaky dependency.
- **Manual cert rotation** → cert-manager or automated rotation.
- **Clickops provisioning** → Self-service or IaC.

## Definition of Done (Toil Work)

- [ ] Task identified and measured; automation or elimination done.
- [ ] Documented and shared; time saved tracked.
- [ ] No new manual step introduced for the same case.

## Common Pitfalls

- **Optimizing instead of eliminating** - Making a manual step faster instead of removing it.
- **No time set aside** - Toil reduction never gets priority; schedule it.
- **New toil** - New service or process adds manual steps; design for zero-toil from the start.