# What Is Harness Engineering? ## Definition **Harness Engineering** is the discipline of designing systems, environments, and feedback loops that enable AI coding agents to produce reliable, maintainable software. Instead of writing code line by line, harness engineers specify intent, constrain execution, and verify output -- shifting their role from author to architect and evaluator. The term is closely associated with Mitchell Hashimoto's staged model of AI tool adoption, where the fifth step is to **engineer the harness**: improve the surrounding system until common agent failures stop repeating. --- ## The Core Formula A coding agent is not just an AI model. It is the product of a model and its surrounding infrastructure: ``` coding agent = AI model + harness ``` The **model** provides raw intelligence -- pattern recognition, code generation, reasoning. The **harness** provides everything else: what the agent knows about your codebase, what it is allowed to do, how its work is verified, and how failures are caught. Improving the harness improves every task the agent performs. Improving a prompt improves one task. This is why harness engineering compounds. --- ## The Computer Architecture Analogy Harness engineering maps cleanly onto the layers of a computer system: | Computer Layer | Harness Layer | Role | |---------------|--------------|------| | **CPU** | AI Model | Raw computational power. Executes instructions. | | **RAM** | Context Window | Working memory. What the model can "see" right now. | | **Operating System** | Harness | Manages resources, enforces security, provides abstractions. | | **Application** | Agent Session | The actual work being performed within the OS constraints. | Just as no one runs applications directly on bare metal, no one should run AI agents without a harness. The operating system does not make the CPU faster, but it makes the CPU *useful* -- scheduling processes, managing memory, enforcing permissions, and providing stable interfaces. Your harness does the same for the model. It manages the context window (memory), enforces architectural constraints (permissions), provides knowledge about your codebase (file system), and defines verification procedures (system calls). --- ## Historical Context: The Evolution Software engineering has gone through several paradigm shifts in how humans interact with AI systems: ### Stage 1: Prompt Engineering (2022-2023) Writing careful instructions to get useful completions from language models. One-shot interactions. No memory, no verification, no persistence. The analogy: hand-crafting assembly language instructions. ### Stage 2: AI-Assisted Development (2023-2024) Autocomplete and chat-based coding tools. Early GitHub Copilot, ChatGPT, early AI-assisted IDEs. The human writes most code; the AI fills in gaps. The analogy: using a calculator to check arithmetic. ### Stage 3: Agentic Coding (2024-2025) AI agents that can read files, run commands, execute tests, and iterate. GitHub Copilot's agent mode and Coding Agent. The human provides intent; the agent produces implementations. The analogy: delegating to a junior developer. ### Stage 4: Harness Engineering (2025+) Systematic investment in the environment that makes agentic coding reliable. The focus shifts from writing good prompts to building good systems. The analogy: building an operating system rather than writing individual programs. This is not a replacement for understanding code. It is a replacement for *writing* code as the primary activity. --- ## Relocating Rigor One of the clearest ways to understand harness engineering is that rigor does not disappear when agents write code. It moves from raw authorship toward specification, constraint design, verification, and review. In traditional engineering, rigor lives in code review, careful typing, mental models of control flow, and years of pattern recognition applied at the keyboard. When an agent writes the code, that rigor must be captured elsewhere: - **Before execution**: In the specification, the constraints, the context provided. - **During execution**: In the linters, tests, type checkers, and structural rules that run automatically. - **After execution**: In human review of the output, where the engineer's taste and judgment are applied. The engineer's job changes from "write correct code" to "build systems that make incorrect code impossible to merge." This is a harder job, not an easier one. It requires deeper architectural thinking, better taste, and more systematic approaches to quality. --- ## Probabilistic Inside, Deterministic at the Edges The central design principle of harness engineering is: > Let the model be probabilistic inside. Make everything at the edges deterministic. AI models are fundamentally probabilistic. They produce different outputs on different runs. They hallucinate. They make mistakes. You cannot make the interior deterministic -- and you should not try. Instead, you make the *boundaries* deterministic: - **Input boundary**: The context is precise, structured, and correct. The agent knows exactly what it needs to know. - **Output boundary**: The verification is automated, comprehensive, and binary. The code either passes all checks or it does not. - **Execution boundary**: The agent can only modify what it should modify. File permissions, module boundaries, and architectural constraints limit the blast radius. This is the same principle behind a well-designed microservice. You do not control what happens inside a service (it might be Python, it might be Go, it might be held together with duct tape). You control the API contract at the boundary. Harness engineering applies this principle to the agent itself. --- ## How It Differs From Just Writing Rules Files Many teams create a `.github/copilot-instructions.md` or `.github/instructions/` file and consider the job done. This is prompt engineering, not harness engineering. The distinction: | Rules File Approach | Harness Engineering Approach | |--------------------|-----------------------------| | One large instruction file | Layered system of context, constraints, and verification | | Instructions are suggestions | Constraints are enforced by tooling | | Static, written once | Evolving, maintained by agents and humans | | Covers coding style | Covers architecture, dependencies, security, testing | | Fails silently when ignored | Fails loudly with actionable error messages | | No feedback loop | Every failure improves the harness | A rules file is a component of a harness. A harness is a *system* -- context engineering, architectural constraints, entropy management, verification loops, and continuous improvement all working together. --- ## The Three Pillars Harness engineering rests on three pillars, as articulated by Martin Fowler and Birgitta Böckeler: ### Pillar 1: Context Engineering Context engineering is the practice of curating, structuring, and delivering the right information to the AI agent at the right time. It answers the question: *What does the agent need to know to do this task well?* This includes: - **Static context**: Repository documentation, architecture decisions, coding conventions. - **Dynamic context**: Test output, linter errors, runtime logs, browser screenshots. - **Progressive disclosure**: A table of contents that points to detailed docs, not a wall of text dumped into every prompt. Context engineering is to harness engineering what memory management is to operating systems. Getting it wrong means the agent either lacks critical information (segfault) or is overwhelmed with irrelevant data (thrashing). See [Context Engineering](context-engineering.md) for the full guide. ### Pillar 2: Architectural Constraints Architectural constraints are automated, enforceable rules that prevent the agent from producing structurally incorrect code -- regardless of what the model "wants" to do. They answer the question: *What must always be true about this codebase?* This includes: - Custom linters with remediation instructions the agent can act on. - Structural tests that verify module boundaries, dependency directions, and layering rules. - Pre-commit hooks that catch violations before code leaves the developer's machine. - CI/CD checks that serve as a final gate. Architectural constraints are to harness engineering what permissions and access controls are to operating systems. They do not tell the agent *how* to do its work. They tell it what *not* to do. See [Architectural Constraints](architectural-constraints.md) for the full guide. ### Pillar 3: Entropy Management Entropy management is the practice of actively fighting the natural decay that occurs in any codebase over time, amplified by the volume of AI-generated code. It answers the question: *How do we keep the codebase healthy as it grows?* This includes: - Periodic "garbage collection" agents that clean up dead code, stale docs, and unused dependencies. - Quality grade tracking that measures codebase health over time. - Doc-gardening agents that keep documentation in sync with code. - Scheduled maintenance windows where agents address accumulated technical debt. Entropy management is to harness engineering what garbage collection is to runtime systems. Without it, the system works fine at first and then gradually degrades until it collapses. See [Entropy Management](entropy-management.md) for the full guide. --- ## Public Sources These sources are useful starting points for the ideas summarized in this document: - Mitchell Hashimoto, "My AI Adoption Journey": https://mitchellh.com/writing/my-ai-adoption-journey - OpenAI, "Harness Engineering: Leveraging Codex in an Agent-First World": https://openai.com/index/harness-copilot/ - Spotify Engineering, 2025 background coding agent series: https://engineering.atspotify.com/2025 - Anthropic, "Building effective agents": https://www.anthropic.com/engineering/building-effective-agents --- ## Who Is a Harness Engineer? A harness engineer is a software engineer who has shifted their primary activity from writing code to designing the systems that produce code. They: - **Design context** that agents can navigate and understand. - **Build constraints** that make entire categories of bugs impossible. - **Create verification loops** that catch what constraints miss. - **Maintain entropy** by scheduling cleanup and monitoring quality. - **Exercise taste** by reviewing agent output and encoding their judgment into the harness. This is not a new job title. It is an evolution of what software engineering has always been -- the relentless pursuit of leverage. Compilers gave us leverage over machine code. Frameworks gave us leverage over boilerplate. Harness engineering gives us leverage over the act of programming itself. --- ## Further Reading - [Core Principles](principles.md) -- The principles that guide harness engineering decisions. - [Getting Started](getting-started.md) -- Set up your first harness in 30 minutes. - [Context Engineering](context-engineering.md) -- Deep dive into the first pillar. - [Architectural Constraints](architectural-constraints.md) -- Deep dive into the second pillar. - [Entropy Management](entropy-management.md) -- Deep dive into the third pillar. - [Anti-Patterns](anti-patterns.md) -- Common mistakes and how to avoid them. - [Glossary](glossary.md) -- Definitions of key terms.