# Principles of Harness Engineering These are the core principles that guide harness engineering decisions. They are not rules to memorize but mental models to internalize. Each principle has been validated across teams building production software with AI agents. --- ## 1. Give Agents a Map, Not a Manual **Bad**: A 2,000-line instruction file covering every possible scenario. **Good**: A 100-line table of contents that points agents to detailed docs when they need them. This is the principle of **progressive disclosure**. Agents, like humans, work best when they get information at the moment of relevance — not all at once. Your top-level configuration files (`.github/copilot-instructions.md` and `AGENTS.md`) should be a map: ```markdown # .github/copilot-instructions.md ## Architecture See docs/architecture.md for system design and module boundaries. ## Testing Run `make test` before committing. See docs/testing-guide.md for patterns. ## API Design All endpoints follow docs/api-conventions.md. Use the openapi spec at api/spec.yaml. ``` The agent reads the map. When it encounters a task involving API design, it reads the referenced document. This keeps the context window focused and avoids the "1,000-page manual" anti-pattern. **Why this works**: Context windows are finite. Every irrelevant token displaces a relevant one. A map costs ~100 tokens and points to thousands of tokens of detail available on demand. --- ## 2. Repository as Single Source of Truth Everything the agent needs to know should live in the repository. Not in a wiki. Not in Confluence. Not in a team member's head. In the repo, version-controlled alongside the code. This means: - Architecture decisions live in `docs/decisions/` or ADR files. - Coding conventions live in linter configs and style guides. - Module boundaries live in structural test files. - Dependency policies live in lockfiles and allowlists. **Why this works**: The repository is the one thing every agent session has access to. External knowledge requires complex integrations, goes stale, and cannot be verified by CI. Repository-resident knowledge is always available, always versioned, and always testable. **Corollary**: If you explain something to an agent in a chat message and it works, encode that explanation in the repository. Chat messages are ephemeral. Repository docs are permanent. --- ## 3. Enforce Invariants, Not Implementations Tell agents *what must be true*, not *how to make it true*. An invariant is a property that must hold regardless of implementation. An implementation instruction is a step-by-step recipe. **Invariant (good)**: "All database access must go through the repository layer. No service may import from `internal/db` directly." **Implementation instruction (bad)**: "When accessing the database, first create a repository struct, then implement the interface, then register it in the dependency injection container, then..." Agents are good at figuring out *how* once they know *what*. They are bad at following long procedural instructions without drifting. Give them the constraint and let the linter verify compliance. ```python # structural_test.py def test_no_direct_db_imports(): """Services must not import from internal/db directly.""" for source_file in glob("services/**/*.py"): content = read(source_file) assert "from internal.db" not in content, ( f"{source_file} imports directly from internal/db. " f"Use the repository layer instead. See docs/data-access.md" ) ``` **Why this works**: Invariants are verifiable by machines. Implementation instructions require human judgment to evaluate compliance. Machines are cheaper and more consistent. --- ## 4. Favor Boring Technology When an agent picks a library, framework, or pattern, it should pick the most widely-used, well-documented, battle-tested option -- not the newest or cleverest. "Boring" means: - Extensive training data exists for the model (higher quality generations). - Stack Overflow answers exist for edge cases. - The library is stable and unlikely to introduce breaking changes. - New team members (human or AI) can onboard quickly. Encode this in your harness: ```markdown # docs/technology-choices.md ## Approved Dependencies - HTTP server: ASP.NET Core (C#) / Giraffe (F#) / Express (TypeScript) - ORM: Entity Framework Core (C#) / Dapper for read-heavy paths (C#) / Prisma (TypeScript) - Testing: xUnit + FluentAssertions (C#) / Expecto (F#) / Pester (PowerShell) / Vitest (TypeScript) ## Not Approved - Any NuGet package outside the `Microsoft.*`, `Azure.*`, or top-100-downloaded set without an ADR - Any library released less than 12 months ago - Any TypeScript package without bundled type definitions When in doubt, use the BCL or a `Microsoft.*` / `Azure.*` package. ``` **Why this works**: Models generate higher-quality code for well-known libraries because they have seen more examples. Novel libraries increase hallucination rates and produce subtly incorrect code that passes a cursory review. --- ## 5. Corrections Are Cheap, Waiting Is Expensive Do not spend 30 minutes crafting the perfect prompt when you could spend 2 minutes writing a rough prompt, let the agent produce output, and spend 5 minutes correcting. The cost structure of agent-driven development inverts traditional assumptions: - **Generating code**: Near-zero cost (seconds, pennies). - **Reviewing code**: Low cost (minutes, human attention). - **Waiting for perfect instructions before starting**: High cost (blocks all downstream work). This does not mean "be sloppy." It means **iterate fast**. Ship a rough spec, review the output, refine the harness, repeat. Each iteration makes the harness better for all future tasks. **The feedback loop**: ``` 1. Agent produces output 2. Human reviews output 3. Human identifies systematic issue 4. Human improves harness (context, constraint, or verification) 5. All future agent sessions benefit ``` **Why this works**: You cannot predict every failure mode in advance. Real failures are more informative than hypothetical ones. The harness improves fastest when it processes real work. --- ## 6. Human Taste Captured Once, Enforced Continuously Every time you make a judgment call during code review -- "I prefer early returns," "this function is too long," "don't use abbreviations in variable names" -- that is taste. Taste is valuable. Taste that lives only in your head is expensive to apply (you must review every line). Taste that is encoded in the harness is free to apply (it runs automatically). **Capture taste as**: - Linter rules with custom messages. - Style guide documents referenced in the agent config. - Structural tests for architectural preferences. - Example files showing "this is what good looks like." ```yaml # .eslint.config.js (example taste capture) rules: max-lines-per-function: - error - max: 50 message: "Functions should be under 50 lines. Extract helpers. See docs/style.md#function-length" no-abbreviations: - warn - message: "Use full words in variable names. 'usr' -> 'user', 'btn' -> 'button'" ``` **Why this works**: Taste is the one thing AI models do not have. But taste can be *described* and *enforced*. Once described, it applies to every line of code the agent produces, forever, without fatigue. --- ## 7. Constrain the Runtime to Enable Autonomy Paradoxically, more constraints lead to more agent autonomy. When the boundaries are clear and enforced, you can let agents run with less supervision. Consider two scenarios: **Scenario A (few constraints)**: The agent can install any dependency, modify any file, use any pattern. You must review every line carefully because anything could go wrong. **Scenario B (many constraints)**: The agent can only use approved dependencies, can only modify files in its assigned module, and must pass all structural tests. You review for logic and correctness only because the harness handles everything else. In Scenario B, the agent has *more effective autonomy* because you trust it enough to let it work without hovering. Constraints are enabling, not restrictive. **Practical application**: - Restrict which directories the agent can modify. - Restrict which dependencies the agent can install (use an allowlist). - Require all changes to pass CI before review. - Use feature flags so agent-produced changes can be deployed without risk. **Why this works**: Trust requires verification. Automated verification requires constraints. Constraints enable trust. Trust enables autonomy. --- ## 8. Parse, Don't Validate (At Boundaries) When data crosses a boundary -- user input, API response, file read, agent output -- parse it into a validated type immediately. Do not pass raw strings through the system and validate them later. This principle, borrowed from Alexis King's influential blog post, applies directly to harness engineering: - Agent output is untrusted input. Parse and validate it immediately. - Configuration files should be typed schemas, not freeform YAML. - Test output should be structured (JSON, TAP) not unstructured (console logs). ```typescript // Bad: validate later function processAgentOutput(output: string) { // ... 50 lines later ... if (!output.includes("export default")) { throw new Error("Missing default export"); } } // Good: parse at the boundary interface ModuleOutput { defaultExport: string; namedExports: string[]; tests: TestCase[]; } function parseAgentOutput(output: string): ModuleOutput { // Parse and validate immediately // Return typed, validated structure // Everything downstream works with ModuleOutput, not string } ``` **Why this works**: Boundaries are where errors originate. Catching them immediately reduces debugging time from hours to seconds. Typed, validated structures prevent entire categories of downstream bugs. --- ## 9. Optimize for Agent Legibility Write your codebase so that agents can understand it, not just humans. This does not mean "dumb it down." It means: - **Name things explicitly**. `UserRegistrationService` over `URS`. `handlePaymentFailure` over `hpf`. - **Use conventional project structures**. `src/`, `tests/`, `docs/`, `config/`. Not clever custom layouts. - **Keep files focused**. One concept per file. Small files are easier to fit in context windows. - **Write doc comments on public interfaces**. Agents use these as primary navigation tools. - **Use explicit types**. TypeScript over JavaScript. Python type hints over bare Python. Agents reason better with type information. ```python # Low legibility class S: def p(self, d: dict) -> dict: r = {} for k, v in d.items(): if self._c(k): r[k] = self._t(v) return r # High legibility class DataSanitizer: def process_input(self, raw_data: dict[str, Any]) -> dict[str, SanitizedValue]: """Process raw input data, keeping only allowed fields and transforming values.""" sanitized = {} for field_name, field_value in raw_data.items(): if self._is_allowed_field(field_name): sanitized[field_name] = self._transform_value(field_value) return sanitized ``` **Why this works**: Agents navigate code by reading it, just like humans. But agents lack the implicit context that humans build over months of working in a codebase. Explicit naming, conventional structure, and clear documentation compensate for this gap. --- ## 10. Agent Failure Is Harness Failure When an agent produces incorrect output, the instinct is to blame the model. Resist this instinct. Ask instead: **What could the harness have done to prevent this?** Every agent failure is an opportunity to improve the harness: | Agent Failure | Harness Improvement | |--------------|-------------------| | Used wrong dependency | Add dependency allowlist | | Violated module boundary | Add structural test | | Produced inconsistent style | Add linter rule | | Missed edge case | Add test template | | Misunderstood architecture | Improve docs/architecture.md | | Generated dead code | Add entropy management agent | This creates the **iterative feedback loop** that is the engine of harness engineering: ``` Agent fails → Human diagnoses root cause → Harness improves → Future agents succeed ``` Over time, the harness accumulates the team's collective judgment, experience, and taste. It becomes the institutional memory of how software should be built in this codebase. **Why this works**: Models improve slowly (quarterly releases). Harnesses improve fast (every sprint). Investing in the harness gives you compounding returns that are independent of model improvements -- and when the model *does* improve, a better harness amplifies that improvement. --- ## Principles Summary | # | Principle | One-Liner | |---|----------|-----------| | 1 | Map, not manual | Progressive disclosure beats information dumps | | 2 | Repo as truth | If it is not in the repo, the agent does not know it | | 3 | Invariants over implementations | Say what, not how | | 4 | Boring technology | Well-known beats cutting-edge | | 5 | Corrections over waiting | Iterate fast, refine later | | 6 | Taste captured once | Encode judgment in automation | | 7 | Constraints enable autonomy | More guardrails, less supervision | | 8 | Parse at boundaries | Validate early, type everything | | 9 | Agent legibility | Write for your least-contexted reader | | 10 | Failure improves harness | Every bug is a harness bug | --- ## Further Reading - [What Is Harness Engineering?](what-is-harness-copilot.md) -- Core concepts and definitions. - [Getting Started](getting-started.md) -- Apply these principles in practice. - [Anti-Patterns](anti-patterns.md) -- What happens when you violate these principles.