# Anti-Patterns in Harness Engineering Anti-patterns are practices that seem reasonable but produce poor results at scale. Each one described here has been observed in real teams working with AI coding agents. Recognizing them early saves weeks of frustration. --- ## 1. The "1,000-Page Instruction Manual" ### What It Looks Like A `.github/copilot-instructions.md` or `AGENTS.md` file that is 500+ lines long, covering every possible scenario, coding convention, edge case, and preference. Often the result of multiple team members adding instructions over time without anyone removing old ones. ```markdown # .github/copilot-instructions.md (anti-pattern: 847 lines) ## Coding Style - Use camelCase for variables - Use PascalCase for classes - Use SCREAMING_SNAKE_CASE for constants - Use kebab-case for file names - Indent with 2 spaces - Maximum line length 100 characters - Always use trailing commas - Always use semicolons - Use single quotes for strings [... 60 more lines of style rules ...] ## API Design [... 120 lines of API conventions ...] ## Database [... 80 lines of database patterns ...] ## Testing [... 100 lines of testing instructions ...] ## Error Handling [... 90 lines of error handling rules ...] ## Security [... 70 lines of security requirements ...] ## Deployment [... 50 lines of deployment notes ...] ``` ### Why It Fails - **Context window saturation**: Every token of this file is loaded into every session, displacing tokens that could be used for task-specific reasoning. - **Attention dilution**: When everything is emphasized, nothing is emphasized. The agent cannot distinguish critical rules from nice-to-haves. - **Instruction conflict**: Long files accumulate contradictions over time. Line 47 says "always use async/await" and line 312 says "prefer callbacks for stream processing." - **Staleness**: Nobody maintains an 847-line instruction file. Sections rot silently. ### The Fix Reduce the top-level file to ~100 lines. Move everything else to linked docs in `docs/`. Use linter configurations for style rules -- they enforce rather than suggest. See [Context Engineering: Progressive Disclosure](context-engineering.md#progressive-disclosure). --- ## 2. Auto-Generating Config Files Instead of Hand-Crafting ### What It Looks Like Using an AI agent to generate the `.github/copilot-instructions.md` or `AGENTS.md` file for your project. "Analyze this codebase and generate a comprehensive configuration file." ### Why It Fails - **Generic output**: The generated config describes what the codebase *is*, not what it *should be*. It captures current patterns including bad ones. - **No taste**: The config lacks the judgment calls that make a harness valuable. It says "this project uses ASP.NET Core" but not "we chose minimal APIs over MVC controllers because..." - **No prioritization**: Everything gets equal weight. The generated config does not know that your layering rules matter more than your variable naming preferences. - **Instant staleness**: The config describes the codebase at the moment of generation. It diverges immediately. ### The Fix Hand-craft your agent configuration. It should reflect your *intent* for the codebase, not its current state. A human's judgment about what matters most, what the top 5 architectural rules are, and what common mistakes to avoid is the entire value of the config file. Use generation as *input* to your thinking, not as the final artifact. "Generate a summary of this codebase" is a fine research step. Adopting the output as your config file is the anti-pattern. --- ## 3. Ignoring Agent Failures Instead of Engineering Fixes ### What It Looks Like The agent produces incorrect output. The human fixes it manually and moves on. The same failure happens next week with a different task. The human fixes it again. No harness improvement occurs. ``` Week 1: Agent uses wrong import pattern → Human fixes manually Week 2: Agent uses wrong import pattern → Human fixes manually Week 3: Agent uses wrong import pattern → Human fixes manually Week 4: "These AI agents are unreliable" ``` ### Why It Fails - **No compounding**: Each manual fix helps exactly one task. A harness fix helps every future task. - **Growing frustration**: The same failures recur, eroding trust in the tooling. - **Blame misattribution**: The failure is attributed to "the model" when it is actually a harness gap. ### The Fix Every agent failure is a harness failure. When the agent makes a mistake, ask: "What context, constraint, or verification would have prevented this?" Then implement that improvement. | Failure | Harness Fix | |---------|------------| | Wrong import pattern | Add structural test for imports | | Inconsistent naming | Add linter rule | | Missing error handling | Add to spec template | | Wrong module placement | Improve architecture docs | | Missed edge case | Add test template with edge case examples | See [Principles: Agent Failure Is Harness Failure](principles.md#10-agent-failure-is-harness-failure). --- ## 4. Over-Relying on Models Without Deterministic Guardrails ### What It Looks Like Trusting the model to "just get it right" without any verification. No tests, no linters, no structural checks. "The model is smart enough to follow the instructions." ### Why It Fails - **Probabilistic drift**: Models are probabilistic. They produce different output on different runs. Without deterministic checks, quality varies randomly. - **Confident errors**: Models produce incorrect code with the same confidence as correct code. Without verification, there is no signal to distinguish the two. - **Silent degradation**: Small errors compound. An unused import today, a boundary violation tomorrow, a security hole next week. ### The Fix Apply the "probabilistic inside, deterministic at the edges" principle. Let the model generate freely, but verify deterministically: ``` Model generates code → Type checker verifies types → Linter verifies style → Structural tests verify architecture → Unit tests verify behavior → Integration tests verify contracts ``` If any check fails, the agent iterates. The human only reviews code that has passed all deterministic checks. --- ## 5. Context Overload and Instruction Rot ### What It Looks Like Two related anti-patterns that often occur together: **Context overload**: Providing so much context that the model's attention is diluted across irrelevant information. **Instruction rot**: Instructions that were once relevant but are no longer accurate, remaining in the config file and misleading the agent. ```markdown # .github/copilot-instructions.md ## Important: Use the PaymentGateway v2 API We migrated to v2 in March 2024. The old v1 endpoints are deprecated. Use PaymentGatewayV2Client from src/clients/payment_v2.py. # Meanwhile, in the actual codebase: # - PaymentGatewayV2Client was renamed to PaymentClient 6 months ago # - The file was moved from src/clients/ to src/infra/payments/ # - The agent follows the stale instruction and creates a file in the wrong place ``` ### Why It Fails - **Stale instructions produce stale code**: The agent faithfully follows instructions that describe a codebase that no longer exists. - **Contradictions cause unpredictable behavior**: When the instruction says one thing and the code says another, the agent picks one arbitrarily. - **Information overload degrades quality**: Models have limited attention. Filling it with stale context displaces useful context. ### The Fix 1. **Review agent config quarterly**: Every 3 months, read your entire agent config and delete anything that is no longer accurate. 2. **Test your docs**: Run the doc-gardening script from [Entropy Management](entropy-management.md#doc-gardening-agents) regularly. 3. **Date your instructions**: Add a `Last verified: YYYY-MM-DD` header to docs sections so you know which ones are likely stale. 4. **Prefer code over docs**: When possible, encode rules in linters and tests instead of docs. Code cannot rot without failing CI. --- ## 6. Skipping Verification Loops ### What It Looks Like Running the agent, glancing at the output, and merging without running tests, linters, or any verification. "It looks right." ### Why It Fails - **Looks right is not right**: AI-generated code often *looks* correct at a glance but contains subtle errors that only surface at runtime or under edge cases. - **Normalizing skipped checks**: Once you skip verification once and nothing breaks, you skip it again. And again. Until something breaks badly. - **No feedback signal**: Without verification, the agent gets no feedback on its output. It cannot self-correct because it does not know it failed. ### The Fix Make verification automatic and non-optional: 1. **Pre-commit hooks**: Run before every commit, no bypass. 2. **CI checks**: Run on every PR, block merging on failure. 3. **Agent instructions**: "Run `make verify` and fix all failures before considering the task complete." The verification loop is not overhead. It is the core mechanism by which harness engineering works. Without it, you are just generating code and hoping. --- ## 7. "Draw the Owl" Mega-Sessions ### What It Looks Like Giving the agent a massive task in a single session: "Build the entire authentication system" or "Refactor the whole API layer." The meme reference: "Step 1: Draw two circles. Step 2: Draw the rest of the owl." ### Why It Fails - **Context window exhaustion**: Long sessions accumulate conversation history that displaces useful context. By the end, the agent may have forgotten critical details from the beginning. - **Error compounding**: A mistake in step 3 of 20 propagates through all subsequent steps. By step 20, the output is far from correct. - **Difficult to review**: A 2,000-line diff is nearly impossible to review meaningfully. Errors hide in volume. - **No checkpoint recovery**: If the session fails at step 15, there is no way to resume from step 14. You restart from scratch. ### The Fix Break large tasks into small, verifiable units. Each unit should: - Take 5-15 minutes of agent work. - Produce a diff small enough to review in 5 minutes. - Pass all verification independently. - Be committable on its own. ``` # Instead of: "Build the authentication system" # Do: 1. "Create the User type and UserRepository interface" (commit) 2. "Implement the UserRepository with database access" (commit) 3. "Create the AuthService with login and registration" (commit) 4. "Add password hashing with bcrypt" (commit) 5. "Create JWT token generation and validation" (commit) 6. "Add ASP.NET Core authentication middleware in Program.cs" (commit) 7. "Add auth to the /users endpoint" (commit) 8. "Write integration tests for the full auth flow" (commit) ``` Each step is verifiable, reviewable, and recoverable. --- ## 8. Premature Harness Optimization ### What It Looks Like Spending two weeks building an elaborate harness -- custom linters, structural tests, quality grade dashboards, entropy management agents -- before the team has run a single agent session. ### Why It Fails - **Building for hypothetical problems**: You do not know what problems your harness needs to solve until agents start working in your codebase. - **Over-engineering constraints**: Rules that seem important in theory may be irrelevant in practice. Rules you never considered may be critical. - **Delayed feedback**: The harness only improves through the feedback loop of agent failures. Delaying agent use delays harness improvement. ### The Fix Start with the minimum viable harness: 1. A ~100-line agent config file. 2. A working test suite. 3. A linter. 4. `make verify` that runs all three. Run agent sessions immediately. Every failure tells you what to add next. Build the harness *in response to* real problems, not in anticipation of hypothetical ones. See [Getting Started](getting-started.md) for the minimal setup and [Principles: Corrections Are Cheap](principles.md#5-corrections-are-cheap-waiting-is-expensive) for why iteration beats planning. --- ## 9. Treating the Harness as a One-Time Setup ### What It Looks Like Building the harness once, deploying it, and never touching it again. "We set up .github/copilot-instructions.md and the linter last quarter. We are good." ### Why It Fails - **The codebase evolves**: New modules, new patterns, new dependencies. A harness that was accurate 3 months ago may be misleading today. - **New failure modes emerge**: As agents tackle more complex tasks, they encounter failure modes that the original harness did not anticipate. - **Team knowledge grows**: You learn things about what works and what does not. That knowledge must be encoded in the harness. - **Models improve**: New model capabilities change what agents can do. Your harness should evolve to take advantage of them. ### The Fix Treat the harness as a living system: - **Weekly**: Review agent failures and add one harness improvement. - **Monthly**: Run quality grades and doc gardening. Prune stale instructions. - **Quarterly**: Major harness review. Update architecture docs, re-evaluate technology choices, assess constraint coverage. The best harnesses are maintained partly by agents themselves -- doc-gardening agents that verify references, quality-grade agents that track health, and garbage-collection agents that clean up entropy. --- ## 10. One-Shotting Complex Projects ### What It Looks Like Attempting to build an entire application or implement all features in a single agent session. **Symptoms:** - Agent context is exhausted mid-implementation - Half-implemented features left undocumented - Next session wastes significant time guessing what the previous session did - No clear progress boundaries or commit points **Why it happens:** - Overestimating what fits in a single context window - No structured feature breakdown - No session lifecycle protocol **The fix:** - Break work into one feature per session - Use a structured feature list (JSON) to track scope - Follow the session lifecycle protocol: start with health check, end with commit + progress update - See [Long-Running Agents](long-running-agents.md) for the complete pattern --- ## 11. Premature Victory Declaration ### What It Looks Like The agent declares the project "complete" after implementing some features while missing others, or marks features as passing without end-to-end verification. **Symptoms:** - Agent outputs "All done!" or "Project complete!" while features are missing - Features marked as implemented but broken when tested - Feature requirements modified to match broken implementations **Why it happens:** - No structured, immutable feature list to verify against - No end-to-end testing requirement - Markdown-based tracking is easily modified by the model **The fix:** - Use JSON feature list with read-only requirements (agents may only change the `passes` field) - Require end-to-end verification before marking any feature complete - Include directive: "It is unacceptable to remove or edit feature requirements" - Browser automation (Puppeteer MCP) for UI verification --- ## Anti-Pattern Diagnostic Checklist Use this checklist when agent output quality is lower than expected: | Symptom | Likely Anti-Pattern | Fix | |---------|-------------------|-----| | Agent ignores important rules | 1,000-page manual (rules lost in noise) | Prune config to 100 lines | | Agent follows outdated patterns | Instruction rot | Audit and update docs | | Same failures repeat weekly | Ignoring failures | Add constraint for each failure | | Agent output "looks right" but has bugs | Skipping verification | Add tests and linter rules | | Large PRs with many issues | Mega-sessions | Break into small tasks | | Agent uses wrong libraries | No technology guardrails | Add dependency allowlist | | Code quality degrades over time | One-time harness | Schedule maintenance | | Agent config feels inadequate | Auto-generated config | Hand-craft with judgment | | Reviews take hours | Over-reliance on models | Add deterministic checks | | Context exhausted mid-project | One-shotting complex projects | Break into one feature per session, use feature list | | Agent says "done" but features missing | Premature victory declaration | Use JSON feature list with immutable requirements | --- ## The Meta Anti-Pattern: Not Having a Harness The most common anti-pattern is not recognized as one: using AI coding agents without any harness at all. No config file. No structural tests. No verification loop. Just prompts into a chat window. This works for trivial tasks. It fails catastrophically for anything that matters: - No consistency between sessions. - No institutional knowledge captured. - No compounding improvement. - Every task starts from zero. If you are reading this and do not have a harness, start today. The [Getting Started Guide](getting-started.md) takes 30 minutes. The return on that 30-minute investment compounds with every task you run through the harness. --- ## Further Reading - [Principles](principles.md) -- The principles these anti-patterns violate. - [Getting Started](getting-started.md) -- The minimum viable harness. - [Context Engineering](context-engineering.md) -- Fixes for context-related anti-patterns. - [Entropy Management](entropy-management.md) -- Fixes for decay-related anti-patterns.