# Specification-Driven Development Specification-Driven Development (SDD) is a workflow where the specification is the primary engineering artifact and code is its expression. Instead of writing code that implies a design, you write a design that produces code. SDD is not new -- it echoes formal methods, design-by-contract, and literate programming. What is new is that AI agents can now *execute* specifications directly, making the workflow practical for everyday software engineering rather than just safety-critical systems. --- ## Why SDD Matters for Harness Engineering In traditional development, the specification is often informal (a Jira ticket, a Slack message, a conversation) and the code is the authoritative artifact. When you need to understand what the system does, you read the code. In harness engineering, this relationship inverts: | Traditional | Specification-Driven | |-------------|---------------------| | Code is the artifact, spec is supplementary | Spec is the artifact, code is supplementary | | Understanding requires reading implementation | Understanding requires reading specification | | Changes start with code edits | Changes start with spec edits | | Code review is the quality gate | Spec review is the quality gate | | Refactoring changes code, preserves behavior | Refactoring changes code, spec stays the same | When AI agents write the code, the specification becomes the durable artifact -- the thing humans maintain, review, and iterate on. Code becomes a derived artifact that can be regenerated. --- ## The Spec as Primary Artifact A specification in SDD is not a requirements document. It is a precise, structured description of what a component does, what its boundaries are, and how its correctness is verified. ### What a Good Spec Includes ```markdown # Spec: User Registration Service ## Purpose Handles new user registration, including validation, duplicate detection, and welcome email dispatch. ## Interface ### Input - `email`: string, valid email format, required - `password`: string, minimum 8 characters, required - `name`: string, 1-100 characters, required ### Output - Success: `User` object with `id`, `email`, `name`, `createdAt` - Failure: `ValidationError` with field-specific messages - Failure: `ConflictError` if email already registered ## Behavior ### Happy Path 1. Validate all input fields 2. Check if email already exists in user repository 3. Hash password using bcrypt with cost factor 12 4. Create user record in database 5. Dispatch welcome email asynchronously (failure does not block registration) 6. Return created user (without password hash) ### Validation Rules - Email: Must match RFC 5322 format - Password: Minimum 8 characters, at least one letter and one digit - Name: Non-empty after trimming whitespace, max 100 characters ### Error Handling - Database connection failure: Return 503 with retry-after header - Email dispatch failure: Log error, do not fail registration - Duplicate email: Return 409 with message "Email already registered" ## Dependencies - `UserRepository` for database access - `PasswordHasher` for bcrypt hashing - `EmailService` for welcome email dispatch - `Validator` for input validation ## Constraints - Must not import from API layer - Must not access database directly (use repository) - Must not log the password at any level - Must be stateless (no instance variables beyond injected dependencies) ## Verification - Unit tests for each validation rule - Unit tests for duplicate email detection - Integration test for full registration flow - Test that password is never logged (grep log output) - Test that email dispatch failure does not fail registration ``` ### What Makes This Different from a Requirements Doc 1. **Precision**: Every input, output, and behavior is specified. No ambiguity. 2. **Verifiability**: Every behavior has a corresponding test case. 3. **Constraints**: Architectural boundaries are explicit, not implicit. 4. **Dependencies**: What is needed is listed, so the agent can wire things up correctly. 5. **Error handling**: Every failure mode is specified, not left to the implementer. --- ## Planning vs. Execution Sessions SDD introduces a critical distinction between two types of agent sessions: ### Planning Sessions In a planning session, the human and agent collaborate to produce a specification. No code is written. The output is a spec document. **Characteristics**: - Focus on *what*, not *how*. - Human provides intent and constraints. - Agent asks clarifying questions and identifies edge cases. - Output is a reviewed, approved specification. - Duration: 15-30 minutes of focused thought. **Example planning prompt**: ``` I need to design a rate limiter for our API. Let's create a spec. Requirements: - Per-user rate limiting (not global) - Configurable limits per endpoint - Return 429 with Retry-After header when limit exceeded - Must work across multiple server instances Let's think through the design. What questions do you have? What edge cases should we consider? ``` ### Execution Sessions In an execution session, the agent takes a finished spec and implements it. The human is not actively involved -- they review the output after. **Characteristics**: - Focus on *how*, given a clear *what*. - Agent reads the spec and produces code. - Verification runs automatically (tests, lints, structural checks). - Human reviews the final output, not intermediate steps. - Duration: 5-30 minutes of agent work, 5-15 minutes of human review. **Example execution prompt**: ``` Implement the rate limiter specified in docs/specs/rate-limiter.md. Follow all constraints listed in the spec. Create the implementation, tests, and documentation. Run `make verify` before considering the task complete. ``` ### Why Separate Them? Planning and execution are different cognitive modes: - **Planning** requires creativity, judgment, and taste -- human strengths. - **Execution** requires consistency, thoroughness, and attention to detail -- agent strengths. Mixing them produces worse results. The agent tries to implement while the human is still deciding. The human micro-manages implementation details instead of thinking about design. --- ## Execution Plans as First-Class Artifacts Between a specification and code, there is an optional but valuable intermediate artifact: the execution plan. ### What Is an Execution Plan? An execution plan is a concrete, ordered list of steps the agent will take to implement a specification. It is more detailed than the spec (which says *what*) but less detailed than code (which says *how exactly*). ```markdown # Execution Plan: Rate Limiter ## Step 1: Create types (src/types/rate-limiter.ts) - Define RateLimitConfig interface - Define RateLimitResult type - Define RateLimitStore interface ## Step 2: Create store implementation (src/Project.Infrastructure/RateLimiting/RateLimitStore.cs) - Implement Azure Cache for Redis-backed RateLimitStore - Use sliding window algorithm - Handle Redis connection failures gracefully (Polly retry + circuit breaker) ## Step 3: Create rate limiter service (src/Project.Application/RateLimiting/RateLimiterService.cs) - Implement rate limit check logic (MediatR pipeline behavior or middleware-callable service) - Load config from `appsettings.json` bound through IOptions - Return RateLimitResult with remaining quota and reset time ## Step 4: Create middleware (src/Project.Web/Middleware/RateLimitMiddleware.cs) - ASP.NET Core middleware using IRateLimiterService - Set Retry-After header on 429 responses - Pass through on success ## Step 5: Write tests - Unit tests for RateLimiterService (tests/Project.UnitTests/RateLimiting/RateLimiterServiceTests.cs) - Integration tests for middleware (tests/Project.IntegrationTests/RateLimitMiddlewareTests.cs) - Tests for Redis failure handling (Polly fallback policy) ## Step 6: Update documentation - Add rate limiter to docs/architecture.md - Add rate limit config to docs/api-conventions.md - Update .github/copilot-instructions.md with new middleware reference ## Step 7: Verification - Run `make verify` - Verify structural tests pass (rate limiter uses approved layers) - Verify no new lint errors ``` ### Why Execution Plans Matter 1. **Reviewability**: A human can review 20 steps in 2 minutes. Reviewing 500 lines of code takes 30 minutes. Catching a structural mistake at the plan stage saves a full implementation cycle. 2. **Breakpoints**: If something goes wrong at Step 4, you do not have to redo Steps 1-3. The plan provides natural checkpoints. 3. **Parallelism**: Steps without dependencies can be executed in parallel. The plan makes this visible. 4. **Learning**: Execution plans document *how* your codebase grows. Over time, they become a library of patterns that inform future specs. --- ## GitHub Spec Kit and Similar Tools Several tools formalize the SDD workflow: ### GitHub Spec Kit A framework for writing machine-executable specifications that live in your repository. Specs are markdown files with structured sections that tools can parse. ### How to Use Spec Files Without Special Tools You do not need a formal framework. A `docs/specs/` directory with well-structured markdown files is sufficient: ``` docs/specs/ rate-limiter.md # Spec for rate limiter user-registration.md # Spec for user registration payment-processing.md # Spec for payment processing _template.md # Template for new specs ``` ### Spec Template ```markdown # Spec: [Component Name] ## Status [ ] Draft | [ ] In Review | [ ] Approved | [ ] Implemented ## Purpose [What this component does and why it exists.] ## Interface ### Input [Typed inputs with validation rules.] ### Output [Typed outputs for success and failure cases.] ## Behavior ### Happy Path [Ordered steps for the normal case.] ### Edge Cases [What happens in unusual situations.] ### Error Handling [How each failure mode is handled.] ## Dependencies [What this component needs from other components.] ## Constraints [Architectural rules this component must follow.] ## Verification [How to verify the implementation is correct.] ## Open Questions [Unresolved design decisions, if any.] ``` --- ## The Four Principles of SDD ### 1. Specify Before You Build Never start an execution session without a reviewed spec. This does not mean the spec must be perfect -- it means the *intent* must be clear enough that the agent (or a different human) could implement it without further questions. **Test**: Can you hand the spec to a different agent session and get a compatible implementation? If not, the spec is insufficient. ### 2. Separate Planning from Execution Use different sessions, different modes, and ideally different prompts for planning versus execution. Planning sessions should ask questions and explore alternatives. Execution sessions should follow the plan and report progress. **Test**: Does your execution session contain the word "should we" or "what if"? If so, you are mixing planning and execution. ### 3. Make Specs Verifiable Every claim in the spec should have a corresponding check -- a test, a lint rule, a structural test, or a manual verification step. If a behavior is specified but not verified, it will eventually drift. **Test**: For every sentence in the Behavior section, is there a corresponding item in the Verification section? ### 4. Evolve Specs with Code When code changes, the spec changes too. When a spec changes, the code is re-evaluated. They are two views of the same thing. Treat spec-code drift as a defect, not a convenience. **Test**: Does your CI check that spec references match the code they describe? If not, add a doc-gardening check. --- ## Best Use Cases for SDD SDD is not for every task. It adds overhead (writing the spec) that pays off only when the task is complex enough. ### SDD Is Worth It For | Use Case | Why | |----------|-----| | New modules or services | The design matters more than the code. A bad design implemented perfectly is still bad. | | API contracts | APIs are boundaries. Getting boundaries right is critical and expensive to change later. | | Complex business logic | Edge cases are where bugs hide. Specifying them forces you to think through them. | | Cross-cutting features | Features that touch multiple modules need coordination that specs provide. | | Safety-critical components | Authentication, authorization, payment processing. The cost of getting it wrong is high. | ### SDD Is Overkill For | Use Case | Why | |----------|-----| | Bug fixes with clear root cause | The spec is the bug report. Just fix it. | | Style changes or refactors | The existing code is the spec. The behavior should not change. | | Adding a field to an existing model | Too small to warrant a spec. Follow the existing pattern. | | Exploratory prototyping | You do not know what you want yet. Spec after you learn, not before. | ### The Decision Rule If the task would take you more than 2 hours to implement manually, write a spec first. If it would take less than 30 minutes, skip the spec. For tasks in between, use judgment -- but when in doubt, write the spec. The time spent specifying is almost always less than the time spent debugging an unspecified implementation. --- ## Workflow Summary ``` 1. Identify task | 2. Is it complex enough for SDD? | | Yes No | | 3. Planning 3. Direct execution session with agent | 4. Write spec | 5. Review spec (human approval) | 6. Write execution plan (optional but recommended) | 7. Execution session (agent implements spec) | 8. Automated verification (tests, lints, structural checks) | 9. Human review (check output against spec) | 10. Merge ``` --- ## Further Reading - [What Is Harness Engineering?](what-is-harness-copilot.md) -- How SDD fits into the discipline. - [Context Engineering](context-engineering.md) -- Specs are a form of context. - [Architectural Constraints](architectural-constraints.md) -- Constraints verify what specs describe. - [Principles: Corrections Are Cheap](principles.md#5-corrections-are-cheap-waiting-is-expensive) -- When to spec vs. when to just start.