--- name: test-traps description: Use when writing or changing tests, adding mocks, or tempted to add test-only methods to production code - prevents testing mock behavior, production pollution with test-only methods, and mocking without understanding dependencies usage: /test-traps auto_trigger: true --- # Test Traps ## Overview Tests must verify real behavior, not mock behavior. Mocks are a means to isolate, not the thing being tested. **Core principle:** Test what the code does, not what the mocks do. **Following strict TDD prevents these anti-patterns.** ## The Iron Laws ``` 1. NEVER test mock behavior 2. NEVER add test-only methods to production classes 3. NEVER mock without understanding dependencies ``` ## Anti-Pattern 1: Testing Mock Behavior **The violation:** ```typescript // ❌ BAD: Testing that the mock exists test('renders sidebar', () => { render(); expect(screen.getByTestId('sidebar-mock')).toBeInTheDocument(); }); ``` **Why this is wrong:** - You're verifying the mock works, not that the component works - Test passes when mock is present, fails when it's not - Tells you nothing about real behavior **your human partner's correction:** "Are we testing the behavior of a mock?" **The fix:** ```typescript // ✅ GOOD: Test real component or don't mock it test('renders sidebar', () => { render(); // Don't mock sidebar expect(screen.getByRole('navigation')).toBeInTheDocument(); }); // OR if sidebar must be mocked for isolation: // Don't assert on the mock - test Page's behavior with sidebar present ``` ### Gate Function ``` BEFORE asserting on any mock element: Ask: "Am I testing real component behavior or just mock existence?" IF testing mock existence: STOP - Delete the assertion or unmock the component Test real behavior instead ``` ## Anti-Pattern 2: Test-Only Methods in Production **The violation:** ```typescript // ❌ BAD: destroy() only used in tests class Session { async destroy() { // Looks like production API! await this._workspaceManager?.destroyWorkspace(this.id); // ... cleanup } } // In tests afterEach(() => session.destroy()); ``` **Why this is wrong:** - Production class polluted with test-only code - Dangerous if accidentally called in production - Violates YAGNI and separation of concerns - Confuses object lifecycle with entity lifecycle **The fix:** ```typescript // ✅ GOOD: Test utilities handle test cleanup // Session has no destroy() - it's stateless in production // In test-utils/ export async function cleanupSession(session: Session) { const workspace = session.getWorkspaceInfo(); if (workspace) { await workspaceManager.destroyWorkspace(workspace.id); } } // In tests afterEach(() => cleanupSession(session)); ``` ### Gate Function ``` BEFORE adding any method to production class: Ask: "Is this only used by tests?" IF yes: STOP - Don't add it Put it in test utilities instead Ask: "Does this class own this resource's lifecycle?" IF no: STOP - Wrong class for this method ``` ## Anti-Pattern 3: Mocking Without Understanding **The violation:** ```typescript // ❌ BAD: Mock breaks test logic test('detects duplicate server', () => { // Mock prevents config write that test depends on! vi.mock('ToolCatalog', () => ({ discoverAndCacheTools: vi.fn().mockResolvedValue(undefined) })); await addServer(config); await addServer(config); // Should throw - but won't! }); ``` **Why this is wrong:** - Mocked method had side effect test depended on (writing config) - Over-mocking to "be safe" breaks actual behavior - Test passes for wrong reason or fails mysteriously **The fix:** ```typescript // ✅ GOOD: Mock at correct level test('detects duplicate server', () => { // Mock the slow part, preserve behavior test needs vi.mock('MCPServerManager'); // Just mock slow server startup await addServer(config); // Config written await addServer(config); // Duplicate detected ✓ }); ``` ### Gate Function ``` BEFORE mocking any method: STOP - Don't mock yet 1. Ask: "What side effects does the real method have?" 2. Ask: "Does this test depend on any of those side effects?" 3. Ask: "Do I fully understand what this test needs?" IF depends on side effects: Mock at lower level (the actual slow/external operation) OR use test doubles that preserve necessary behavior NOT the high-level method the test depends on IF unsure what test depends on: Run test with real implementation FIRST Observe what actually needs to happen THEN add minimal mocking at the right level Red flags: - "I'll mock this to be safe" - "This might be slow, better mock it" - Mocking without understanding the dependency chain ``` ## Anti-Pattern 4: Incomplete Mocks **The violation:** ```typescript // ❌ BAD: Partial mock - only fields you think you need const mockResponse = { status: 'success', data: { userId: '123', name: 'Alice' } // Missing: metadata that downstream code uses }; // Later: breaks when code accesses response.metadata.requestId ``` **Why this is wrong:** - **Partial mocks hide structural assumptions** - You only mocked fields you know about - **Downstream code may depend on fields you didn't include** - Silent failures - **Tests pass but integration fails** - Mock incomplete, real API complete - **False confidence** - Test proves nothing about real behavior **The Iron Rule:** Mock the COMPLETE data structure as it exists in reality, not just fields your immediate test uses. **The fix:** ```typescript // ✅ GOOD: Mirror real API completeness const mockResponse = { status: 'success', data: { userId: '123', name: 'Alice' }, metadata: { requestId: 'req-789', timestamp: 1234567890 } // All fields real API returns }; ``` ### Gate Function ``` BEFORE creating mock responses: Check: "What fields does the real API response contain?" Actions: 1. Examine actual API response from docs/examples 2. Include ALL fields system might consume downstream 3. Verify mock matches real response schema completely Critical: If you're creating a mock, you must understand the ENTIRE structure Partial mocks fail silently when code depends on omitted fields If uncertain: Include all documented fields ``` ## Anti-Pattern 5: Integration Tests as Afterthought **The violation:** ``` ✅ Implementation complete ❌ No tests written "Ready for testing" ``` **Why this is wrong:** - Testing is part of implementation, not optional follow-up - TDD would have caught this - Can't claim complete without tests **The fix:** ``` TDD cycle: 1. Write failing test 2. Implement to pass 3. Refactor 4. THEN claim complete ``` ## Anti-Pattern 6: Timeout-Based Waiting **The violation:** ```typescript // Wait an arbitrary amount of time, then check await new Promise(r => setTimeout(r, 50)); const result = getResult(); expect(result).toBeDefined(); ``` ```python time.sleep(0.5) assert get_result() is not None ``` **Why this is wrong:** - Passes on fast machines, fails in CI under load (or vice versa). - Either too short (flaky) or too long (slow suite) — every value is wrong somewhere. - Hides the actual condition the test cares about. - A real bug in the async machinery looks identical to a "too-short sleep." **The fix — poll for the actual condition:** ```typescript await waitFor(() => getResult() !== undefined); const result = getResult(); expect(result).toBeDefined(); ``` | Scenario | Pattern | |----------|---------| | Wait for event | `waitFor(() => events.find(e => e.type === 'DONE'))` | | Wait for state | `waitFor(() => machine.state === 'ready')` | | Wait for count | `waitFor(() => items.length >= 5)` | | Wait for file | `waitFor(() => fs.existsSync(path))` | | Complex condition | `waitFor(() => obj.ready && obj.value > 10)` | Minimal generic poller: ```typescript async function waitFor( condition: () => T | undefined | null | false, description = 'condition', timeoutMs = 5000 ): Promise { const start = Date.now(); while (true) { const result = condition(); if (result) return result; if (Date.now() - start > timeoutMs) { throw new Error(`Timeout waiting for ${description} after ${timeoutMs}ms`); } await new Promise(r => setTimeout(r, 10)); } } ``` See `example.ts` for domain-specific helpers (`waitForEvent`, `waitForEventCount`, `waitForEventMatch`) drawn from a real debugging session. **Polling pitfalls:** - **Polling too fast** (`setTimeout(check, 1)`) wastes CPU. **Fix:** poll every ~10ms. - **No timeout** means a never-met condition hangs the suite forever. **Fix:** always include a timeout with a descriptive error. - **Stale data** — if you read state once before the loop, you'll never see updates. **Fix:** call the getter *inside* the loop. **When an arbitrary timeout IS correct:** ```typescript // Tool ticks every 100ms; observe 2 ticks of partial output. await waitForEvent(manager, 'TOOL_STARTED'); // condition first await new Promise(r => setTimeout(r, 200)); // then a documented, calculated wait // 200ms = 2 ticks at the documented 100ms cadence ``` Requirements: (1) wait for the triggering condition first, (2) the delay is based on a *known* interval (not a guess), (3) a comment explains why a sleep is correct here. **Real-world impact** (debugging session, 2025-10-03): 15 flaky tests across 3 files, pass rate 60% → 100%, suite 40% faster, race conditions gone. ## When Mocks Become Too Complex **Warning signs:** - Mock setup longer than test logic - Mocking everything to make test pass - Mocks missing methods real components have - Test breaks when mock changes **your human partner's question:** "Do we need to be using a mock here?" **Consider:** Integration tests with real components often simpler than complex mocks ## TDD Prevents These Anti-Patterns **Why TDD helps:** 1. **Write test first** → Forces you to think about what you're actually testing 2. **Watch it fail** → Confirms test tests real behavior, not mocks 3. **Minimal implementation** → No test-only methods creep in 4. **Real dependencies** → You see what the test actually needs before mocking **If you're testing mock behavior, you violated TDD** - you added mocks without watching test fail against real code first. ## Quick Reference | Anti-Pattern | Fix | |--------------|-----| | Assert on mock elements | Test real component or unmock it | | Test-only methods in production | Move to test utilities | | Mock without understanding | Understand dependencies first, mock minimally | | Incomplete mocks | Mirror real API completely | | Tests as afterthought | TDD - tests first | | Timeout-based waiting | Poll for the actual condition with `waitFor()` | | Over-complex mocks | Consider integration tests | ## Red Flags - Assertion checks for `*-mock` test IDs - Methods only called in test files - Mock setup is >50% of test - Test fails when you remove mock - Can't explain why mock is needed - Mocking "just to be safe" ## The Bottom Line **Mocks are tools to isolate, not things to test.** If TDD reveals you're testing mock behavior, you've gone wrong. Fix: Test real behavior or question why you're mocking at all.