purpose: Core lesson on agentic review systems and specialized agent focus
type: educational-content
course: tactical-ai-consulting
lesson: 06
topic: agent-review-documentation
last-updated: 2025-09-30
Lesson 6 explores how letting agents focus on specific, narrow domains creates more effective AI systems. This covers agentic review systems, automated documentation generation, and the strategic use of specialized agents for maximum impact.
# Lesson 06: Let Your Agents Focus - Review and Documentation
## Core Principle: Specialization Over Generalization
**The Problem:**
Generic AI agents trying to do everything become masters of nothing. Context overload, decision paralysis, and inconsistent outputs plague general-purpose systems.
**The Solution:**
Create narrow, focused agents that excel at single domains. Let them build expertise through repetition and focused training.
```
┌─────────────────────────────────────────────────┐
│ GENERALIST AGENT (WEAK) │
│ ┌──────────┬──────────┬──────────┬──────────┐ │
│ │ Review │ Document │ Test │ Deploy │ │
│ │ (Poor) │ (Poor) │ (Poor) │ (Poor) │ │
│ └──────────┴──────────┴──────────┴──────────┘ │
└─────────────────────────────────────────────────┘
↓ TRANSFORM TO ↓
┌──────────────┐ ┌──────────────┐ ┌──────────────┐
│ REVIEWER │ │ DOCUMENTER │ │ TESTER │
│ (Expert) │ │ (Expert) │ │ (Expert) │
│ │ │ │ │ │
│ - Security │ │ - API docs │ │ - Unit │
│ - Quality │ │ - Guides │ │ - Integration│
│ - Standards │ │ - Examples │ │ - E2E │
└──────────────┘ └──────────────┘ └──────────────┘
```
## Agentic Review Systems
### What is an Agentic Review?
A review system where specialized AI agents evaluate work products against domain-specific criteria, providing consistent, expert-level feedback.
### Key Characteristics
1. **Domain Expertise**: Agent trained on specific review criteria
2. **Consistency**: Same standards applied every time
3. **Rapid Feedback**: Near-instant review cycles
4. **Learning Loop**: Reviews improve with accumulated knowledge
### Review Agent Architecture
```xml
Security Reviewer
OWASP Top 10, secure coding, vulnerability detection
Backend API security only
- JWT validation present
- Token expiration enforced
- Refresh token rotation implemented
- All inputs sanitized
- Type checking enforced
- Length limits applied
- Parameterized queries only
- ORM usage validated
- No string concatenation in queries
- Issue description
- Severity level
- Location (file:line)
- Remediation steps
- Overall security score (0-100)
- Pass/fail determination
- Blocking issues count
```
### Review Types and Agents
| Review Type | Agent Focus | Evaluation Criteria |
|-------------|-------------|---------------------|
| Security Review | Vulnerabilities, attack vectors | OWASP Top 10, CVE patterns |
| Code Quality | Maintainability, patterns | Clean code, SOLID principles |
| Performance | Speed, efficiency | Time complexity, resource usage |
| Documentation | Completeness, clarity | Coverage, examples, accuracy |
| Architecture | Design patterns, scalability | System design, dependencies |
| Accessibility | WCAG compliance | A11y standards, semantic HTML |
### Implementation Pattern
```python
# Review Agent Implementation Example
class SecurityReviewAgent:
def __init__(self):
self.expertise = ["OWASP", "secure-coding", "crypto"]
self.severity_levels = ["critical", "high", "medium", "low"]
def review(self, code_path: str) -> ReviewReport:
"""Perform security review of codebase"""
findings = []
# 1. Authentication checks
auth_issues = self._check_authentication(code_path)
findings.extend(auth_issues)
# 2. Input validation
input_issues = self._check_input_validation(code_path)
findings.extend(input_issues)
# 3. SQL injection vectors
sql_issues = self._check_sql_patterns(code_path)
findings.extend(sql_issues)
# 4. Cryptography usage
crypto_issues = self._check_crypto(code_path)
findings.extend(crypto_issues)
# 5. Generate report
return self._generate_report(findings)
def _check_authentication(self, path: str) -> list:
"""Check authentication implementation"""
issues = []
# Pattern: JWT without expiration
if self._find_pattern(path, r"jwt\.encode.*(?!exp)"):
issues.append({
"severity": "critical",
"type": "authentication",
"message": "JWT created without expiration",
"remediation": "Add 'exp' claim to JWT payload"
})
return issues
def _generate_report(self, findings: list) -> ReviewReport:
"""Generate structured review report"""
critical_count = sum(1 for f in findings if f["severity"] == "critical")
return ReviewReport(
findings=findings,
score=self._calculate_score(findings),
passed=critical_count == 0,
blocking_issues=critical_count
)
```
### Multi-Agent Review Workflow
```
┌──────────────┐
│ Code Submit │
└──────┬───────┘
│
├─────────────────────────────────────────────┐
│ │
↓ ↓
┌──────────────┐ ┌──────────────┐
│ Security │ │ Quality │
│ Reviewer │ │ Reviewer │
└──────┬───────┘ └──────┬───────┘
│ │
└─────────────────┬───────────────────────────┘
↓
┌──────────────┐
│ Aggregator │
│ Agent │
└──────┬───────┘
│
↓
┌──────────────┐
│ Final │
│ Report │
└──────────────┘
```
## Agentic Documentation Generation
### Why Agents for Documentation?
**Problems with manual documentation:**
- Always out of sync with code
- Inconsistent formatting
- Missing examples
- Poor coverage
**Agent advantages:**
- Generated from source truth (code)
- Always current
- Consistent structure
- Can generate multiple formats
### Documentation Agent Types
#### 1. API Documentation Agent
```xml
Generate complete API documentation from code
- HTTP method
- Route path
- Parameters (query, path, body)
- Response schemas
- Status codes
- Authentication requirements
- Field names and types
- Validation rules
- Relationships
- Example values
OpenAPI 3.0 specification with examples
Human-readable API guide with curl examples
Postman collection for testing
```
**Example Output:**
```markdown
# User Authentication API
## POST /api/auth/login
Authenticate user and receive JWT token.
### Request Body
```json
{
"email": "user@example.com",
"password": "securePassword123"
}
```
### Response (200 OK)
```json
{
"token": "eyJhbGciOiJIUzI1NiIs...",
"user": {
"id": "123",
"email": "user@example.com",
"role": "user"
}
}
```
### Error Responses
| Status | Description |
|--------|-------------|
| 401 | Invalid credentials |
| 422 | Validation error |
| 500 | Server error |
### Example (curl)
```bash
curl -X POST https://api.example.com/auth/login \
-H "Content-Type: application/json" \
-d '{"email":"user@example.com","password":"pass123"}'
```
```
#### 2. Code Documentation Agent
Generates inline documentation and guides from code structure:
```python
class CodeDocAgent:
def document_function(self, function_node):
"""Generate comprehensive function documentation"""
return {
"signature": self._extract_signature(function_node),
"purpose": self._infer_purpose(function_node),
"parameters": self._document_parameters(function_node),
"returns": self._document_return(function_node),
"examples": self._generate_examples(function_node),
"errors": self._document_errors(function_node)
}
def _infer_purpose(self, node):
"""Infer function purpose from name and implementation"""
# Analyze function name
name_parts = node.name.split('_')
# Analyze operations performed
operations = self._extract_operations(node)
# Generate natural language description
return self._generate_description(name_parts, operations)
```
#### 3. Architecture Documentation Agent
Creates system-level documentation:
```xml
System architecture and design patterns
- High-level architecture diagram
- Component relationships
- Data flow
- External dependencies
- Per-component documentation
- Responsibilities
- Interfaces
- Configuration
- Infrastructure requirements
- Deployment topology
- Scaling considerations
- Monitoring setup
- Architecture diagrams (C4 model)
- Sequence diagrams
- Entity relationship diagrams
- Infrastructure diagrams
```
### Documentation Generation Workflow
```
┌─────────────┐
│ Code Base │
└──────┬──────┘
│
├──────────────────────────────────┐
│ │
↓ ↓
┌──────────────┐ ┌──────────────┐
│ API Doc │ │ Code Doc │
│ Agent │ │ Agent │
│ │ │ │
│ - Endpoints │ │ - Functions │
│ - Schemas │ │ - Classes │
│ - Examples │ │ - Modules │
└──────┬───────┘ └──────┬───────┘
│ │
└─────────────┬────────────────────┘
↓
┌──────────────┐
│ Arch Doc │
│ Agent │
│ │
│ - System │
│ - Components │
│ - Diagrams │
└──────┬───────┘
│
↓
┌──────────────┐
│ Publishing │
│ Agent │
│ │
│ - Format │
│ - Deploy │
│ - Version │
└──────────────┘
```
## Tactic #6: Create Specialized Review Agents
### The Tactic
Instead of relying on human reviewers or generic AI assistants, create domain-specific review agents that become experts in their narrow field.
### Implementation Steps
**Step 1: Identify Review Domains**
```yaml
review-domains:
- security
- performance
- accessibility
- code-quality
- documentation
- architecture
- compliance
- user-experience
```
**Step 2: Define Domain Expertise**
For each domain, specify:
- Evaluation criteria (what to check)
- Severity levels (how critical)
- Remediation patterns (how to fix)
- Knowledge base (reference materials)
**Step 3: Create Agent Specifications**
```yaml
agent:
name: accessibility-reviewer
domain: web-accessibility
expertise:
- WCAG 2.1 AA standards
- ARIA patterns
- Semantic HTML
- Keyboard navigation
- Screen reader compatibility
evaluation-criteria:
- alt-text-present: critical
- heading-hierarchy: high
- form-labels: critical
- color-contrast: high
- keyboard-accessible: critical
tools:
- axe-core
- pa11y
- lighthouse
output:
format: structured-report
includes:
- violations
- warnings
- best-practices
- remediation-steps
```
**Step 4: Implement Agent Logic**
```python
class AccessibilityReviewAgent:
def __init__(self):
self.standards = self._load_wcag_standards()
self.severity_map = {
"no_alt_text": "critical",
"poor_contrast": "high",
"missing_label": "critical"
}
def review_html(self, html_path: str) -> AccessibilityReport:
"""Perform comprehensive accessibility review"""
violations = []
# Check 1: Images have alt text
img_issues = self._check_images(html_path)
violations.extend(img_issues)
# Check 2: Form inputs have labels
form_issues = self._check_forms(html_path)
violations.extend(form_issues)
# Check 3: Heading hierarchy
heading_issues = self._check_headings(html_path)
violations.extend(heading_issues)
# Check 4: Color contrast
contrast_issues = self._check_contrast(html_path)
violations.extend(contrast_issues)
# Check 5: Keyboard navigation
keyboard_issues = self._check_keyboard(html_path)
violations.extend(keyboard_issues)
return self._generate_report(violations)
def _check_images(self, path: str) -> list:
"""Verify all images have descriptive alt text"""
issues = []
images = self._parse_images(path)
for img in images:
if not img.has_alt:
issues.append({
"type": "no_alt_text",
"severity": "critical",
"element": img.tag,
"line": img.line_number,
"remediation": f"Add alt attribute: 
"
})
elif len(img.alt) < 3:
issues.append({
"type": "insufficient_alt",
"severity": "high",
"element": img.tag,
"remediation": "Provide more descriptive alt text"
})
return issues
```
**Step 5: Integrate into Workflow**
```yaml
# CI/CD Pipeline Integration
stages:
- build
- review
- test
- deploy
review:
parallel:
- security-review:
agent: security-reviewer
blocking: true # Fail build on critical issues
- accessibility-review:
agent: accessibility-reviewer
blocking: true
- performance-review:
agent: performance-reviewer
blocking: false # Warning only
- quality-review:
agent: code-quality-reviewer
blocking: false
aggregation:
agent: report-aggregator
outputs:
- consolidated-report.md
- github-pr-comment
- slack-notification
```
### Benefits of Specialized Review Agents
**1. Consistency**
- Same criteria applied every time
- No reviewer fatigue
- No subjective interpretation
**2. Speed**
- Near-instant feedback
- Parallel review execution
- No scheduling delays
**3. Expertise**
- Deep domain knowledge
- Up-to-date with standards
- Pattern recognition from thousands of reviews
**4. Scalability**
- Review every PR, every commit
- No bottlenecks
- Cost-effective at scale
**5. Learning**
- Agents improve over time
- Accumulate pattern library
- Adapt to project-specific needs
### Anti-Patterns to Avoid
**DON'T:**
- Create one "super reviewer" agent
- Make agents too broad in scope
- Skip human review entirely for critical systems
- Ignore agent feedback without investigation
- Let agents block without clear remediation paths
**DO:**
- Create focused, specialized agents
- Combine agent + human review for critical paths
- Continuously refine agent criteria
- Provide clear, actionable feedback
- Make remediation easy to understand
## Case Study: Security Review Agent in Production
### Context
A fintech startup needed to ensure all API endpoints met security standards before deployment. Manual security reviews took 3-5 days and often missed common issues.
### Solution
Implemented specialized security review agent:
```python
class FinTechSecurityReviewer:
CRITICAL_CHECKS = [
"authentication_required",
"authorization_verified",
"input_sanitized",
"sql_parameterized",
"sensitive_data_encrypted",
"rate_limiting_applied",
"audit_logging_present"
]
def review_endpoint(self, endpoint_code):
"""Review API endpoint against fintech security standards"""
findings = []
for check in self.CRITICAL_CHECKS:
result = getattr(self, f"_check_{check}")(endpoint_code)
if not result.passed:
findings.append(result)
# Fintech-specific checks
findings.extend(self._check_pci_compliance(endpoint_code))
findings.extend(self._check_data_retention(endpoint_code))
findings.extend(self._check_audit_trail(endpoint_code))
return SecurityReport(
endpoint=endpoint_code.path,
findings=findings,
passed=self._all_critical_passed(findings),
compliance_score=self._calculate_compliance(findings)
)
```
### Results
- Review time: 5 days → 5 minutes
- Critical issues caught: 60% → 98%
- False positives: < 5%
- Developer satisfaction: High (clear, immediate feedback)
- Security incidents: Reduced by 85%
## Action Items
1. **Identify Your Review Domains**
- List all areas requiring expert review
- Prioritize by risk and frequency
- Define expertise requirements
2. **Create First Specialized Agent**
- Pick highest-impact domain
- Define evaluation criteria
- Implement and test
- Integrate into workflow
3. **Measure and Iterate**
- Track agent accuracy
- Collect developer feedback
- Refine criteria
- Expand to new domains
4. **Build Agent Library**
- Document agent specifications
- Share across projects
- Create reusable components
- Establish best practices
## Summary
**Key Takeaways:**
1. **Specialization beats generalization** - Focused agents excel at specific tasks
2. **Consistency at scale** - Agents apply same standards every time
3. **Rapid feedback loops** - Instant reviews accelerate development
4. **Expertise accumulation** - Agents improve with each review
5. **Multi-agent systems** - Combine specialized agents for comprehensive coverage
**The Formula:**
```
Specialized Agent = Narrow Domain + Expert Criteria + Consistent Evaluation + Clear Output
```
**Next Steps:**
- Implement your first review agent
- Measure impact on quality and velocity
- Expand to additional domains
- Build multi-agent review systems
- Lesson 11: Domain-Specific Agents (Advanced specialization)
- agent-design-patterns.md (Implementation architectures)
- specialization-strategy.md (When and how to specialize)