# Enterprise Guide (Level 3)

*Setup time: 1-2 weeks. Requires organizational coordination.*

This guide covers building organization-wide harness engineering infrastructure for companies with multiple teams, diverse codebases, and enterprise requirements around security, compliance, and governance. The goal is to make harness engineering a first-class part of your engineering platform.

---

## Prerequisites

- Multiple teams have completed [Level 2 (Team)](small-team.md) adoption
- You have an engineering platform team (or someone willing to act as one)
- Leadership buy-in for AI coding tool adoption at scale
- Existing CI/CD infrastructure and developer experience tooling

## Step 1: Organization-Wide Harness Strategy

### Assess the current state

Before building enterprise infrastructure, understand where you are:

1. **Survey your teams.** Which teams use AI coding tools? Which tools? What works, what doesn't?
2. **Catalog existing harnesses.** Collect every AGENTS.md, .github/copilot-instructions.md, and similar file across your org. What patterns emerge?
3. **Identify shared pain points.** What mistakes are multiple teams making independently?
4. **Map your constraints.** Security policies, compliance requirements, deployment standards, architectural guidelines.

### Define the strategy layers

| Layer | Scope | Owner |
|-------|-------|-------|
| Organization defaults | Apply to all repos | Platform team |
| Domain standards | Apply to a business domain (payments, user-facing, internal tools) | Domain leads |
| Team conventions | Apply to a specific team's repos | Team lead |
| Project specifics | Apply to a single repo | Project maintainers |

Each layer inherits from the one above. A project's effective harness is the combination of all four layers.

### The "golden path" model

Your harness becomes the new golden path -- the blessed way to build software at your company. Just as you provide service templates, CI pipelines, and deployment tooling, you now provide harness templates that encode your organization's standards.

```
service-template/
  AGENTS.md                                # Org defaults + service-type specifics
  .github/
    copilot-instructions.md                # Org-wide Copilot context, plus service-specific addenda
    instructions/                          # Scoped rules (testing, security, observability)
    prompts/                               # Org-blessed slash commands
    agents/                                # Org-blessed custom agents
  .pre-commit-config.yaml                  # Standard pre-commit hooks
  .github/workflows/
    harness-checks.yml                     # Standard harness CI checks
```

When a team creates a new service, they start with a template that already has a working harness. They customize from there.

## Step 2: Custom Linter Development Program

At enterprise scale, generic linters are not enough. You need custom rules that encode your organization's specific architectural decisions.

### Why custom linters matter

Generic linters catch generic problems (unused imports, formatting). Custom linters catch organizational problems:

- "All services must use the internal authentication library, not raw JWT"
- "Database migrations must be backward-compatible for zero-downtime deploys"
- "gRPC services must include field masks for update operations"
- "All public API endpoints must have rate limiting configured"

These are the rules that AI agents violate most often because they are specific to your organization and not represented in training data.

### Building a custom linter program

**Phase 1: Collect rules (Week 1)**

- Gather architectural rules from architecture documents, post-mortems, and team leads
- Prioritize by frequency of violation and severity of impact
- Start with 10-20 rules across the org

**Phase 2: Implement rules (Week 2-4)**

Choose your implementation approach:

| Approach | Best for | Tools |
|----------|----------|-------|
| AST-based rules | Language-specific structural checks | ESLint custom rules, Semgrep, ast-grep |
| Pattern matching | Cross-language text patterns | Semgrep, grep-based CI scripts |
| Structural tests | Architecture boundary enforcement | ArchUnit, dependency-cruiser |
| Schema validation | Configuration correctness | JSON Schema, OPA/Rego |

**Phase 3: Roll out gradually (Week 3-6)**

1. Run new rules in audit mode (log violations, don't block)
2. Share violation reports with teams, gather feedback
3. Fix common violations automatically where possible
4. Promote to blocking mode after teams have had time to address existing violations

**Phase 4: Maintain and extend (Ongoing)**

- Assign rule ownership to the team that proposed each rule
- Track false-positive rates -- rules with high false positives erode trust
- Retire rules that no longer apply
- Add new rules from post-mortems and architectural decisions

### Example: Semgrep rules for organizational standards

```yaml
rules:
  - id: use-internal-auth
    patterns:
      - pattern: |
          import jwt from 'jsonwebtoken'
    message: "Use Project.Auth.JwtIssuer instead of System.IdentityModel directly. See go/auth-docs."
    languages: [typescript, javascript]
    severity: ERROR

  - id: require-rate-limiting
    patterns:
      - pattern: |
          app.get($PATH, $HANDLER)
      - pattern-not-inside: |
          app.get($PATH, rateLimiter(...), $HANDLER)
    message: "All public minimal API endpoints must call .RequireRateLimiting(\"global\") on the route group. See go/api-standards."
    languages: [typescript, javascript]
    severity: ERROR
```

## Step 3: Observability and Monitoring Agent Performance

You can't improve what you don't measure. At enterprise scale, you need visibility into how agents perform across the organization.

### Metrics to track

**Productivity metrics:**
- Agent delegation ratio: what percentage of tasks are delegated to agents?
- First-pass CI success rate: how often does agent-generated code pass CI on the first try?
- Time-to-merge for agent-generated PRs vs. human-written PRs
- Lines of code generated per engineering hour

**Quality metrics:**
- Post-merge defect rate for agent-generated code vs. human-written code
- Number of harness rule violations caught in CI per week (should decrease over time)
- Security vulnerability rate in agent-generated code
- Test coverage of agent-generated code

**Harness health metrics:**
- Number of active rules per repository
- Rule freshness: when was each rule last updated?
- False positive rate per rule
- Coverage: what percentage of repos have a harness?

### Building a dashboard

Create a dashboard (Grafana, Datadog, or a simple internal tool) that shows:

1. **Org-wide agent adoption** -- which teams are using agents, how much
2. **CI pass rates by team** -- identify teams whose harness needs improvement
3. **Top rule violations** -- what mistakes are still happening most often
4. **Harness coverage** -- which repos are unprotected
5. **Trend lines** -- are things getting better or worse over time

### Data collection approaches

- **CI metadata:** Annotate CI runs with whether the PR was agent-generated (look for co-author lines, PR labels, or branch naming conventions)
- **Git analysis:** Use git log analysis to track agent contribution patterns
- **Tool telemetry:** Most AI coding tools provide usage analytics
- **Survey data:** Quarterly developer surveys on agent effectiveness

## Step 4: Security and Compliance Considerations

### Security risks specific to AI agents

| Risk | Mitigation |
|------|-----------|
| Agent introduces dependency with known vulnerability | Automated dependency scanning in CI (Dependabot, Snyk) |
| Agent exposes secrets in code | Secret scanning in pre-commit and CI (detect-secrets, GitHub secret scanning) |
| Agent generates code with injection vulnerabilities | SAST tools in CI (Semgrep, CodeQL) |
| Agent accesses sensitive systems | Restrict agent tool access via sandboxing and permissions |
| Agent writes code that violates data privacy regulations | Custom lint rules for PII handling patterns |

### Compliance frameworks

If your organization is subject to SOC 2, HIPAA, PCI-DSS, or similar:

1. **Document your agent usage policy.** Which tasks can agents perform? Which require human-only work?
2. **Maintain audit trails.** Agent-generated commits should be identifiable (co-author tags, labels).
3. **Enforce separation of duties.** The person who prompted the agent should not be the sole reviewer.
4. **Regular access reviews.** Review what tools and data agents have access to quarterly.

### Agent sandboxing

For enterprise deployments, agents should operate within defined boundaries:

- **Network access:** Restrict which external services agents can call
- **File system access:** Limit agents to the repository directory
- **Secret access:** Never pass production secrets to agent contexts
- **Execution scope:** Agents should not be able to deploy to production directly

## Step 5: Multi-Team Coordination

### The harness working group

Establish a cross-team working group (3-5 people from different teams) that meets biweekly to:

- Review and approve organization-wide harness rules
- Share effective patterns across teams
- Coordinate major harness changes
- Maintain the golden path templates
- Triage new rule proposals

### Propagating changes

When the platform team updates organization-wide rules, those changes need to reach all repos:

**Option 1: Template sync (recommended)**

Use a tool like Cookiecutter or an internal template-sync bot that creates PRs against repos when the template changes.

**Option 2: Shared package**

Publish harness configuration as an internal package:

```json
{
  "devDependencies": {
    "@company/harness-config": "^2.0.0"
  }
}
```

Teams install the package and extend it locally.

**Option 3: Central CI step**

Add a centrally-managed CI step that all repos include:

```yaml
- name: Org harness checks
  uses: company/harness-checks@v2
```

The platform team updates the action; repos get changes automatically.

### Handling team autonomy

Teams should be able to:
- Add rules stricter than the org default
- Add domain-specific rules for their area
- Request exceptions to org rules (with documented justification)

Teams should not be able to:
- Disable org-level rules without exception approval
- Weaken security-related constraints
- Opt out of harness checks entirely

## Step 6: Training and Enablement Programs

### Tiered training program

**Tier 1: Awareness (all engineers, 1 hour)**
- What is harness engineering and why it matters
- How to read and follow the AGENTS.md in your repo
- How to report agent mistakes to improve the harness

**Tier 2: Practitioner (active agent users, half day)**
- Writing effective AGENTS.md entries
- The observe-mistake-engineer-fix cycle
- Understanding your CI harness checks
- Agent delegation decision framework

**Tier 3: Harness engineer (champions, 2 days)**
- Writing custom lint rules
- Designing verification loops
- Building entropy management systems
- Contributing to the golden path

### Ongoing enablement

- **Monthly office hours:** Harness working group answers questions
- **Slack channel or forum:** Async discussion of harness topics
- **Internal blog posts:** Share wins, new patterns, case studies
- **Quarterly retrospective:** What's working, what isn't, what's next

## Step 7: Measuring ROI and Success Metrics

### The business case

Frame harness engineering ROI in terms leadership understands:

| Metric | Before harness | After harness (target) |
|--------|---------------|----------------------|
| Developer throughput | Baseline | 30-50% increase in features shipped |
| Time to onboard new engineers | X weeks | 40-60% reduction |
| Post-deploy defect rate | Baseline | 20-40% reduction |
| CI pass rate for PRs | ~60-70% | 85-95% |
| Time spent on code review | X hours/week | 30-50% reduction (agents handle style/convention) |

### How to measure

**Before you start:** Establish baselines for 2-4 weeks. Measure what you can without changing anything.

**During rollout:** Track the same metrics weekly. Expect a dip during adoption (learning curve) followed by improvement.

**After stabilization:** Compare 3-month rolling averages against baseline.

### Reporting

Create a quarterly report for leadership that covers:
1. Adoption metrics (teams, repos, engineers using harness)
2. Quality impact (defect rates, CI pass rates)
3. Productivity impact (throughput, cycle time)
4. Investment (time spent building and maintaining harness)
5. Next quarter priorities

## Step 8: Governance and Escalation Policies

### Rule governance

All organization-wide rules follow this lifecycle:

1. **Proposal:** Anyone can propose a rule via PR to the harness template repo
2. **Review:** Harness working group reviews within 1 week
3. **Trial:** Rule runs in audit mode for 2 weeks across pilot repos
4. **Approval:** Working group approves based on trial data
5. **Rollout:** Rule goes to blocking mode across all repos
6. **Retirement:** Rules can be retired if no longer relevant

### Exception process

When a team needs to deviate from an org-wide rule:

1. **Request:** File an issue explaining why the exception is needed
2. **Scope:** Define the scope (which repos, how long, what alternative safeguard)
3. **Approval:** Harness working group approves or suggests alternatives
4. **Documentation:** Exception is documented in the repo's AGENTS.md
5. **Review:** Exceptions are reviewed quarterly and renewed or revoked

### Incident response

When agent-generated code causes a production incident:

1. **Immediate:** Follow normal incident response procedures
2. **Post-mortem:** Include harness analysis -- what rule would have prevented this?
3. **Rule creation:** Write and deploy the rule within one sprint
4. **Retrospective:** Share the learning across the organization

### Escalation matrix

| Issue | First responder | Escalation |
|-------|----------------|------------|
| Agent generates insecure code | Security team | CISO, harness working group |
| Agent violates compliance requirement | Compliance team | Legal, harness working group |
| Agent produces consistently poor output in a repo | Team lead | Platform team |
| New harness rule is causing widespread CI failures | Platform team | VP Engineering |
| Agent accesses unauthorized data or systems | Security team | CISO, incident response |

## Common Pitfalls at Enterprise Scale

| Pitfall | How to avoid it |
|---|---|
| Building enterprise infrastructure before teams are ready | Require Level 2 maturity before Level 3 investment |
| Top-down mandates without team input | Use the working group model. Rules come from practitioners. |
| Measuring the wrong things | Focus on outcomes (quality, speed) not vanity metrics (lines generated) |
| One-size-fits-all rules | Allow team and domain-level customization within org guardrails |
| Treating harness as a one-time project | It's ongoing infrastructure. Budget accordingly. |
| Ignoring developer experience | If the harness slows developers down, they'll route around it |

## Success Criteria

Your Level 3 harness is working when:

- [ ] Every new repo starts from a golden path template with a working harness
- [ ] Organization-wide lint rules catch org-specific violations across all repos
- [ ] You have a dashboard showing agent performance metrics across the org
- [ ] Security and compliance requirements are enforced via automated harness checks
- [ ] A cross-team working group actively maintains and evolves the harness
- [ ] New engineers at any level can be productive with agents within their first week
- [ ] Post-mortems routinely generate new harness rules
- [ ] Quarterly reporting shows measurable improvement in quality and throughput