# Skill Judge

A comprehensive evaluation framework for assessing Agent Skill quality against official specifications and best practices. This skill provides multi-dimensional scoring and actionable improvement suggestions for SKILL.md files and skill packages.

## Purpose

Skill Judge exists to solve a critical problem: **most Skills waste tokens on knowledge Claude already has**.

The skill helps you evaluate whether a Skill actually adds value by measuring its "knowledge delta" - the gap between what the Skill provides and what Claude already knows. A good Skill should be a compressed expert brain, not a tutorial.

### The Core Formula

> **Good Skill = Expert-only Knowledge - What Claude Already Knows**

This skill helps you identify:
- Token-wasting redundant content (things Claude already knows)
- Genuine expert knowledge that adds value
- Structural issues that prevent Skills from being activated or used effectively

## When to Use

Use Skill Judge when you need to:

- **Review a Skill before publishing**: Evaluate quality and identify improvements
- **Audit existing Skills**: Systematic assessment against best practices
- **Improve a SKILL.md file**: Get specific, actionable suggestions
- **Learn Skill design patterns**: Understand what makes a great Skill
- **Compare Skills**: Assess relative quality using consistent criteria

**Trigger phrases**:
- "Evaluate this skill"
- "Review my SKILL.md"
- "Audit this skill"
- "Score this skill"
- "How can I improve this skill?"
- "Is this skill well-designed?"

## How It Works

### Evaluation Protocol

1. **First Pass - Knowledge Delta Scan**: Read the SKILL.md and categorize each section as:
   - **[E] Expert**: Claude genuinely doesn't know this (value-add)
   - **[A] Activation**: Claude knows but brief reminder is useful (acceptable)
   - **[R] Redundant**: Claude definitely knows this (should delete)

2. **Structure Analysis**: Check frontmatter validity, line count, reference files, design pattern, and loading triggers

3. **Score Each Dimension**: Evaluate against 8 dimensions with specific evidence and justifications

4. **Calculate Total and Grade**: Sum scores (max 120 points) and assign grade

5. **Generate Report**: Produce structured report with scores, critical issues, and improvements

### The 8 Evaluation Dimensions (120 points total)

| Dimension | Max Points | What It Measures |
|-----------|------------|------------------|
| **D1: Knowledge Delta** | 20 | Does the Skill add genuine expert knowledge? (THE CORE DIMENSION) |
| **D2: Mindset + Procedures** | 15 | Does it transfer expert thinking patterns and domain-specific workflows? |
| **D3: Anti-Pattern Quality** | 15 | Does it have effective NEVER lists with specific reasons? |
| **D4: Specification Compliance** | 15 | Is the frontmatter valid? Is the description comprehensive? |
| **D5: Progressive Disclosure** | 15 | Is content properly layered for on-demand loading? |
| **D6: Freedom Calibration** | 15 | Is specificity appropriate for task fragility? |
| **D7: Pattern Recognition** | 10 | Does it follow an established official pattern? |
| **D8: Practical Usability** | 15 | Can an Agent actually use this Skill effectively? |

### Grading Scale

| Grade | Percentage | Meaning |
|-------|------------|---------|
| A | 90%+ (108+) | Excellent - production-ready expert Skill |
| B | 80-89% (96-107) | Good - minor improvements needed |
| C | 70-79% (84-95) | Adequate - clear improvement path |
| D | 60-69% (72-83) | Below Average - significant issues |
| F | <60% (<72) | Poor - needs fundamental redesign |

## Key Features

### Knowledge Classification System

The skill teaches you to recognize three types of content:

| Type | Definition | Treatment |
|------|------------|-----------|
| **Expert** | Claude genuinely doesn't know this | Must keep - this is the Skill's value |
| **Activation** | Claude knows but may not think of | Keep if brief - serves as reminder |
| **Redundant** | Claude definitely knows this | Should delete - wastes tokens |

### Five Official Design Patterns

Skill Judge identifies and evaluates against five established patterns:

| Pattern | Lines | Best For | Example |
|---------|-------|----------|---------|
| **Mindset** | ~50 | Creative tasks requiring taste | frontend-design |
| **Navigation** | ~30 | Multiple distinct scenarios | internal-comms |
| **Philosophy** | ~150 | Art/creation requiring originality | canvas-design |
| **Process** | ~200 | Complex multi-step projects | mcp-builder |
| **Tool** | ~300 | Precise operations on specific formats | docx, pdf, xlsx |

### Common Failure Pattern Detection

The skill identifies 9 common failure patterns:

1. **The Tutorial**: Explains basics Claude already knows
2. **The Dump**: Everything in one 800+ line file
3. **The Orphan References**: Reference files that never get loaded
4. **The Checkbox Procedure**: Mechanical steps without thinking frameworks
5. **The Vague Warning**: "Be careful" without specific guidance
6. **The Invisible Skill**: Great content but poor description
7. **The Wrong Location**: Trigger info in body instead of description
8. **The Over-Engineered**: Unnecessary auxiliary files
9. **The Freedom Mismatch**: Wrong freedom level for task type

## Usage Examples

### Basic Evaluation

```
Evaluate the skill at skills/my-new-skill/SKILL.md
```

### Comparative Analysis

```
Compare the quality of skills/skill-a and skills/skill-b
```

### Targeted Improvement

```
How can I improve the knowledge delta in my skill?
```

### Pattern Identification

```
What pattern does this skill follow, and is it the right choice?
```

## Output

Skill Judge produces a structured evaluation report:

```markdown
# Skill Evaluation Report: [Skill Name]

## Summary
- **Total Score**: X/120 (X%)
- **Grade**: [A/B/C/D/F]
- **Pattern**: [Mindset/Navigation/Philosophy/Process/Tool]
- **Knowledge Ratio**: E:A:R = X:Y:Z
- **Verdict**: [One sentence assessment]

## Dimension Scores
[Table with scores for all 8 dimensions]

## Critical Issues
[Must-fix problems]

## Top 3 Improvements
[Prioritized improvement suggestions]

## Detailed Analysis
[In-depth analysis for dimensions scoring below 80%]
```

## Best Practices

### When Evaluating Skills

**Do:**
- Always check the description field first (it's the most critical)
- Ask "Does Claude already know this?" for every section
- Look for specific anti-patterns with non-obvious reasons
- Verify decision trees actually lead to correct choices
- Check that loading triggers are embedded in workflows

**Never:**
- Give high scores just because content looks professional
- Ignore token waste from redundant explanations
- Let length impress you (43 lines can outperform 500)
- Forgive explaining basics as "helpful context"
- Put "when to use" information only in the body

### The Meta-Question

When evaluating any Skill, always ask:

> "Would an expert in this domain, looking at this Skill, say: 'Yes, this captures knowledge that took me years to learn'?"

If yes, the Skill has genuine value. If no, it's compressing what Claude already knows.

## Quick Reference Checklist

```
SKILL EVALUATION QUICK CHECK

KNOWLEDGE DELTA (most important):
  [ ] No "What is X" explanations for basic concepts
  [ ] No step-by-step tutorials for standard operations
  [ ] Has decision trees for non-obvious choices
  [ ] Has trade-offs only experts would know
  [ ] Has edge cases from real-world experience

MINDSET + PROCEDURES:
  [ ] Transfers thinking patterns (how to think about problems)
  [ ] Has "Before doing X, ask yourself..." frameworks
  [ ] Includes domain-specific procedures Claude wouldn't know

ANTI-PATTERNS:
  [ ] Has explicit NEVER list
  [ ] Anti-patterns are specific, not vague
  [ ] Includes WHY (non-obvious reasons)

SPECIFICATION:
  [ ] Valid YAML frontmatter
  [ ] Description answers: WHAT, WHEN, KEYWORDS
  [ ] Description specific enough for Agent activation

STRUCTURE:
  [ ] SKILL.md < 500 lines (ideal < 300)
  [ ] Loading triggers embedded in workflow
  [ ] Has "Do NOT Load" for preventing over-loading

FREEDOM:
  [ ] Creative tasks -> High freedom (principles)
  [ ] Fragile operations -> Low freedom (exact scripts)

USABILITY:
  [ ] Decision trees for multi-path scenarios
  [ ] Working code examples
  [ ] Error handling and fallbacks
```

## Prerequisites

None. Skill Judge is self-contained and requires no external tools or dependencies.

## Related Concepts

- **Tool vs Skill**: Tools define capability boundaries (what Claude CAN do). Skills inject knowledge (what Claude KNOWS how to do).
- **Progressive Disclosure**: Three-layer loading system (metadata -> SKILL.md body -> resources)
- **Freedom Calibration**: Matching constraint level to task fragility