---
name: tools-page-seo-optimizer
description: "Framework-agnostic SEO workflow for any site with multiple tool, product, or feature pages. Covers duplicate content, unique meta tags, heading hierarchy, internal linking, URL slugs, E-E-A-T, content registry pattern for scaling 50–500 pages, and blog content strategy for position 50–68 keywords."
category: seo
risk: safe
source: community
source_type: community
author: whoisabhishekadhikari
date_added: "2026-06-19"
tags: [seo, tools-pages, product-pages, duplicate-content, content-registry, meta-tags, internal-linking, url-slugs, e-e-a-t, framework-agnostic]
tools: [claude-code, cursor, codex-cli, gemini-cli, opencode]
version: 1.0.0
---

# Tools Page SEO Optimizer

You are an expert in technical SEO and content strategy for sites with large collections of tool, product, or feature pages. Your workflow is framework-agnostic — applies to Django, Rails, Laravel, Express, Next.js, Nuxt, Astro, WordPress, and static HTML.

Derived from a real audit that found 93 of 105 tool pages sharing identical template prose and ranking at average position 68. This skill is the playbook that fixes it.

---

## Quick-Start Decision Tree

```
Full audit from scratch?              → Run all phases in order
All tool pages rank the same?         → Phase 2 (Content Registry) first
Meta titles/descriptions all generic? → Phase 1 (Meta Tags)
Tool pages buried / hard to navigate? → Phase 5 (Internal Linking)
Bad URL slugs?                        → Phase 6 (URL Slug Hygiene)
Site looks authorless to Google?      → Phase 7 (E-E-A-T)
Stuck at position 50–68 on keywords?  → Phase 9 (Blog Content Strategy)
Fixes deployed but unsure they're live? → Phase 10 (Live Verification)
```

---

## Phase 0 — Codebase Reconnaissance

**Before writing any code**, locate these in the codebase. Names vary by framework — adapt.

| What to find | Common locations |
|---|---|
| URL routing | `routes.rb`, `urls.py`, `routes/`, `pages/`, `app/` |
| Head / meta template | `_head.html`, `layout.js`, `base.html`, `app.blade.php` |
| Tool/page registry | config file, database seed, JSON, `lib/guides.js`, `data/tools.js` |

**Answer these before writing a single line:**

1. How are tool pages generated — static files, database loop, config registry, CMS?
2. Where is the shared template that renders `<title>`, `<meta name="description">`, `<h1>`?
3. Does each tool have its own content fields, or does every tool fall back to the same template prose?
4. Is there a central list of all tool slugs you can iterate over programmatically?

---

## Phase 1 — Meta Titles & Descriptions

### The core problem

Every tool page sharing the same `<title>` template with only the tool name swapped in
is the single most common reason tool sites rank poorly. Google treats near-identical titles
as duplicate pages and demotes all of them.

### Title tag formula

```
{Tool Name} | {Specific Outcome} — {Brand}
```

| ✅ Good | ❌ Bad |
|---|---|
| `Meta Tag Generator \| Create Perfect SEO Titles Free — MySite` | `Meta Tag Generator - MySite Tools` |
| `Broken Link Finder \| Scan Any Page for Dead URLs — MySite` | `Broken Link Finder - Free Online Tool \| MySite` |

**Rules:**
- ≤ 60 characters total
- Primary keyword in the first 40 characters
- Every tool has a **unique** title — no two tools share the same one
- Include "Free" where accurate — measurably improves CTR

### Meta description formula

```
{What it does — one action sentence}. {Key differentiator}. {CTA}.
```

Example: `Scan any webpage for broken links in seconds. Checks internal and external URLs,
exports results as CSV. Free, no account needed.`

**Rules:**
- 120–160 characters
- Action verbs: Generate, Scan, Check, Analyze, Convert, Build, Find
- Every tool gets a **custom** description — zero template filler

### Implementation (any framework)

```html
<!-- Generic template pattern -->
<title>{{ tool.meta_title | default(tool.name + " | " + site_name) }}</title>
<meta name="description" content="{{ tool.meta_description | default(tool.tagline) }}">
```

### Validation script — run before every deploy

```python
# validate_meta.py
import json, sys

tools = json.load(open('data/tools.json'))
errors = []

for t in tools:
    slug  = t.get('slug', '?')
    title = t.get('meta_title', '')
    desc  = t.get('meta_description', '')
    if not title:          errors.append(f"MISSING TITLE: {slug}")
    elif len(title) > 60:  errors.append(f"TITLE TOO LONG ({len(title)}): {slug}")
    if not desc:           errors.append(f"MISSING DESC: {slug}")
    elif len(desc) < 120:  errors.append(f"DESC TOO SHORT ({len(desc)}): {slug}")
    elif len(desc) > 160:  errors.append(f"DESC TOO LONG ({len(desc)}): {slug}")

if errors:
    print('\n'.join(errors)); sys.exit(1)
print(f"✅ All {len(tools)} tools passed meta validation")
```

---

## Phase 2 — Content Registry (The Highest-Leverage Fix)

**Root cause of poor rankings on tool sites:** 80–95% of tool pages share identical
template prose. Google sees them as thin, near-duplicate pages and ranks none well.
Fix this before anything else.

### Diagnosis

```bash
# Find shared prose in your templates — if these strings appear in a shared template
# file, you have the problem
D1=$(grep -rn "powerful tool that helps" templates/ src/ 2>/dev/null | head -5)
[ -n "$D1" ] && echo "  ✗ Shared template prose found" || echo "  ✓ No shared prose"
D2=$(grep -rn "easy to use" templates/ src/ 2>/dev/null | head -5)
[ -n "$D2" ] && echo "  ✗ Template filler found"
```

### Registry entry structure (framework-agnostic)

```yaml
# data/tools/meta-tag-generator.yaml  (or JSON, DB columns, JS object — adapt to your stack)
slug: meta-tag-generator
name: Meta Tag Generator
meta_title: "Meta Tag Generator | Create Perfect SEO Titles & Descriptions Free"
meta_description: "Generate optimized title tags and meta descriptions with live character
  counters. Enforces Google's 60/160 char limits. Instant, free, no account needed."

introduction: >
  The meta tag generator creates the two most critical on-page SEO elements —
  your title tag and meta description — with live character counters that enforce
  Google's recommended limits before you publish. [80+ unique words minimum]

best_practices:
  - "Include your primary keyword within the first 40 characters of the title"
  - "Write a unique description per page — duplicate descriptions waste crawl budget"
  - "Use action verbs in descriptions: Generate, Find, Check, Analyze"

how_to_steps:
  - name: "Enter your page details"
    text: "Type your target keyword, page topic, and a brief summary of the content"
  - name: "Check the live character counters"
    text: "Keep title ≤60 chars and description ≤160 chars"
  - name: "Copy and paste the output"
    text: "Paste the generated tags into your HTML <head> section"

faqs:
  - q: "Does Google always use my meta description?"
    a: "No — Google rewrites descriptions ~63% of the time. Write them anyway for
       social shares and some SERPs."
  - q: "What happens if my title is over 60 characters?"
    a: "Google truncates it with an ellipsis, cutting off your message mid-sentence."

related_tools:
  - og-tag-generator
  - schema-markup-generator
  - heading-analyzer
```

### Minimum viable unique content per tool

| Field | Minimum | Priority |
|---|---|---|
| `meta_title` | Unique, ≤60 chars | 🔴 Critical |
| `meta_description` | Unique, 120–160 chars | 🔴 Critical |
| `introduction` | 80+ unique words | 🔴 Critical |
| `best_practices` | 3–5 tool-specific items | 🟡 High |
| `how_to_steps` | 3 real steps for THIS tool | 🟡 High |
| `faqs` | 2 tool-specific Q&As | 🟡 High |
| `related_tools` | 2–4 slug references | 🟢 Medium |

**Rule: complete one tool fully before starting the next.**

---

## Phase 3 — H1 and Heading Hierarchy

### H1 formula

```
{Tool Name} | {Outcome Phrase}
```

**Rules:**
- One `<h1>` per page — only the tool name/title
- Must be unique per page

### Heading hierarchy

```
h1 — Tool name (one per page)
  h2 — Major sections: "How It Works", "Best Practices", "FAQs", "Related Tools"
    h3 — Subsections: individual FAQ items, feature callouts, step headers
```

Never skip levels. No h1 → h3 without an h2.

```bash
# Audit heading hierarchy on a live page
curl -s "https://yourdomain.com/tools/meta-tag-generator" \
  | grep -oE '<h[1-6][^>]*>.*?</h[1-6]>'
```

---

## Phase 4 — Accessibility

Accessibility failures lower Core Web Vitals scores — a direct ranking signal.

Every icon-only interactive element needs `aria-label`:

```html
<button aria-label="Copy to clipboard"><svg>...</svg></button>
<button aria-label="Go to next page">›</button>
<input type="search" aria-label="Search tools" placeholder="Search...">
```

```bash
# Find icon-only buttons missing aria-label
B=$(grep -rn "<button" templates/ 2>/dev/null | grep -v "aria-label" | grep -v ">[A-Za-z]" | head -5)
[ -n "$B" ] && echo "  ⚠ Icon buttons missing aria-label:" && echo "$B" || echo "  ✓ Buttons have aria-labels"
```

---

## Phase 5 — Internal Linking

Internal links between tools are how PageRank flows through your site. A tool with no
inbound internal links is effectively invisible to Google even with great content.

### Hub-and-Spoke model

```
Homepage
  └── Category: Keyword Tools
        ├── Keyword Density Checker  ←→  Keyword Suggestion Tool
        └── SERP Preview Tool        ←→  Meta Tag Generator
  └── Category: Technical SEO
        ├── XML Sitemap Visualizer   ←→  Robots.txt Creator
        └── Robots.txt Creator       ←→  Redirect Generator
```

### Rules

- Every tool links **to** at least 2 related tools (use `related_tools` from registry)
- Every tool is linked **from** at least 2 other tools or category pages
- No orphan tools — every tool reachable within 3 clicks from homepage

```bash
# Orphan detection — tools with too few inbound references
for slug in $(cat data/slugs.txt 2>/dev/null); do
  C=$(grep -rl "$slug" templates/ 2>/dev/null | wc -l | tr -d ' ')
  [ "$C" -lt 2 ] && echo "  ORPHAN RISK: $slug ($C refs)"
done
```

### Template implementation

```html
{% if tool.related_tools %}
<section>
  <h2>Related Tools</h2>
  {% for slug in tool.related_tools %}
  {% set rel = get_tool(slug) %}
  <a href="/tools/{{ slug }}">{{ rel.name }} — {{ rel.tagline }}</a>
  {% endfor %}
</section>
{% endif %}
```

---

## Phase 6 — URL Slug Hygiene

| ✅ Good | ❌ Bad | Problem |
|---|---|---|
| `/tools/meta-tag-generator` | `/tools/tool-1` | No keywords |
| `/tools/keyword-density-checker` | `/tools/free-online-keyword-density-checker-tool-free` | Keyword stuffed |
| `/tools/broken-link-finder` | `/tools/brokenLinkFinder` | camelCase |

**Formula:** `{primary-keyword-phrase}` — lowercase, hyphens, no stop words, no "free" / "online" / "tool" padding.

```bash
# Audit — list longest slugs (likely stuffed)
curl -s "https://yourdomain.com/sitemap.xml" \
  | grep -oE '<loc>[^<]+' | sed 's/<loc>//' \
  | grep "/tools/" \
  | awk -F'/tools/' '{print length($2), $2}' | sort -n | tail -20
```

If renaming a slug, always 301 redirect old → new and update all internal links.

---

## Phase 7 — E-E-A-T Signals

Tool sites rank poorly when they look authorless and dateless.

### Author byline + date (every tool page)

```html
<p class="tool-byline">
  Built by <a href="/about">Your Name</a>
  <time datetime="{{ tool.updated_at }}"> · Updated {{ tool.updated_at | date }}</time>
</p>
```

### Trust pillars section

```html
<section class="trust-pillars">
  <div>✅ <strong>100% Free</strong> — no account, no credit card</div>
  <div>🔒 <strong>Privacy First</strong> — your data never leaves your browser</div>
  <div>⚡ <strong>Instant Results</strong> — processed in under 1 second</div>
</section>
```

### About / author page

Create `/about` with: real name, credentials, why you built the tools, contact info.
Link to it from every tool page byline. This is the single highest-impact E-E-A-T fix
for solo-built tool sites.

---

## Phase 8 — Scaling to 100+ Tools

When the content registry pattern is working for 10–20 tools, the next challenge is
scaling it to 100+ without losing quality or introducing duplicates.

### Batch completion gate

Never commit a partial batch. Before every commit touching tool content:

```bash
# Count tools with introduction content vs total tools
python3 -c "
import json
tools = json.load(open('data/tools.json'))
total   = len(tools)
done    = sum(1 for t in tools if t.get('introduction','').strip())
print(f'{done}/{total} tools have introduction content')
if done < total:
    missing = [t['slug'] for t in tools if not t.get('introduction','').strip()]
    print('Missing:', missing)
"
```

Only commit when the count is **100% complete**. A partial batch (e.g. 93/105) means
12 tools still have thin template prose — enough for Google to flag the site as inconsistent.

### Batch writing order

Prioritise tools in this order:
1. Tools already receiving impressions in Google Search Console (low-hanging fruit)
2. Tools in your most-linked categories (PageRank concentration)
3. Remaining tools alphabetically

### Build verification before commit

```bash
# Confirm build compiles cleanly after batch content additions
npm run build        # Next.js / Nuxt
python manage.py check  # Django
rails assets:precompile  # Rails
# Zero errors = safe to commit
```

---

## Phase 9 — Blog Content Strategy (Position 50–68 Keywords)

Tool pages rank well for transactional keywords ("meta tag generator", "check broken links").
But informational keywords ("how to write meta descriptions", "what is keyword density") sit
at position 50–68 — too deep to get clicks — because tool pages aren't the right content
format for them. Blog posts are.

### Diagnosis: find your 50–68 keywords

In Google Search Console → Search Results → filter by Position > 49 AND Position < 69.
These are queries where you have enough authority to rank but the wrong page type is ranking.

### Blog post targeting formula

```
Post title: {Informational keyword} — {Year} Guide
Target keyword: the exact query from GSC
Content length: 1,000–1,500 words
Internal links: link to 2–3 relevant tools from within the post body
```

Example mapping:

| GSC keyword (pos 50–68) | Blog post title | Tool to link |
|---|---|---|
| "how to write meta descriptions" | "How to Write Meta Descriptions That Get Clicks (2025)" | meta-tag-generator |
| "what is keyword density" | "Keyword Density: What It Is and How to Check It" | keyword-density-checker |
| "how to find broken links" | "How to Find and Fix Broken Links on Any Website" | broken-link-finder |
| "xml sitemap best practices" | "XML Sitemap Best Practices for 2025" | xml-sitemap-visualizer |

### Blog post structure (SEO-optimised)

```
H1: {Target keyword} — the exact GSC query, naturally phrased
Intro (100 words): answer the question directly in the first paragraph
  H2: What is {topic}?
  H2: Why it matters for SEO
  H2: How to {action} — step by step
    H3: Step 1
    H3: Step 2
    H3: Step 3
  H2: Common mistakes
  H2: {Tool name} — try it free   ← internal link to your tool
Conclusion: summarise + CTA to the tool
```

### Blog post meta requirements

- `meta_title`: include year where relevant ("2025") — improves CTR on informational queries
- `meta_description`: answer the question in one sentence + "Free tool included"
- `canonical`: must point to the exact blog URL
- `datePublished` + `dateModified` in schema — critical for freshness signals

### Internal link rule for blog posts

Every blog post must contain at least **2 contextual inline links** to relevant tools,
not just a "Related Tools" sidebar. Inline links within body copy pass significantly
more PageRank than sidebar links.

```html
<!-- Good — inline contextual link -->
<p>Use our <a href="/tools/meta-tag-generator">meta tag generator</a> to preview
how your title and description appear in Google results before publishing.</p>

<!-- Weak — sidebar only, no body link -->
<aside>Related: Meta Tag Generator</aside>
```

---

## Phase 10 — Live Deployment Verification

A fix that compiles cleanly can still fail in production. After every push, verify
the live site — not just the build.

```bash
seo:verify() {
  local D="$1"; local F=0
  for p in "/" "/tools/meta-tag-generator" "/blog" "/category" "/privacy" "/terms"; do
    local C=$(curl -so /dev/null -w "%{http_code}" "$D$p")
    echo "$C $p"; [ "$C" = "200" ] || ((F++))
  done
  local C=$(curl -so /dev/null -w "%{http_code}" "$D/tools/this-slug-does-not-exist-xyz")
  echo "Soft 404 check: $C (expect 404)"; [ "$C" = "404" ] || { echo "  ✗ Soft 404"; ((F++)); }
  curl -s "$D/tools/meta-tag-generator" | grep -qi "canonical" && echo "  ✓ Canonical present" || { echo "  ✗ Canonical missing"; ((F++)); }
  local C2=$(curl -so /dev/null -w "%{http_code}" "$D/favicon.ico")
  echo "Favicon: $C2 (expect 200)"; [ "$C2" = "200" ] || { echo "  ✗ Favicon missing"; ((F++)); }
  local J=$(curl -s "$D/tools/meta-tag-generator" | grep -c "application/ld+json" || true)
  [ "$J" -ge 1 ] && echo "  ✓ Schema: $J blocks" || { echo "  ✗ No schema found"; ((F++)); }
  return $F
}
```

### Expected results

| Check | Expected |
|---|---|
| All key pages | 200 |
| Invalid tool slug | 404 |
| `<link rel="canonical">` present | Yes |
| `/favicon.ico` | 200 |
| `application/ld+json` blocks | ≥ 1 per tool page |

If any check fails — **do not move on**. Diagnose and fix before the next phase.

---

## Phase 11 — Pre-Commit Validation

```bash
seo:validate() {
  python3 validate_meta.py || return 1
  python3 -c "
import json
from collections import Counter
tools = json.load(open('data/tools.json', 'r'))
titles = [t.get('meta_title', '') for t in tools]
dups = [t for t, c in Counter(titles).items() if c > 1 and t]
print(dups) if dups else print('All titles unique')
"
  python3 -c "
import json
tools = json.load(open('data/tools.json', 'r'))
missing = [t['slug'] for t in tools if not t.get('introduction', '').strip()]
print(f'Missing intro ({len(missing)}):', missing[:10])
"
}
```

---

## Consolidated Runners

```bash
# Quick check — meta validation + live site verification
seo:quick() { seo:verify "$PROD_URL" && seo:validate; }
# Full check — quick + duplicate title check
seo:full()  { seo:quick; }
```

---

## Master Issue Control Table

| # | Issue | Severity | Phase |
|---|---|---|---|
| 1 | Duplicate / template prose across tool pages | 🔴 Critical | 2 |
| 2 | Missing or generic meta titles | 🔴 Critical | 1 |
| 3 | Missing or generic meta descriptions | 🔴 Critical | 1 |
| 4 | H1 shared across all tools | 🔴 Critical | 3 |
| 5 | Orphan tool pages (no inbound internal links) | 🟡 High | 5 |
| 6 | Missing related tool links | 🟡 High | 5 |
| 7 | Keyword-stuffed or keywordless URL slugs | 🟡 High | 6 |
| 8 | No author byline or last-updated date | 🟡 High | 7 |
| 9 | Heading hierarchy violations | 🟢 Medium | 3 |
| 10 | Missing aria-labels on icon buttons | 🟢 Medium | 4 |

---

## Best Practices

| ✅ Do | ❌ Don't |
|-------|----------|
| Write unique meta title + description per tool | Use template prose shared across 80+ pages |
| Complete one tool's content fully before next | Batch-write partial entries across many tools |
| Link 2+ related tools from every tool page | Leave orphan tools with zero internal links |
| Use `{primary-keyword}` in URL slug | Pad slugs with "free" / "online" / "tool" |
| Add author byline + date to every tool page | Show authorless, dateless content |
| Blog about informational keywords (pos 50–68) | Rely on tool pages to rank for "how to" queries |
| 301 redirect old slugs when renaming | Delete old slugs without redirects |

---

## Key Principles

1. **Duplicate content first, always.** Identical template prose across 80+ pages is the
   root cause on almost every underperforming tool site. No other fix matters until this is done.

2. **Complete one tool fully before the next.** Never write partial entries across many tools.
   A half-written registry entry is worse than no entry — it signals thin content at scale.

3. **Internal links are PageRank distribution.** A tool with great content but zero inbound
   internal links is invisible to Google. Every tool needs at least 2 inbound links.

4. **URL slugs are permanent.** A clean slug outperforms a stuffed one from day one.
   Get them right before indexing — renaming later costs ranking momentum even with 301s.

5. **E-E-A-T is not decoration.** Authorless, dateless tool pages trigger quality rater
   guidelines as potential spam. A real name and a real date is the minimum baseline.

6. **Validate before every commit.** Catching a missing description before deploy is free.
   Fixing it after indexing costs weeks.

---

## Related Skills

- [schema-markup-generator](../schema-markup-generator/SKILL.md) — JSON-LD structured data (HowTo, FAQPage, WebApplication) for tool pages
- [social-metadata-hardening](../social-metadata-hardening/SKILL.md) — OG tags and social sharing previews for tool pages
- [indexing-issue-auditor](../indexing-issue-auditor/SKILL.md) — full crawl audit and redirect mapping after slug changes
- [pagespeed-enhancer](../pagespeed-enhancer/SKILL.md) — Lighthouse / Core Web Vitals audit for tool pages
- [wordpress-centric-high-seo-optimized-blogwriting-skill](../wordpress-centric-high-seo-optimized-blogwriting-skill/SKILL.md) — blog post writing with SEO structure
- [vibecode-production-qa-validator](../vibecode-production-qa-validator/SKILL.md) — end-to-end production QA including deployment verification

---

## When to Use

This skill is applicable to execute the workflow or actions described in the overview.
Use it whenever the user mentions poor rankings, tools not getting indexed, all tool pages ranking the same, duplicate content warnings, "how do I make each tool page unique", thin content, or Google not ranking tool pages.

## Limitations

- Use this skill only when the task clearly matches the scope described above.
- Do not treat the output as a substitute for environment-specific validation, testing, or expert review.
- Stop and ask for clarification if required inputs, permissions, safety boundaries, or success criteria are missing.
- Content registry assumes a structured data source (JSON, YAML, DB) — static HTML tool pages will need a migration step first.
- Technical SEO factors (page speed, Core Web Vitals, render blocking) are delegated to the pagespeed-enhancer skill.


---
name: top-web-vulnerabilities
description: "Provide a comprehensive, structured reference for the 100 most critical web application vulnerabilities organized by category. This skill enables systematic vulnerability identification, impact assessment, and remediation guidance across the full spectrum of web security threats."
risk: offensive
source: community
author: zebbern
date_added: "2026-02-27"
---

> **⚠ AUTHORIZED USE ONLY**
> This skill is for educational purposes or authorized security assessments only.
> You must have explicit, written permission from the system owner before using this tool.
> Misuse of this tool is illegal and strictly prohibited.

> **Mandatory confirmation gate**
> Before running any command that probes, exploits, changes, persists on, extracts data from, or attempts credential access against a target:
> 1. Ask the user to state the exact target URL, IP, account, or resource.
> 2. Ask the user to confirm written authorization and the permitted scope.
> 3. Show the exact command(s) and explain their expected effect.
> 4. Wait for explicit confirmation in the current conversation.
>
> Without that confirmation, remain read-only and provide defensive guidance only. Prefer a sandbox, disposable VM, or controlled lab.

# Top 100 Web Vulnerabilities Reference

## Purpose

Provide a comprehensive, structured reference for the 100 most critical web application vulnerabilities organized by category. This skill enables systematic vulnerability identification, impact assessment, and remediation guidance across the full spectrum of web security threats. Content organized into 15 major vulnerability categories aligned with industry standards and real-world attack patterns.

## Prerequisites

- Basic understanding of web application architecture (client-server model, HTTP protocol)
- Familiarity with common web technologies (HTML, JavaScript, SQL, XML, APIs)
- Understanding of authentication and authorization concepts
- Access to web application security testing tools (Burp Suite, OWASP ZAP)
- Knowledge of secure coding principles recommended

## Outputs and Deliverables

- Complete vulnerability catalog with definitions, root causes, impacts, and mitigations
- Category-based vulnerability groupings for systematic assessment
- Quick reference for security testing and remediation
- Foundation for vulnerability assessment checklists and security policies

---

## Core Workflow

### Phase 1: Injection Vulnerabilities Assessment

Evaluate injection attack vectors targeting data processing components:

**SQL Injection (1)**
- Definition: Malicious SQL code inserted into input fields to manipulate database queries
- Root Cause: Lack of input validation, improper use of parameterized queries
- Impact: Unauthorized data access, data manipulation, database compromise
- Mitigation: Use parameterized queries/prepared statements, input validation, least privilege database accounts

**Cross-Site Scripting - XSS (2)**
- Definition: Injection of malicious scripts into web pages viewed by other users
- Root Cause: Insufficient output encoding, lack of input sanitization
- Impact: Session hijacking, credential theft, website defacement
- Mitigation: Output encoding, Content Security Policy (CSP), input sanitization

**Command Injection (5, 11)**
- Definition: Execution of arbitrary system commands through vulnerable applications
- Root Cause: Unsanitized user input passed to system shells
- Impact: Full system compromise, data exfiltration, lateral movement
- Mitigation: Avoid shell execution, whitelist valid commands, strict input validation

**XML Injection (6), LDAP Injection (7), XPath Injection (8)**
- Definition: Manipulation of XML/LDAP/XPath queries through malicious input
- Root Cause: Improper input handling in query construction
- Impact: Data exposure, authentication bypass, information disclosure
- Mitigation: Input validation, parameterized queries, escape special characters

**Server-Side Template Injection - SSTI (13)**
- Definition: Injection of malicious code into template engines
- Root Cause: User input embedded directly in template expressions
- Impact: Remote code execution, server compromise
- Mitigation: Sandbox template engines, avoid user input in templates, strict input validation

### Phase 2: Authentication and Session Security

Assess authentication mechanism weaknesses:

**Session Fixation (14)**
- Definition: Attacker sets victim's session ID before authentication
- Root Cause: Session ID not regenerated after login
- Impact: Session hijacking, unauthorized account access
- Mitigation: Regenerate session ID on authentication, use secure session management

**Brute Force Attack (15)**
- Definition: Systematic password guessing using automated tools
- Root Cause: Lack of account lockout, rate limiting, or CAPTCHA
- Impact: Unauthorized access, credential compromise
- Mitigation: Account lockout policies, rate limiting, MFA, CAPTCHA

**Session Hijacking (16)**
- Definition: Attacker steals or predicts valid session tokens
- Root Cause: Weak session token generation, insecure transmission
- Impact: Account takeover, unauthorized access
- Mitigation: Secure random token generation, HTTPS, HttpOnly/Secure cookie flags

**Credential Stuffing and Reuse (22)**
- Definition: Using leaked credentials to access accounts across services
- Root Cause: Users reusing passwords, no breach detection
- Impact: Mass account compromise, data breaches
- Mitigation: MFA, breach password checks, unique credential requirements

**Insecure "Remember Me" Functionality (85)**
- Definition: Weak persistent authentication token implementation
- Root Cause: Predictable tokens, inadequate expiration controls
- Impact: Unauthorized persistent access, session compromise
- Mitigation: Strong token generation, proper expiration, secure storage

**CAPTCHA Bypass (86)**
- Definition: Circumventing bot detection mechanisms
- Root Cause: Weak CAPTCHA algorithms, improper validation
- Impact: Automated attacks, credential stuffing, spam
- Mitigation: reCAPTCHA v3, layered bot detection, rate limiting

### Phase 3: Sensitive Data Exposure

Identify data protection failures:

**IDOR - Insecure Direct Object References (23, 42)**
- Definition: Direct access to internal objects via user-supplied references
- Root Cause: Missing authorization checks on object access
- Impact: Unauthorized data access, privacy breaches
- Mitigation: Access control validation, indirect reference maps, authorization checks

**Data Leakage (24)**
- Definition: Inadvertent disclosure of sensitive information
- Root Cause: Inadequate data protection, weak access controls
- Impact: Privacy breaches, regulatory penalties, reputation damage
- Mitigation: DLP solutions, encryption, access controls, security training

**Unencrypted Data Storage (25)**
- Definition: Storing sensitive data without encryption
- Root Cause: Failure to implement encryption at rest
- Impact: Data breaches if storage compromised
- Mitigation: Full-disk encryption, database encryption, secure key management

**Information Disclosure (33)**
- Definition: Exposure of system details through error messages or responses
- Root Cause: Verbose error handling, debug information in production
- Impact: Reconnaissance for further attacks, credential exposure
- Mitigation: Generic error messages, disable debug mode, secure logging

### Phase 4: Security Misconfiguration

Assess configuration weaknesses:

**Missing Security Headers (26)**
- Definition: Absence of protective HTTP headers (CSP, X-Frame-Options, HSTS)
- Root Cause: Inadequate server configuration
- Impact: XSS attacks, clickjacking, protocol downgrade
- Mitigation: Implement CSP, X-Content-Type-Options, X-Frame-Options, HSTS

**Default Passwords (28)**
- Definition: Unchanged default credentials on systems/applications
- Root Cause: Failure to change vendor defaults
- Impact: Unauthorized access, system compromise
- Mitigation: Mandatory password changes, strong password policies

**Directory Listing (29)**
- Definition: Web server exposes directory contents
- Root Cause: Improper server configuration
- Impact: Information disclosure, sensitive file exposure
- Mitigation: Disable directory indexing, use default index files

**Unprotected API Endpoints (30)**
- Definition: APIs lacking authentication or authorization
- Root Cause: Missing security controls on API routes
- Impact: Unauthorized data access, API abuse
- Mitigation: OAuth/API keys, access controls, rate limiting

**Open Ports and Services (31)**
- Definition: Unnecessary network services exposed
- Root Cause: Failure to minimize attack surface
- Impact: Exploitation of vulnerable services
- Mitigation: Port scanning audits, firewall rules, service minimization

**Misconfigured CORS (35)**
- Definition: Overly permissive Cross-Origin Resource Sharing policies
- Root Cause: Wildcard origins, improper CORS configuration
- Impact: Cross-site request attacks, data theft
- Mitigation: Whitelist trusted origins, validate CORS headers

**Unpatched Software (34)**
- Definition: Systems running outdated vulnerable software
- Root Cause: Neglected patch management
- Impact: Exploitation of known vulnerabilities
- Mitigation: Patch management program, vulnerability scanning, automated updates

### Phase 5: XML-Related Vulnerabilities

Evaluate XML processing security:

**XXE - XML External Entity Injection (37)**
- Definition: Exploitation of XML parsers to access files or internal systems
- Root Cause: External entity processing enabled
- Impact: File disclosure, SSRF, denial of service
- Mitigation: Disable external entities, use safe XML parsers

**XEE - XML Entity Expansion (38)**
- Definition: Excessive entity expansion causing resource exhaustion
- Root Cause: Unlimited entity expansion allowed
- Impact: Denial of service, parser crashes
- Mitigation: Limit entity expansion, configure parser restrictions

**XML Bomb (Billion Laughs) (39)**
- Definition: Crafted XML with nested entities consuming resources
- Root Cause: Recursive entity definitions
- Impact: Memory exhaustion, denial of service
- Mitigation: Entity expansion limits, input size restrictions

**XML Denial of Service (65)**
- Definition: Specially crafted XML causing excessive processing
- Root Cause: Complex document structures without limits
- Impact: CPU/memory exhaustion, service unavailability
- Mitigation: Schema validation, size limits, processing timeouts

### Phase 6: Broken Access Control

Assess authorization enforcement:

**Inadequate Authorization (40)**
- Definition: Failure to properly enforce access controls
- Root Cause: Weak authorization policies, missing checks
- Impact: Unauthorized access to sensitive resources
- Mitigation: RBAC, centralized IAM, regular access reviews

**Privilege Escalation (41)**
- Definition: Gaining elevated access beyond intended permissions
- Root Cause: Misconfigured permissions, system vulnerabilities
- Impact: Full system compromise, data manipulation
- Mitigation: Least privilege, regular patching, privilege monitoring

**Forceful Browsing (43)**
- Definition: Direct URL manipulation to access restricted resources
- Root Cause: Weak access controls, predictable URLs
- Impact: Unauthorized file/directory access
- Mitigation: Server-side access controls, unpredictable resource paths

**Missing Function-Level Access Control (44)**
- Definition: Unprotected administrative or privileged functions
- Root Cause: Authorization only at UI level
- Impact: Unauthorized function execution
- Mitigation: Server-side authorization for all functions, RBAC

### Phase 7: Insecure Deserialization

Evaluate object serialization security:

**Remote Code Execution via Deserialization (45)**
- Definition: Arbitrary code execution through malicious serialized objects
- Root Cause: Untrusted data deserialized without validation
- Impact: Complete system compromise, code execution
- Mitigation: Avoid deserializing untrusted data, integrity checks, type validation

**Data Tampering (46)**
- Definition: Unauthorized modification of serialized data
- Root Cause: Missing integrity verification
- Impact: Data corruption, privilege manipulation
- Mitigation: Digital signatures, HMAC validation, encryption

**Object Injection (47)**
- Definition: Malicious object instantiation during deserialization
- Root Cause: Unsafe deserialization practices
- Impact: Code execution, unauthorized access
- Mitigation: Type restrictions, class whitelisting, secure libraries

### Phase 8: API Security Assessment

Evaluate API-specific vulnerabilities:

**Insecure API Endpoints (48)**
- Definition: APIs without proper security controls
- Root Cause: Poor API design, missing authentication
- Impact: Data breaches, unauthorized access
- Mitigation: OAuth/JWT, HTTPS, input validation, rate limiting

**API Key Exposure (49)**
- Definition: Leaked or exposed API credentials
- Root Cause: Hardcoded keys, insecure storage
- Impact: Unauthorized API access, abuse
- Mitigation: Secure key storage, rotation, environment variables

**Lack of Rate Limiting (50)**
- Definition: No controls on API request frequency
- Root Cause: Missing throttling mechanisms
- Impact: DoS, API abuse, resource exhaustion
- Mitigation: Rate limits per user/IP, throttling, DDoS protection

**Inadequate Input Validation (51)**
- Definition: APIs accepting unvalidated user input
- Root Cause: Missing server-side validation
- Impact: Injection attacks, data corruption
- Mitigation: Strict validation, parameterized queries, WAF

**API Abuse (75)**
- Definition: Exploiting API functionality for malicious purposes
- Root Cause: Excessive trust in client input
- Impact: Data theft, account takeover, service abuse
- Mitigation: Strong authentication, behavior analysis, anomaly detection

### Phase 9: Communication Security

Assess transport layer protections:

**Man-in-the-Middle Attack (52)**
- Definition: Interception of communication between parties
- Root Cause: Unencrypted channels, compromised networks
- Impact: Data theft, session hijacking, impersonation
- Mitigation: TLS/SSL, certificate pinning, mutual authentication

**Insufficient Transport Layer Security (53)**
- Definition: Weak or outdated encryption for data in transit
- Root Cause: Outdated protocols (SSLv2/3), weak ciphers
- Impact: Traffic interception, credential theft
- Mitigation: TLS 1.2+, strong cipher suites, HSTS

**Insecure SSL/TLS Configuration (54)**
- Definition: Improperly configured encryption settings
- Root Cause: Weak ciphers, missing forward secrecy
- Impact: Traffic decryption, MITM attacks
- Mitigation: Modern cipher suites, PFS, certificate validation

**Insecure Communication Protocols (55)**
- Definition: Use of unencrypted protocols (HTTP, Telnet, FTP)
- Root Cause: Legacy systems, security unawareness
- Impact: Traffic sniffing, credential exposure
- Mitigation: HTTPS, SSH, SFTP, VPN tunnels

### Phase 10: Client-Side Vulnerabilities

Evaluate browser-side security:

**DOM-based XSS (56)**
- Definition: XSS through client-side JavaScript manipulation
- Root Cause: Unsafe DOM manipulation with user input
- Impact: Session theft, credential harvesting
- Mitigation: Safe DOM APIs, CSP, input sanitization

**Insecure Cross-Origin Communication (57)**
- Definition: Improper handling of cross-origin requests
- Root Cause: Relaxed CORS/SOP policies
- Impact: Data leakage, CSRF attacks
- Mitigation: Strict CORS, CSRF tokens, origin validation

**Browser Cache Poisoning (58)**
- Definition: Manipulation of cached content
- Root Cause: Weak cache validation
- Impact: Malicious content delivery
- Mitigation: Cache-Control headers, HTTPS, integrity checks

**Clickjacking (59, 71)**
- Definition: UI redress attack tricking users into clicking hidden elements
- Root Cause: Missing frame protection
- Impact: Unintended actions, credential theft
- Mitigation: X-Frame-Options, CSP frame-ancestors, frame-busting

**HTML5 Security Issues (60)**
- Definition: Vulnerabilities in HTML5 APIs (WebSockets, Storage, Geolocation)
- Root Cause: Improper API usage, insufficient validation
- Impact: Data leakage, XSS, privacy violations
- Mitigation: Secure API usage, input validation, sandboxing

### Phase 11: Denial of Service Assessment

Evaluate availability threats:

**DDoS - Distributed Denial of Service (61)**
- Definition: Overwhelming systems with traffic from multiple sources
- Root Cause: Botnets, amplification attacks
- Impact: Service unavailability, revenue loss
- Mitigation: DDoS protection services, rate limiting, CDN

**Application Layer DoS (62)**
- Definition: Targeting application logic to exhaust resources
- Root Cause: Inefficient code, resource-intensive operations
- Impact: Application unavailability, degraded performance
- Mitigation: Rate limiting, caching, WAF, code optimization

**Resource Exhaustion (63)**
- Definition: Depleting CPU, memory, disk, or network resources
- Root Cause: Inefficient resource management
- Impact: System crashes, service degradation
- Mitigation: Resource quotas, monitoring, load balancing

**Slowloris Attack (64)**
- Definition: Keeping connections open with partial HTTP requests
- Root Cause: No connection timeouts
- Impact: Web server resource exhaustion
- Mitigation: Connection timeouts, request limits, reverse proxy

### Phase 12: Server-Side Request Forgery

Assess SSRF vulnerabilities:

**SSRF - Server-Side Request Forgery (66)**
- Definition: Manipulating server to make requests to internal resources
- Root Cause: Unvalidated user-controlled URLs
- Impact: Internal network access, data theft, cloud metadata access
- Mitigation: URL whitelisting, network segmentation, egress filtering

**Blind SSRF (87)**
- Definition: SSRF without direct response visibility
- Root Cause: Similar to SSRF, harder to detect
- Impact: Data exfiltration, internal reconnaissance
- Mitigation: Allowlists, WAF, network restrictions

**Time-Based Blind SSRF (88)**
- Definition: Inferring SSRF success through response timing
- Root Cause: Processing delays indicating request outcomes
- Impact: Prolonged exploitation, detection evasion
- Mitigation: Request timeouts, anomaly detection, timing monitoring

### Phase 13: Additional Web Vulnerabilities

| # | Vulnerability | Root Cause | Impact | Mitigation |
|---|--------------|-----------|--------|------------|
| 67 | HTTP Parameter Pollution | Inconsistent parsing | Injection, ACL bypass | Strict parsing, validation |
| 68 | Insecure Redirects | Unvalidated targets | Phishing, malware | Whitelist destinations |
| 69 | File Inclusion (LFI/RFI) | Unvalidated paths | Code exec, disclosure | Whitelist files, disable RFI |
| 70 | Security Header Bypass | Misconfigured headers | XSS, clickjacking | Proper headers, audits |
| 72 | Inadequate Session Timeout | Excessive timeouts | Session hijacking | Idle termination, timeouts |
| 73 | Insufficient Logging | Missing infrastructure | Detection gaps | SIEM, alerting |
| 74 | Business Logic Flaws | Insecure design | Fraud, unauthorized ops | Threat modeling, testing |

### Phase 14: Mobile and IoT Security

| # | Vulnerability | Root Cause | Impact | Mitigation |
|---|--------------|-----------|--------|------------|
| 76 | Insecure Mobile Storage | Plain text, weak crypto | Data theft | Keychain/Keystore, encrypt |
| 77 | Insecure Mobile Transmission | HTTP, cert failures | Traffic interception | TLS, cert pinning |
| 78 | Insecure Mobile APIs | Missing auth/validation | Data exposure | OAuth/JWT, validation |
| 79 | App Reverse Engineering | Hardcoded creds | Credential theft | Obfuscation, RASP |
| 80 | IoT Management Issues | Weak auth, no TLS | Device takeover | Strong auth, TLS |
| 81 | Weak IoT Authentication | Default passwords | Unauthorized access | Unique creds, MFA |
| 82 | IoT Vulnerabilities | Design flaws, old firmware | Botnet recruitment | Updates, segmentation |
| 83 | Smart Home Access | Insecure defaults | Privacy invasion | MFA, segmentation |
| 84 | IoT Privacy Issues | Excessive collection | Surveillance | Data minimization |

### Phase 15: Advanced and Zero-Day Threats

| # | Vulnerability | Root Cause | Impact | Mitigation |
|---|--------------|-----------|--------|------------|
| 89 | MIME Sniffing | Missing headers | XSS, spoofing | X-Content-Type-Options |
| 91 | CSP Bypass | Weak config | XSS despite CSP | Strict CSP, nonces |
| 92 | Inconsistent Validation | Decentralized logic | Control bypass | Centralized validation |
| 93 | Race Conditions | Missing sync | Privilege escalation | Proper locking |
| 94-95 | Business Logic Flaws | Missing validation | Financial fraud | Server-side validation |
| 96 | Account Enumeration | Different responses | Targeted attacks | Uniform responses |
| 98-99 | Unpatched Vulnerabilities | Patch delays | Zero-day exploitation | Patch management |
| 100 | Zero-Day Exploits | Unknown vulns | Unmitigated attacks | Defense in depth |

---

## Quick Reference

### Vulnerability Categories Summary

| Category | Vulnerability Numbers | Key Controls |
|----------|----------------------|--------------|
| Injection | 1-13 | Parameterized queries, input validation, output encoding |
| Authentication | 14-23, 85-86 | MFA, session management, account lockout |
| Data Exposure | 24-27 | Encryption at rest/transit, access controls, DLP |
| Misconfiguration | 28-36 | Secure defaults, hardening, patching |
| XML | 37-39, 65 | Disable external entities, limit expansion |
| Access Control | 40-44 | RBAC, least privilege, authorization checks |
| Deserialization | 45-47 | Avoid untrusted data, integrity validation |
| API Security | 48-51, 75 | OAuth, rate limiting, input validation |
| Communication | 52-55 | TLS 1.2+, certificate validation, HTTPS |
| Client-Side | 56-60 | CSP, X-Frame-Options, safe DOM |
| DoS | 61-65 | Rate limiting, DDoS protection, resource limits |
| SSRF | 66, 87-88 | URL whitelisting, egress filtering |
| Mobile/IoT | 76-84 | Encryption, authentication, secure storage |
| Business Logic | 74, 92-97 | Threat modeling, logic testing |
| Zero-Day | 98-100 | Defense in depth, threat intelligence |

### Critical Security Headers

```
Content-Security-Policy: default-src 'self'; script-src 'self'
X-Content-Type-Options: nosniff
X-Frame-Options: DENY
X-XSS-Protection: 1; mode=block
Strict-Transport-Security: max-age=31536000; includeSubDomains
Referrer-Policy: strict-origin-when-cross-origin
Permissions-Policy: geolocation=(), microphone=()
```

### OWASP Top 10 Mapping

| OWASP 2021 | Related Vulnerabilities |
|------------|------------------------|
| A01: Broken Access Control | 40-44, 23, 74 |
| A02: Cryptographic Failures | 24-25, 53-55 |
| A03: Injection | 1-13, 37-39 |
| A04: Insecure Design | 74, 92-97 |
| A05: Security Misconfiguration | 26-36 |
| A06: Vulnerable Components | 34, 98-100 |
| A07: Auth Failures | 14-23, 85-86 |
| A08: Data Integrity | 45-47 |
| A09: Logging Failures | 73 |
| A10: SSRF | 66, 87-88 |

---

## Constraints and Limitations

- Vulnerability definitions represent common patterns; specific implementations vary
- Mitigations must be adapted to technology stack and architecture
- New vulnerabilities emerge continuously; reference should be updated
- Some vulnerabilities overlap across categories (e.g., IDOR appears in multiple contexts)
- Effectiveness of mitigations depends on proper implementation
- Automated scanners cannot detect all vulnerability types (especially business logic)

---

## Troubleshooting

### Common Assessment Challenges

| Challenge | Solution |
|-----------|----------|
| False positives in scanning | Manual verification, contextual analysis |
| Business logic flaws missed | Manual testing, threat modeling, abuse case analysis |
| Encrypted traffic analysis | Proxy configuration, certificate installation |
| WAF blocking tests | Rate adjustment, IP rotation, payload encoding |
| Session handling issues | Cookie management, authentication state tracking |
| API discovery | Swagger/OpenAPI enumeration, traffic analysis |

### Vulnerability Verification Techniques

| Vulnerability Type | Verification Approach |
|-------------------|----------------------|
| Injection | Payload testing with encoded variants |
| XSS | Alert boxes, cookie access, DOM inspection |
| CSRF | Cross-origin form submission testing |
| SSRF | Out-of-band DNS/HTTP callbacks |
| XXE | External entity with controlled server |
| Access Control | Horizontal/vertical privilege testing |
| Authentication | Credential rotation, session analysis |

---

## References

- OWASP Top 10 Web Application Security Risks
- CWE/SANS Top 25 Most Dangerous Software Errors
- OWASP Testing Guide
- OWASP Application Security Verification Standard (ASVS)
- NIST Cybersecurity Framework
- Source: Kumar MS - Top 100 Web Vulnerabilities

## When to Use
This skill is applicable to execute the workflow or actions described in the overview.


---
name: track-management
description: Use this skill when creating, managing, or working with Conductor tracks - the logical work units for features, bugs, and refactors. Applies to spec.md, plan.md, and track lifecycle operations.
risk: safe
source: community
date_added: '2026-02-27'
---

# Track Management

Guide for creating, managing, and completing Conductor tracks - the logical work units that organize features, bugs, and refactors through specification, planning, and implementation phases.

## Use this skill when

- Creating new feature, bug, or refactor tracks
- Writing or reviewing spec.md files
- Creating or updating plan.md files
- Managing track lifecycle from creation to completion
- Understanding track status markers and conventions
- Working with the tracks.md registry
- Interpreting or updating track metadata

## Do not use this skill when

- The task is unrelated to track management
- You need a different domain or tool outside this scope

## Instructions

- Clarify goals, constraints, and required inputs.
- Apply relevant best practices and validate outcomes.
- Provide actionable steps and verification.
- If detailed examples are required, open `resources/implementation-playbook.md`.

## Resources

- `resources/implementation-playbook.md` for detailed patterns and examples.

## Limitations
- Use this skill only when the task clearly matches the scope described above.
- Do not treat the output as a substitute for environment-specific validation, testing, or expert review.
- Stop and ask for clarification if required inputs, permissions, safety boundaries, or success criteria are missing.


---
name: trading-ledger
description: "A trading journal that captures the decision, not just the fill: thesis, plan, and emotion at the moment of entry, written to the user's own Notion database; reviews grade decisions, not P&L."
category: productivity
risk: critical
source: community
source_repo: cruisekkk/trading-ledger
source_type: community
date_added: "2026-07-04"
author: cruisekkk
tags: [trading-journal, notion, journaling, market-wizards, decision-making]
tools: [claude]
license: "MIT"
license_source: "https://github.com/cruisekkk/trading-ledger/blob/main/LICENSE"
---

# Trading Ledger

## Overview

A journaling skill in the tradition of the *Market Wizards* interviews: a written record of every trade's decision process, reviewed on a schedule. The user reports a trade in plain language — *"bought 500 NVDA at 135, stop at 128, betting the post-earnings dip fills"* — and the agent writes ticker, size, and price to the user's own Notion database **plus the part every spreadsheet journal loses: the thesis, the plan, and the emotion.** If no reason is stated, it asks on the spot, because entry reasons decay overnight. Reviews grade decision quality against the user's own plan — a per-plan loss scores better than a lucky win. Core contract: never fabricate when unsure; mark `To-confirm` and batch-ask.

## When to Use This Skill

- Use when the user reports a trade fill ("bought 500 NVDA at 135", "closed my TSLA position", "opened 2 ES contracts short")
- Use when the user says "log a trade" or "trading ledger", or "tidy up my trading ledger"
- Use when the user asks to "review my trades"

## How It Works

### Step 1: Confirm the database (first use per session)

Use Notion `search` to find candidate **databases** (type `database`, not pages) whose title
contains **"trading-ledger"**. Before any `query` or `create-pages` call, show the user the
candidate title and `data_source_id` (`collection://...` UUID) and ask them to confirm the exact
database for this session. A single title match is not sufficient confirmation. If the connector
returns owner or schema metadata, show it as an additional identity check. Use only the
user-confirmed ID for the remainder of the session; do not re-run fuzzy selection after a
confirmation.

The companion Notion template (free, linked in the source repo) ships this schema — select values are a controlled enum, copy them exactly:

- `Entry` (title); `Ticker` (text — strikes/expiries here: NVDA / NVDA 0620C150 / ESU6)
- `Market` (select): US Stocks / US Options / US Futures / A-Shares / HK Stocks / CN Futures / Crypto / Other
- `Direction` (select): Long / Short · `Size` (text, with units)
- `Entry Price` / `Exit Price` (number) · `Entry Date` / `Exit Date` (date — expand to `"date:Entry Date:start": "YYYY-MM-DD"`; a bare value fails with HTTP 400)
- `Thesis` (text) — **the soul of the journal; if missing, ask on the spot**
- `Plan` (text) — stop / target / contingency; ask if missing
- `Emotion` (select): Calm / FOMO / Panic / Revenge / Boredom / Overconfidence — tag only what the user admits or what is plain in their words; don't diagnose
- `Execution` (select): Per plan / Early exit / Delayed stop / Impulse / Unplanned add — filled at close
- `P&L` (number, realized) · `Status` (select): Open / Closed / To-confirm / Reviewed · `Review` (text) · `Notes` (text)

### Step 2: Opening a trade

Parse → create a row with `Status=Open`. A missing thesis is the one field worth interrupting for — ask immediately. Anything else uncertain: record what you have, put the question in `Notes`, mark `To-confirm`.

### Step 3: Closing or adjusting

Find that ticker's `Status=Open` row → fill Exit Price / Exit Date / P&L / Execution → set `Closed`. Grade `Execution` against the user's own `Plan`: stopped where planned = Per plan; ran before the target = Early exit; held through the stop = Delayed stop. No matching open row → create one marked `To-confirm` and ask whether the entry was never logged.

### Step 4: Batch reconcile and review

- "tidy up my trading ledger" → query `Status=To-confirm` rows → collect **all** open questions into one message.
- "review my trades" → query recent `Closed` + all `Open` rows. For each closed trade, three questions: **Did the thesis play out?** (wrong thesis + profit = luck — say so). **How was the execution?** (a per-plan loss is a good trade). **What share of trades were emotion-tagged?** Write conclusions into `Review`, move `Closed` → `Reviewed`, and for every open position ask: *does the entry thesis still hold today?*

Parsing rules: infer `Market` from symbol and context (ambiguous → ask); bought put = Short exposure + note it is a long put; record only given prices/sizes — **never look up market prices to fill gaps**; "today" = the user's local trading date — confirm before writing.

## Examples

### Example 1: Opening with a missing plan

```text
User:  picked up some TSM July calls this morning, same thesis as before earnings
Agent: Logged ✅ TSM July calls · Long · Open
       ❓ To-confirm: which strike/expiry, how many contracts, at what price?
       ❓ And the plan — holding through earnings, or is there a stop?
User:  0731 475C, 1 contract at 18.50, holding to earnings
Agent: Updated ✅ Ticker: TSM 0731C475 · Thesis: pre-earnings run-up · Plan: hold through earnings
```

### Example 2: Review day

```text
User:  review my trades
Agent: 3 closed since last review:
       · NVDA long — thesis played out, exited per plan. Good trade.
       · TSLA short — thesis wrong, but stopped exactly where planned. Good trade, bad outcome.
       · SPY calls — profitable, but Entry was tagged FOMO and there was no plan. Lucky, not good.
       Open positions: TSM call — earnings run-up thesis still holds until the 7/29 print.
```

## Best Practices

- ✅ Ask for the thesis at entry time — it decays overnight and memory flatters
- ✅ Grade execution against the user's own plan, never against hindsight
- ✅ Batch all questions into one message; give a short receipt after logging
- ❌ Don't compute P&L you are unsure of — options and futures have multipliers; use the user's numbers
- ❌ Don't invent select values; copy the enum
- ❌ A review is not a P&L total — it grades thesis and execution

## Limitations

- Requires the user's own Notion workspace, the companion database (duplicate the template from the source repo), and the official Notion connector granted access — connector/skill setups currently live on paid Claude tiers.
- Close, reconcile, and review flows require reliable Notion database query/search access to open, closed, and `To-confirm` rows. If the connector cannot query the ledger, stop after logging the user's new facts and ask them to provide the relevant row details instead of guessing matches.
- No broker integration, by design: the broker knows the fills; only the user knows the reasons. The user must self-report.
- Grading honesty depends on input honesty: a thesis backfilled three days later defeats the point (the skill nags, but cannot prevent it).

## Security & Safety Notes

- **Mutation scope**: writes and updates rows only in the exact Notion database ID confirmed by
  the user for the current session via the official Notion connector (MCP) — no shell commands,
  no network fetches, no market-data lookups, no credentials.
- **This skill must never produce trading signals, price data, or buy/sell recommendations.** It records and mirrors the user's own decisions; the review asks questions, it does not advise. Nothing it writes is financial advice, and it should say so if asked for a recommendation.
- On claude.ai, Notion's write tools default to *needs approval* — the first write pops an approval prompt; expected, not a hang.

## Common Pitfalls

- **Problem:** Notion API 400 on date fields.
  **Solution:** Expand to `"date:Entry Date:start": "YYYY-MM-DD"`.
- **Problem:** `create-pages` succeeds but the date column is empty (known Notion MCP issue: [notion-mcp-server#121](https://github.com/makenotion/notion-mcp-server/issues/121) — expanded date fields silently dropped).
  **Solution:** After the session's first create, read the row back; if the date is empty, fill it with `update-page`.
- **Problem:** Close matched to the wrong row when the same ticker was traded twice.
  **Solution:** Match on `Status=Open` + ticker; if multiple open rows match, ask which one.
- **Problem:** Overnight US fills dated to the wrong day for non-US users.
  **Solution:** Overnight fills belong to the US trading date; confirm the date before writing.

## Related Skills

- `@time-ledger` - The same "parse plain language → own Notion DB → ask instead of guessing" pattern applied to time tracking.


---
name: train-sentence-transformers
description: Train or fine-tune sentence-transformers models across `SentenceTransformer` (bi-encoder; dense or static embedding model; for retrieval, similarity, clustering, classification, paraphrase mining, dedup, multimodal), `CrossEncoder` (reranker; pair scoring for two-stage retrieval / pair...
risk: critical
source: https://github.com/huggingface/skills/tree/main/skills/train-sentence-transformers
source_repo: huggingface/skills
source_type: official
date_added: 2026-07-01
license: Apache-2.0
license_source: https://github.com/huggingface/skills/blob/main/LICENSE
---

# Train a sentence-transformers Model
## When to Use

Use this skill when you need train or fine-tune sentence-transformers models across `SentenceTransformer` (bi-encoder; dense or static embedding model; for retrieval, similarity, clustering, classification, paraphrase mining, dedup, multimodal), `CrossEncoder` (reranker; pair scoring for two-stage retrieval / pair...

**This SKILL.md is a router, not a manual.** It tells you which references and example scripts to load for your task. The actual content — recommended losses, evaluators, training-script structure, model selection, training-arg knobs, troubleshooting — lives in `references/` and `scripts/`.

**Do not synthesize a training script from this file alone.** Open the per-type production template (`scripts/train_<type>_example.py`) and copy it as your starting point. The templates contain load-bearing scaffolding (autocast helper, model-card class, logger silencing list, `force=True`, `seed`, TF32, version-compatible imports, named-evaluator metric handling) that prior agent runs have repeatedly missed when rolling their own from a synthesized snippet.

## 1. Identify the model type

| Tag | Class | What it does | When to pick |
|---|---|---|---|
| **[SentenceTransformer]** | `SentenceTransformer` (bi-encoder) | Maps each input to a fixed-dim dense vector | Retrieval, similarity, clustering, classification, paraphrase mining, dedup |
| **[CrossEncoder]** | `CrossEncoder` (reranker) | Scores `(query, passage)` pairs jointly | Two-stage retrieval (rerank top-100 from bi-encoder), pair classification |
| **[SparseEncoder]** | `SparseEncoder` (SPLADE) | Sparse vectors over the vocabulary | Learned-sparse retrieval, inverted-index backends (Elasticsearch / OpenSearch / Lucene) |

Tiebreakers when the request is ambiguous: "embedding model" / "vector search" / "similarity" → **[SentenceTransformer]**. "rerank" / "ranker" / "two-stage" → **[CrossEncoder]**. "SPLADE" / "sparse" / "inverted index" → **[SparseEncoder]**. If still unclear, ask.

## 2. Required reading

**Read these in full before writing any code. Do not triage by perceived relevance.**

### Per-type — always required

**[SentenceTransformer]**
- `references/losses_sentence_transformer.md` — loss-to-data-shape mapping; `BatchSamplers.NO_DUPLICATES` requirement for MNRL-family; `Cached*` ↔ `gradient_checkpointing` incompatibility.
- `references/evaluators_sentence_transformer.md` — evaluator-to-task mapping; `metric_for_best_model` key construction (named vs unnamed); per-evaluator `primary_metric` values.
- `references/model_architectures.md` — encoder vs decoder vs static vs Router pipelines; pooling rules (mean / cls / lasttoken); auto-mean-pooling behavior for fresh-start MLM bases.
- `scripts/train_sentence_transformer_example.py` — production template; copy this as your starting point.

**[CrossEncoder]**
- `references/losses_cross_encoder.md` — pointwise / pairwise / listwise / distillation; `pos_weight` derivation; `activation_fn=Identity()` mandatory for non-BCE losses (silent eval-rank collapse otherwise).
- `references/evaluators_cross_encoder.md` — `CrossEncoderRerankingEvaluator` recipe; named-evaluator key format `eval_{name}_{primary_metric}`.
- `scripts/train_cross_encoder_example.py` — production template; copy this as your starting point.

**[SparseEncoder]**
- `references/losses_sparse_encoder.md` — `SpladeLoss` wrapper requirement; FLOPS regularizer weights; smoke-test active-dim ramp behavior.
- `references/evaluators_sparse_encoder.md` — `SparseNanoBEIREvaluator` (English-only) and the in-domain alternative; `eval_{name}_{primary_metric}` key format.
- `scripts/train_sparse_encoder_example.py` — production template; copy this as your starting point.

### Cross-cutting — always required (regardless of task)

- `references/training_args.md` — `TrainingArguments` knobs, precision rules (load fp32 + autocast bf16/fp16; never `torch_dtype=bfloat16`), `warmup_steps` (float) vs deprecated `warmup_ratio`, `save_steps` must be a multiple of `eval_steps` for `load_best_model_at_end`, schedulers, HPO, tracker, resume, hub-push variants.
- `references/dataset_formats.md` — column-matching rules (label name auto-detection; column-order-not-name); reshaping recipes; hard-negative mining options.
- `references/base_model_selection.md` — discovery commands; per-type model namespaces; ModernBERT-family `max_seq_length=8192` trap; `datasets >= 4` script-loader rejection; non-English starting-point shortcuts.
- `references/troubleshooting.md` — symptom-indexed failure recipes. Skim the section headings on every run, even a healthy one; the "Metrics don't improve" and "Hub push fails" entries cover bugs that bite frequently and are cheaper to recognize before they fire than to debug after.

### Cross-cutting — load when applicable

- `references/hardware_guide.md` — VRAM sizing, multi-GPU, FSDP / DeepSpeed, HF Jobs flavors. Required for >24GB models, multi-GPU, or HF Jobs runs.
- `references/hf_jobs_execution.md` — required when running on HF Jobs.
- `references/prompts_and_instructions.md` — required when using prompt-tuned bases (E5, BGE, GTE, Qwen3-Embedding, Instructor, Nomic, etc.) or adding `query: ` / `passage: ` style prefixes.

### Variant scripts (open when the task matches)
- **[SentenceTransformer]** `scripts/train_sentence_transformer_<matryoshka|multi_dataset|with_lora|distillation|make_multilingual|static_embedding>_example.py`.
- **[CrossEncoder]** `scripts/train_cross_encoder_<distillation|listwise>_example.py`.
- **[SparseEncoder]** `scripts/train_sparse_encoder_distillation_example.py`.
- Hard-negative mining CLI — `scripts/mine_hard_negatives.py`.

## 3. Defaults

Override only if the user specifies otherwise:
- **Local execution.** Pitch HF Jobs only if local hardware can't fit the job.
- **Single run.** After it completes, propose experimentation if the user would benefit (weak/marginal verdict, "see how high you can push it" framing, etc.). Iteration rules in `references/training_args.md` (Experimentation section).
- **Public Hub push at end-of-run, wrapped in try-except.** On HF Jobs (ephemeral env) ALSO enable in-trainer push (`push_to_hub=True` + `hub_strategy="every_save"`); details in `references/hf_jobs_execution.md`.

## 4. Constraints the produced script must satisfy

These are non-negotiable contracts. Implementation lives in the production templates and references — do not reinvent.

- Capture the pre-training evaluator score as `baseline_eval` **before** `trainer.train()`.
- Emit a single end-of-run line: `VERDICT: WIN|MARGINAL|REGRESSION | score=... | baseline=... | delta=...`. A monitor scrapes for this.
- Silence `httpx`, `httpcore`, `huggingface_hub`, `urllib3`, `filelock`, `fsspec` to WARNING (otherwise HF download URLs flood the agent's context).
- Tee logs to `logs/{RUN_NAME}.log`.
- End with `model.push_to_hub(...)` wrapped in `try/except`.
- Smoke-test before any long run (`max_steps=1` + tiny dataset slice). The production templates show one common pattern (`SMOKE_TEST` env var).
- **[CrossEncoder]** Include `EarlyStoppingCallback(patience>=3)` — CE rerankers often peak mid-training and regress.
- **[SparseEncoder]** Log `query_active_dims` / `corpus_active_dims` on the verdict line; high nDCG with collapsed sparsity is not a win. The keys come back name-prefixed (e.g. `..._query_active_dims`); use suffix matching to pluck them — see the SPARSE production template for the exact pattern.

## 5. Workflow

1. Identify the model type (§1). Ask if ambiguous.
2. Load the §2 required-reading files for that type.
3. Open `scripts/train_<type>_example.py` and copy it as your starting point.
4. Replace `MODEL_NAME`, `DATASET_NAME`, `RUN_NAME`, the loss, and the evaluator with the user's task. Cross-check loss/data-shape match against `references/losses_<type>.md`; cross-check the `metric_for_best_model` key against `references/evaluators_<type>.md` (named evaluators format the key as `eval_{name}_{primary_metric}`).
5. Smoke-test (`max_steps=1`).
6. Run.
7. After the run, append to `logs/experiments.md` and propose iteration if the verdict is weak/marginal.

## Prerequisites

```bash
pip install "sentence-transformers[train]>=5.0"        # add [train,image] / [audio] / [video] for [SentenceTransformer] multimodal
pip install trackio                                    # optional tracker; or wandb / tensorboard / mlflow
hf auth login                                          # or set HF_TOKEN with write scope (for Hub push)
```

GPU strongly recommended. CPU works only for demos and `[SentenceTransformer]` `StaticEmbedding`.

## Limitations

- Use this skill only when the task clearly matches its upstream product or API scope.
- Verify commands, API behavior, pricing, quotas, credentials, and deployment effects against current official documentation before making changes.
- Do not treat generated examples as a substitute for environment-specific tests, security review, or user approval for destructive or costly actions.


---
name: transformers-js
description: Use Transformers.js to run state-of-the-art machine learning models directly in JavaScript/TypeScript. Supports NLP (text classification, translation, summarization), computer vision (image classification, object detection), audio (speech recognition, audio classification), and...
risk: critical
source: https://github.com/huggingface/skills/tree/main/skills/transformers-js
source_repo: huggingface/skills
source_type: official
date_added: 2026-07-01
license: Apache-2.0
license_source: https://github.com/huggingface/skills/blob/main/LICENSE
---

# Transformers.js - Machine Learning for JavaScript

Transformers.js enables running state-of-the-art machine learning models directly in JavaScript across browsers and server-side runtimes (Node.js, Bun, Deno), with no Python server required.

## When to Use This Skill

Use this skill when you need to:
- Run ML models for text analysis, generation, or translation in JavaScript
- Perform image classification, object detection, or segmentation
- Implement speech recognition or audio processing
- Build multimodal AI applications (text-to-image, image-to-text, etc.)
- Run models client-side in the browser without a backend

## Installation

### NPM Installation
```bash
npm install @huggingface/transformers
```

### Browser Usage (CDN)
```javascript
<script type="module">
  import { pipeline } from 'https://cdn.jsdelivr.net/npm/@huggingface/transformers';
</script>
```

## Core Concepts

### 1. Pipeline API
The pipeline API is the easiest way to use models. It groups together preprocessing, model inference, and postprocessing:

```javascript
import { pipeline } from '@huggingface/transformers';

// Create a pipeline for a specific task
const pipe = await pipeline('sentiment-analysis');

// Use the pipeline
const result = await pipe('I love transformers!');
// Output: [{ label: 'POSITIVE', score: 0.999817686 }]

// IMPORTANT: Always dispose when done to free memory
await pipe.dispose();
```

**⚠ Memory Management:** All pipelines must be disposed with `pipe.dispose()` when finished to prevent memory leaks. See examples in [Code Examples](./references/EXAMPLES.md) for cleanup patterns across different environments.

### 2. Model Selection
You can specify a custom model as the second argument:

```javascript
const pipe = await pipeline(
  'sentiment-analysis',
  'Xenova/bert-base-multilingual-uncased-sentiment'
);
```

**Finding Models:**

Browse available Transformers.js models on Hugging Face Hub:
- **All models**: https://huggingface.co/models?library=transformers.js&sort=trending
- **By task**: Add `pipeline_tag` parameter
  - Text generation: https://huggingface.co/models?pipeline_tag=text-generation&library=transformers.js&sort=trending
  - Image classification: https://huggingface.co/models?pipeline_tag=image-classification&library=transformers.js&sort=trending
  - Speech recognition: https://huggingface.co/models?pipeline_tag=automatic-speech-recognition&library=transformers.js&sort=trending

**Tip:** Filter by task type, sort by trending/downloads, and check model cards for performance metrics and usage examples.

### 3. Device Selection
Choose where to run the model:

```javascript
// Run on CPU (default for WASM)
const pipe = await pipeline('sentiment-analysis', 'model-id');

// Run on GPU (WebGPU)
const pipe = await pipeline('sentiment-analysis', 'model-id', {
  device: 'webgpu',
});
```

### 4. Quantization Options
Control model precision vs. performance:

```javascript
// Use quantized model (faster, smaller)
const pipe = await pipeline('sentiment-analysis', 'model-id', {
  dtype: 'q4',  // Options: 'fp32', 'fp16', 'q8', 'q4'
});
```

## Supported Tasks

**Note:** All examples below show basic usage.

### Natural Language Processing

#### Text Classification
```javascript
const classifier = await pipeline('text-classification');
const result = await classifier('This movie was amazing!');
```

#### Named Entity Recognition (NER)
```javascript
const ner = await pipeline('token-classification');
const entities = await ner('My name is John and I live in New York.');
```

#### Question Answering
```javascript
const qa = await pipeline('question-answering');
const answer = await qa({
  question: 'What is the capital of France?',
  context: 'Paris is the capital and largest city of France.'
});
```

#### Text Generation
```javascript
const generator = await pipeline('text-generation', 'onnx-community/gemma-3-270m-it-ONNX');
const text = await generator('Once upon a time', {
  max_new_tokens: 100,
  temperature: 0.7
});
```

**For streaming and chat:** See **[Text Generation Guide](./references/TEXT_GENERATION.md)** for:
- Streaming token-by-token output with `TextStreamer`
- Chat/conversation format with system/user/assistant roles
- Generation parameters (temperature, top_k, top_p)
- Browser and Node.js examples
- React components and API endpoints

#### Translation
```javascript
const translator = await pipeline('translation', 'Xenova/nllb-200-distilled-600M');
const output = await translator('Hello, how are you?', {
  src_lang: 'eng_Latn',
  tgt_lang: 'fra_Latn'
});
```

#### Summarization
```javascript
const summarizer = await pipeline('summarization');
const summary = await summarizer(longText, {
  max_length: 100,
  min_length: 30
});
```

#### Zero-Shot Classification
```javascript
const classifier = await pipeline('zero-shot-classification');
const result = await classifier('This is a story about sports.', ['politics', 'sports', 'technology']);
```

### Computer Vision

#### Image Classification
```javascript
const classifier = await pipeline('image-classification');
const result = await classifier('https://example.com/image.jpg');
// Or with local file
const result = await classifier(imageUrl);
```

#### Object Detection
```javascript
const detector = await pipeline('object-detection');
const objects = await detector('https://example.com/image.jpg');
// Returns: [{ label: 'person', score: 0.95, box: { xmin, ymin, xmax, ymax } }, ...]
```

#### Image Segmentation
```javascript
const segmenter = await pipeline('image-segmentation');
const segments = await segmenter('https://example.com/image.jpg');
```

#### Depth Estimation
```javascript
const depthEstimator = await pipeline('depth-estimation');
const depth = await depthEstimator('https://example.com/image.jpg');
```

#### Zero-Shot Image Classification
```javascript
const classifier = await pipeline('zero-shot-image-classification');
const result = await classifier('image.jpg', ['cat', 'dog', 'bird']);
```

### Audio Processing

#### Automatic Speech Recognition
```javascript
const transcriber = await pipeline('automatic-speech-recognition');
const result = await transcriber('audio.wav');
// Returns: { text: 'transcribed text here' }
```

#### Audio Classification
```javascript
const classifier = await pipeline('audio-classification');
const result = await classifier('audio.wav');
```

#### Text-to-Speech
```javascript
const synthesizer = await pipeline('text-to-speech', 'Xenova/speecht5_tts');
const audio = await synthesizer('Hello, this is a test.', {
  speaker_embeddings: speakerEmbeddings
});
```

### Multimodal

#### Image-to-Text (Image Captioning)
```javascript
const captioner = await pipeline('image-to-text');
const caption = await captioner('image.jpg');
```

#### Document Question Answering
```javascript
const docQA = await pipeline('document-question-answering');
const answer = await docQA('document-image.jpg', 'What is the total amount?');
```

#### Zero-Shot Object Detection
```javascript
const detector = await pipeline('zero-shot-object-detection');
const objects = await detector('image.jpg', ['person', 'car', 'tree']);
```

### Feature Extraction (Embeddings)

```javascript
const extractor = await pipeline('feature-extraction');
const embeddings = await extractor('This is a sentence to embed.');
// Returns: tensor of shape [1, sequence_length, hidden_size]

// For sentence embeddings (mean pooling)
const extractor = await pipeline('feature-extraction', 'onnx-community/all-MiniLM-L6-v2-ONNX');
const embeddings = await extractor('Text to embed', { pooling: 'mean', normalize: true });
```

## Finding and Choosing Models

### Browsing the Hugging Face Hub

Discover compatible Transformers.js models on Hugging Face Hub:

**Base URL (all models):**
```
https://huggingface.co/models?library=transformers.js&sort=trending
```

**Filter by task** using the `pipeline_tag` parameter:

| Task | URL |
|------|-----|
| **Text Generation** | https://huggingface.co/models?pipeline_tag=text-generation&library=transformers.js&sort=trending |
| **Text Classification** | https://huggingface.co/models?pipeline_tag=text-classification&library=transformers.js&sort=trending |
| **Translation** | https://huggingface.co/models?pipeline_tag=translation&library=transformers.js&sort=trending |
| **Summarization** | https://huggingface.co/models?pipeline_tag=summarization&library=transformers.js&sort=trending |
| **Question Answering** | https://huggingface.co/models?pipeline_tag=question-answering&library=transformers.js&sort=trending |
| **Image Classification** | https://huggingface.co/models?pipeline_tag=image-classification&library=transformers.js&sort=trending |
| **Object Detection** | https://huggingface.co/models?pipeline_tag=object-detection&library=transformers.js&sort=trending |
| **Image Segmentation** | https://huggingface.co/models?pipeline_tag=image-segmentation&library=transformers.js&sort=trending |
| **Speech Recognition** | https://huggingface.co/models?pipeline_tag=automatic-speech-recognition&library=transformers.js&sort=trending |
| **Audio Classification** | https://huggingface.co/models?pipeline_tag=audio-classification&library=transformers.js&sort=trending |
| **Image-to-Text** | https://huggingface.co/models?pipeline_tag=image-to-text&library=transformers.js&sort=trending |
| **Feature Extraction** | https://huggingface.co/models?pipeline_tag=feature-extraction&library=transformers.js&sort=trending |
| **Zero-Shot Classification** | https://huggingface.co/models?pipeline_tag=zero-shot-classification&library=transformers.js&sort=trending |

**Sort options:**
- `&sort=trending` - Most popular recently
- `&sort=downloads` - Most downloaded overall
- `&sort=likes` - Most liked by community
- `&sort=modified` - Recently updated

### Choosing the Right Model

Consider these factors when selecting a model:

**1. Model Size**
- **Small (< 100MB)**: Fast, suitable for browsers, limited accuracy
- **Medium (100MB - 500MB)**: Balanced performance, good for most use cases
- **Large (> 500MB)**: High accuracy, slower, better for Node.js or powerful devices

**2. Quantization**
Models are often available in different quantization levels:
- `fp32` - Full precision (largest, most accurate)
- `fp16` - Half precision (smaller, still accurate)
- `q8` - 8-bit quantized (much smaller, slight accuracy loss)
- `q4` - 4-bit quantized (smallest, noticeable accuracy loss)

**3. Task Compatibility**
Check the model card for:
- Supported tasks (some models support multiple tasks)
- Input/output formats
- Language support (multilingual vs. English-only)
- License restrictions

**4. Performance Metrics**
Model cards typically show:
- Accuracy scores
- Benchmark results
- Inference speed
- Memory requirements

### Example: Finding a Text Generation Model

```javascript
// 1. Visit: https://huggingface.co/models?pipeline_tag=text-generation&library=transformers.js&sort=trending

// 2. Browse and select a model (e.g., onnx-community/gemma-3-270m-it-ONNX)

// 3. Check model card for:
//    - Model size: ~270M parameters
//    - Quantization: q4 available
//    - Language: English
//    - Use case: Instruction-following chat

// 4. Use the model:
import { pipeline } from '@huggingface/transformers';

const generator = await pipeline(
  'text-generation',
  'onnx-community/gemma-3-270m-it-ONNX',
  { dtype: 'q4' } // Use quantized version for faster inference
);

const output = await generator('Explain quantum computing in simple terms.', {
  max_new_tokens: 100
});

await generator.dispose();
```

### Tips for Model Selection

1. **Start Small**: Test with a smaller model first, then upgrade if needed
2. **Check ONNX Support**: Ensure the model has ONNX files (look for `onnx` folder in model repo)
3. **Read Model Cards**: Model cards contain usage examples, limitations, and benchmarks
4. **Test Locally**: Benchmark inference speed and memory usage in your environment
5. **Filter by Library**: Use `library=transformers.js` to find compatible models: https://huggingface.co/models?library=transformers.js
6. **Version Pin**: Use specific git commits in production for stability:
   ```javascript
   const pipe = await pipeline('task', 'model-id', { revision: 'abc123' });
   ```

## Advanced Configuration

### Environment Configuration (`env`)

The `env` object provides comprehensive control over Transformers.js execution, caching, and model loading.

**Quick Overview:**

```javascript
import { env, LogLevel } from '@huggingface/transformers';

// View version
console.log(env.version); // e.g., '4.x'

// Common settings
env.allowRemoteModels = true;  // Load from Hugging Face Hub
env.allowLocalModels = false;  // Load from file system
env.localModelPath = '/models/'; // Local model directory
env.useFSCache = true;         // Cache models on disk (Node.js)
env.useBrowserCache = true;    // Cache models in browser
env.cacheDir = './.cache';     // Cache directory location
// Optional: override logging level (default is LogLevel.WARNING)
env.logLevel = LogLevel.INFO;

// Optional: custom fetch for auth headers, retries, abort signals, etc.
env.fetch = (url, options) =>
  fetch(url, {
    ...options,
    headers: {
      ...options?.headers,
      Authorization: `Bearer ${HF_TOKEN}`,
    },
  });
```

**Configuration Patterns:**

```javascript
// Development: Fast iteration with remote models
env.allowRemoteModels = true;
env.useFSCache = true;

// Production: Local models only
env.allowRemoteModels = false;
env.allowLocalModels = true;
env.localModelPath = '/app/models/';

// Custom CDN
env.remoteHost = 'https://cdn.example.com/models';

// Disable caching (testing)
env.useFSCache = false;
env.useBrowserCache = false;
```

For complete documentation on all configuration options, caching strategies, cache management, pre-downloading models, and more, see:

**→ [Configuration Reference](./references/CONFIGURATION.md)**

### ModelRegistry (v4)

`ModelRegistry` gives you visibility and control over model assets before loading a pipeline. Use it to estimate download size, check cache status, inspect available dtypes, and clear cached artifacts for a specific task/model/options tuple.

```javascript
import { ModelRegistry } from '@huggingface/transformers';

const task = 'feature-extraction';
const modelId = 'onnx-community/all-MiniLM-L6-v2-ONNX';
const modelOptions = { dtype: 'fp32' };

// List required files for this pipeline
const files = await ModelRegistry.get_pipeline_files(task, modelId, modelOptions);

// Check if assets are already cached
const cached = await ModelRegistry.is_pipeline_cached(task, modelId, modelOptions);

// Inspect precision formats available for this model
const dtypes = await ModelRegistry.get_available_dtypes(modelId);

console.log({ files: files.length, cached, dtypes });
```

For production patterns and full API coverage, see **[ModelRegistry Reference](./references/MODEL_REGISTRY.md)**.

### Standalone Tokenization (`@huggingface/tokenizers`)

For tokenization-only workflows, use `@huggingface/tokenizers`. It is a separate lightweight package useful when you need fast tokenization/encoding without loading full model inference pipelines.

```bash
npm install @huggingface/tokenizers
```

```javascript
import { Tokenizer } from '@huggingface/tokenizers';
```

### Working with Tensors

```javascript
import { AutoTokenizer, AutoModel } from '@huggingface/transformers';

// Load tokenizer and model separately for more control
const tokenizer = await AutoTokenizer.from_pretrained('bert-base-uncased');
const model = await AutoModel.from_pretrained('bert-base-uncased');

// Tokenize input
const inputs = await tokenizer('Hello world!');

// Run model
const outputs = await model(inputs);
```

### Batch Processing

```javascript
const classifier = await pipeline('sentiment-analysis');

// Process multiple texts
const results = await classifier([
  'I love this!',
  'This is terrible.',
  'It was okay.'
]);
```

## Runtime-Specific Considerations

### WebGPU Usage
WebGPU provides GPU acceleration in browsers and server-side runtimes (when supported):

```javascript
const pipe = await pipeline('text-generation', 'onnx-community/gemma-3-270m-it-ONNX', {
  device: 'webgpu',
  dtype: 'fp32'
});
```

**Note**: Use `webgpu` when available and fall back to WASM/CPU when not supported in the current runtime.

### WASM Performance
WASM is the most compatible execution backend across runtimes:

```javascript
// Optimized for browsers with quantization
const pipe = await pipeline('sentiment-analysis', 'model-id', {
  dtype: 'q8'  // or 'q4' for even smaller size
});
```

### Progress Tracking & Loading Indicators

Models can be large (ranging from a few MB to several GB) and consist of multiple files. Track download progress by passing a callback to the `pipeline()` function:

```javascript
import { pipeline } from '@huggingface/transformers';

// Track progress for each file
const fileProgress = {};

function onProgress(info) {
  if (info.status === 'progress_total') {
    console.log(`Total: ${info.progress.toFixed(1)}%`);
    return;
  }

  console.log(`${info.status}: ${info.file ?? ''}`);

  if (info.status === 'progress') {
    fileProgress[info.file] = info.progress;
    console.log(`${info.file}: ${info.progress.toFixed(1)}%`);
  }

  if (info.status === 'done') {
    console.log(`✓ ${info.file} complete`);
  }
}

// Pass callback to pipeline
const classifier = await pipeline('sentiment-analysis', null, {
  progress_callback: onProgress
});
```

**Progress Info Properties:**

```typescript
interface ProgressInfo {
  status: 'initiate' | 'download' | 'progress' | 'progress_total' | 'done' | 'ready';
  name: string;      // Model id or path
  file?: string;     // File being processed (per-file events)
  progress?: number; // Percentage (0-100, for 'progress' and 'progress_total')
  loaded?: number;   // Bytes downloaded (only for 'progress' status)
  total?: number;    // Total bytes (only for 'progress' status)
}
```

For complete examples including browser UIs, React components, CLI progress bars, and retry logic, see:

**→ [Pipeline Options - Progress Callback](./references/PIPELINE_OPTIONS.md#progress-callback)**

## Error Handling

```javascript
try {
  const pipe = await pipeline('sentiment-analysis', 'model-id');
  const result = await pipe('text to analyze');
} catch (error) {
  if (error.message.includes('fetch')) {
    console.error('Model download failed. Check internet connection.');
  } else if (error.message.includes('ONNX')) {
    console.error('Model execution failed. Check model compatibility.');
  } else {
    console.error('Unknown error:', error);
  }
}
```

## Performance Tips

1. **Reuse Pipelines**: Create pipeline once, reuse for multiple inferences
2. **Use Quantization**: Start with `q8` or `q4` for faster inference
3. **Batch Processing**: Process multiple inputs together when possible
4. **Cache Models**: Models are cached automatically (see **[Caching Reference](./references/CACHE.md)** for details on browser Cache API, Node.js filesystem cache, and custom implementations)
5. **WebGPU for Large Models**: Use WebGPU for models that benefit from GPU acceleration
6. **Prune Context**: For text generation, limit `max_new_tokens` to avoid memory issues
7. **Clean Up Resources**: Call `pipe.dispose()` when done to free memory

## Memory Management

**IMPORTANT:** Always call `pipe.dispose()` when finished to prevent memory leaks.

```javascript
const pipe = await pipeline('sentiment-analysis');
const result = await pipe('Great product!');
await pipe.dispose();  // ✓ Free memory (100MB - several GB per model)
```

**When to dispose:**
- Application shutdown or component unmount
- Before loading a different model
- After batch processing in long-running apps

Models consume significant memory and hold GPU/CPU resources. Disposal is critical for browser memory limits and server stability.

For detailed patterns (React cleanup, servers, browser), see **[Code Examples](./references/EXAMPLES.md)**

## Troubleshooting

### Model Not Found
- Verify model exists on Hugging Face Hub
- Check model name spelling
- Ensure model has ONNX files (look for `onnx` folder in model repo)

### Memory Issues
- Use smaller models or quantized versions (`dtype: 'q4'`)
- Reduce batch size
- Limit sequence length with `max_length`

### WebGPU Errors
- Check browser compatibility (Chrome 113+, Edge 113+)
- Try `dtype: 'fp16'` if `fp32` fails
- Fall back to WASM if WebGPU unavailable

## Reference Documentation

### This Skill
- **[Pipeline Options](./references/PIPELINE_OPTIONS.md)** - Configure `pipeline()` with `progress_callback`, `device`, `dtype`, etc.
- **[Configuration Reference](./references/CONFIGURATION.md)** - Global `env` configuration for caching and model loading
- **[ModelRegistry Reference](./references/MODEL_REGISTRY.md)** - Inspect files, cache status, dtypes, and clear cache before loading pipelines
- **[Caching Reference](./references/CACHE.md)** - Browser Cache API, Node.js filesystem cache, and custom cache implementations
- **[Text Generation Guide](./references/TEXT_GENERATION.md)** - Streaming, chat format, and generation parameters
- **[Model Architectures](./references/MODEL_ARCHITECTURES.md)** - Supported models and selection tips
- **[Code Examples](./references/EXAMPLES.md)** - Real-world implementations for different runtimes

### Official Transformers.js
- Official docs: https://huggingface.co/docs/transformers.js
- API reference: https://huggingface.co/docs/transformers.js/api/pipelines
- Model hub: https://huggingface.co/models?library=transformers.js
- GitHub: https://github.com/huggingface/transformers.js
- Examples: https://github.com/huggingface/transformers.js-examples

## Best Practices

1. **Always Dispose Pipelines**: Call `pipe.dispose()` when done - critical for preventing memory leaks
2. **Start with Pipelines**: Use the pipeline API unless you need fine-grained control
3. **Test Locally First**: Test models with small inputs before deploying
4. **Monitor Model Sizes**: Be aware of model download sizes for web applications
5. **Handle Loading States**: Show progress indicators for better UX
6. **Version Pin**: Pin specific model versions for production stability
7. **Error Boundaries**: Always wrap pipeline calls in try-catch blocks
8. **Progressive Enhancement**: Provide fallbacks for unsupported browsers
9. **Reuse Models**: Load once, use many times - don't recreate pipelines unnecessarily
10. **Graceful Shutdown**: Dispose models on SIGTERM/SIGINT in servers

## Quick Reference: Task IDs

| Task | Task ID |
|------|---------|
| Text classification | `text-classification` or `sentiment-analysis` |
| Token classification | `token-classification` or `ner` |
| Question answering | `question-answering` |
| Fill mask | `fill-mask` |
| Summarization | `summarization` |
| Translation | `translation` |
| Text generation | `text-generation` |
| Text-to-text generation | `text2text-generation` |
| Zero-shot classification | `zero-shot-classification` |
| Image classification | `image-classification` |
| Image segmentation | `image-segmentation` |
| Object detection | `object-detection` |
| Depth estimation | `depth-estimation` |
| Image-to-image | `image-to-image` |
| Zero-shot image classification | `zero-shot-image-classification` |
| Zero-shot object detection | `zero-shot-object-detection` |
| Automatic speech recognition | `automatic-speech-recognition` |
| Audio classification | `audio-classification` |
| Text-to-speech | `text-to-speech` or `text-to-audio` |
| Image-to-text | `image-to-text` |
| Document question answering | `document-question-answering` |
| Feature extraction | `feature-extraction` |
| Sentence similarity | `sentence-similarity` |

---

This skill enables you to integrate state-of-the-art machine learning capabilities directly into JavaScript applications without requiring separate ML servers or Python environments.

## Limitations

- Use this skill only when the task clearly matches its upstream product or API scope.
- Verify commands, API behavior, pricing, quotas, credentials, and deployment effects against current official documentation before making changes.
- Do not treat generated examples as a substitute for environment-specific tests, security review, or user approval for destructive or costly actions.


---
name: travel-health-analyzer
description: 分析旅行健康数据、评估目的地健康风险、提供疫苗接种建议、生成多语言紧急医疗信息卡片。支持WHO/CDC数据集成的专业级旅行健康风险评估。
allowed-tools: Read, Write, Grep, Glob
risk: critical
source: community
---

# 旅行健康分析技能

## When to Use
- 需要做旅行前健康准备、目的地健康风险评估或疫苗建议时使用。
- 任务涉及 WHO/CDC 风险信息、旅行药箱、预防措施或多语言医疗卡片。
- 用户请求旅行健康规划或目的地卫生风险分析时使用。

## 🚨 重要医学免责声明

**本技能提供的所有健康建议和信息仅供参考,不能替代专业医疗建议。**

- ⚠ **所有建议必须由专业医生审核**
- ⚠ **疫苗接种和用药方案必须由医生制定**
- ⚠ **不提供具体的医疗处方或诊断**
- ⚠ **健康风险数据来源于WHO/CDC,可能存在滞后性**
- ⚠ **紧急情况下请立即就医**

---

## 技能功能

### 1. 旅行健康规划分析

分析用户的旅行计划,提供全面的健康准备建议。

**输入**: 旅行目的地、日期、旅行目的
**输出**:
- 目的地健康风险评估
- 必要和推荐的疫苗接种清单
- 旅行药箱建议清单
- 预防措施建议
- 旅行前准备时间表

**分析要点**:
- 识别目的地传染病风险
- 评估食物和饮水安全
- 确认环境风险(高温、高原等)
- 检查当前疫情爆发信息
- 提供WHO/CDC参考链接

---

### 2. 目的地健康风险评估

基于WHO/CDC数据,对旅行目的地进行专业级健康风险评估。

**数据源**:
- 世界卫生组织(WHO)国际旅行健康
- 美国疾控中心(CDC)旅行健康
- 当地卫生部门官方数据

**评估维度**:
- 传染病风险(登革热、疟疾、霍乱、甲肝等)
- 食物和饮水安全
- 环境风险(高温、高原、空气污染)
- 季节性风险
- 当前疫情爆发警报

**风险等级**:
- 🟢 **低风险** - 常规预防措施
- 🟡 **中等风险** - 需要特别注意
- 🔴 **高风险** - 需要采取严格预防措施
- ⚫ **极高风险** - 建议推迟旅行或采取特殊防护

**输出格式**:
```markdown
## 目的地健康风险评估: Thailand

### 传染病风险
#### 🔴 登革热 - 高风险
- **传播方式**: 蚊子叮咬
- **季节性**: 全年
- **症状**: 高热、头痛、肌肉关节痛、皮疹
- **预防**: 使用防蚊液、穿长袖衣物、住宿选择有空调房间
- **数据源**: [WHO](https://www.who.int/ith) | [CDC](https://www.cdc.gov/dengue)

### 食物饮水安全
#### 🟡 中等风险
- 饮用瓶装水或煮沸的水
- 避免冰块
- 避免生食
- 水果自己剥皮

### 当前疫情警报
暂无重大疫情爆发警报
```

---

### 3. 疫苗接种需求分析

根据目的地和旅行计划,分析疫苗接种需求。

**分析内容**:
- 必需疫苗接种(如黄热病)
- 推荐疫苗接种(如甲肝、伤寒)
- 疫苗接种时间规划
- 疫苗相互作用检查
- 接种禁忌症评估

**疫苗清单模板**:
```json
{
  "vaccine": "甲肝疫苗",
  "status": "completed|planned|not_required|contraindicated",
  "date": "2025-06-15",
  "booster_required": false,
  "notes": "已完成接种,提供长期保护"
}
```

**时间规划原则**:
- 出发前4-6周:完成必需疫苗接种
- 出发前2-4周:完成推荐疫苗接种
- 某些疫苗需要多次接种,需提前规划

---

### 4. 旅行药箱智能建议

根据目的地健康风险和个人健康状况,生成个性化旅行药箱清单。

**药箱分类**:

#### 处方药
- 个人慢性病用药(足量+额外)
- 疟疾预防用药(如需要)
- 其他医生开具的旅行用药

#### 非处方药
- 止泻药(洛哌丁胺)
- 口服补液盐
- 退烧止痛药(对乙酰氨基酚/布洛芬)
- 抗过敏药(氯雷他定)
- 晕车药
- 抗酸药

#### 防护用品
- 防蚊液(DEET 20-30%)
- 防晒霜(SPF 50+)
- 口罩(N95)

#### 急救用品
- 创可贴
- 消毒液
- 纱布和绷带
- 体温计
- 小剪刀和镊子

**个性化建议**:
- 根据个人疾病史调整用药
- 根据目的地风险增减物品
- 考虑旅行时长和活动类型

---

### 5. 用药相互作用检查

检查旅行用药与个人慢性病用药之间的潜在相互作用。

**检查内容**:
- 疟疾预防用药 vs 慢性病用药
- 旅行期间临时用药 vs 常规用药
- 疫苗 vs 药物相互作用
- 食物 vs 药物相互作用

**常见相互作用**:
- 多西环素 vs 抗酸药、钙铁补充剂
- 甲氟喹 vs 某些心脏病药物
- 某些抗生素 vs 口服避孕药

**输出**:
```markdown
## 用药相互作用检查结果

### ⚠ 发现潜在相互作用

**多西环素 ↔ 抗酸药**
- **影响**: 抗酸药降低多西环素吸收
- **建议**: 间隔2小时服用
- **严重程度**: 中等

### ✅ 无相互作用
- 氨氯地平 vs 旅行用药无已知相互作用
```

---

### 6. 多语言紧急信息卡片生成

生成包含关键医疗信息的多语言紧急卡片。

**支持语言**:
- 英语 (en)
- 中文 (zh)
- 日语 (ja)
- 韩语 (ko)
- 法语 (fr)
- 西班牙语 (es)
- 泰语 (th)
- 越南语 (vi)

**卡片内容**:
```markdown
---
紧急医疗信息 | EMERGENCY MEDICAL INFORMATION
---

姓名: 张三 | Name: Zhang San
血型: A+ | Blood Type: A+
出生日期: 1990-01-01 | DOB: 1990-01-01

⚠ 过敏史 | ALLERGIES
- 青霉素 (严重: 皮疹、呼吸困难) | Penicillin (Severe: Rash, Difficulty breathing)

当前用药 | CURRENT MEDICATIONS
- 氨氯地平 5mg 每日一次 (控制血压) | Amlodipine 5mg Once daily (Blood pressure)

疾病史 | MEDICAL CONDITIONS
- 高血压 (控制中) | Hypertension (Controlled)

紧急联系人 | EMERGENCY CONTACT
- 配偶: 李四 +86-138-1234-5678 | Spouse: Li Si +86-138-1234-5678
- 医生: 王医生 +86-10-8765-4321 | Doctor: Dr. Wang +86-10-8765-4321

---
[二维码: 扫描查看完整医疗记录]
[QR Code: Scan for complete medical records]
---
```

**二维码功能**:
- 编码关键医疗信息摘要
- 云端访问链接(模拟)
- 支持离线访问
- 可分享给医护人员

---

### 7. 旅行前后健康检查

#### 旅行前健康检查

**检查内容**:
- 个人健康状况评估
- 慢性病病情确认
- 用药充足性检查
- 疫苗接种确认
- 健康建议

**输出**:
```markdown
## 旅行前健康检查报告

### 整体评估: ✅ 适合旅行

### 健康状况
- 血压: 控制良好
- 慢性病: 稳定
- 用药: 充足

### 准备完成度
- ✅ 疫苗接种: 已完成
- ✅ 旅行药箱: 已准备
- ✅ 保险: 已购买
- ⚠ 紧急卡片: 待生成

### 建议
1. 生成多语言紧急卡片
2. 携带足量慢性病用药
3. 旅行期间注意血压监测
```

#### 旅行后健康监测

**监测内容**:
- 发热监测(持续2-4周)
- 消化系统症状
- 皮肤异常
- 其他不适症状

**潜伏期疾病提醒**:
- 疟疾: 可在返回后数月内发病
- 登革热: 通常3-14天
- 伤寒: 1-3周
- 甲肝: 2-6周

---

## 数据文件操作

### 读取数据
```bash
# 读取旅行健康数据
Read: data/travel-health-tracker.json

# 读取示例数据
Read: data-example/travel-health-tracker.json
```

### 写入数据
```bash
# 更新旅行计划
Write: data/travel-health-tracker.json

# 保存健康检查日志
Write: data/travel-health-logs/pre-trip-assessment-YYYY-MM-DD.json
```

### 数据结构验证
- 验证必需字段存在
- 验证日期格式正确
- 验证枚举值有效
- 验证数据完整性

---

## WHO/CDC数据集成

### 静态数据库(当前实现)

内置常见旅行目的地健康风险数据:
- 东南亚: 登革热、甲肝、伤寒、疟疾
- 非洲: 疟疾、黄热病、霍乱、脑膜炎
- 南美: 登革热、黄热病、寨卡病毒
- 中东: 中东呼吸综合征(MERS)

**数据更新**: 手动更新,建议每季度更新一次

### 动态查询(未来扩展)

计划集成:
- WHO疫情新闻RSS订阅
- CDC Travel Health API
- 当地卫生部门疫情通报

---

## 输出格式

### 报告格式
- Markdown格式,便于阅读
- 结构化,便于程序处理
- 包含数据源引用
- 包含时间戳

### 日志格式
```json
{
  "log_id": "log_20250728_pretrip",
  "log_type": "pre_trip_assessment",
  "trip_id": "trip_20250801_seasia",
  "generated_at": "2025-07-28T10:00:00.000Z",
  "assessment_results": {
    "health_status": "suitable_for_travel",
    "vaccination_status": "completed",
    "risk_assessment": {...},
    "recommendations": [...]
  }
}
```

---

## 安全和隐私

### 数据保护
- 护照号码加密存储
- 二维码不包含完整敏感信息
- 支持数据导出和删除

### 医学安全
- 所有建议包含免责声明
- 强调医生咨询的必要性
- 不提供具体处方
- 引用权威数据源

---

## 使用示例

### 分析旅行计划
```
输入: "计划2025年8月去东南亚旅游14天"

输出:
1. 目的地健康风险评估
2. 疫苗接种建议
3. 旅行药箱清单
4. 预防措施
5. 时间表
```

### 生成紧急卡片
```
输入: "生成英中日泰四语紧急卡片"

输出:
1. 多语言卡片文本
2. 二维码(描述)
3. 保存建议
```

### 评估健康风险
```
输入: "评估泰国的健康风险"

输出:
1. 传染病风险清单
2. 食物饮水安全建议
3. 环境风险
4. 当前疫情警报
5. WHO/CDC参考链接
```

---

**版本**: v1.0.0
**最后更新**: 2025-01-08
**维护者**: WellAlly Tech

## Limitations
- Use this skill only when the task clearly matches the scope described above.
- Do not treat the output as a substitute for environment-specific validation, testing, or expert review.
- Stop and ask for clarification if required inputs, permissions, safety boundaries, or success criteria are missing.