# webspec-index

Query WHATWG, W3C, and TC39 web specifications from the command line.

## Features

- **Full-text search** across HTML, DOM, URL, CSS, ECMAScript, and 70+ other specifications
- **Cross-reference tracking** — see incoming/outgoing references between spec sections
- **Graph traversal** — build cross-reference graphs with JSON, Mermaid, or Graphviz DOT output
- **PR previews** — query spec sections as modified by an open WHATWG or TC39 proposal PR, with section-level diffs
- **Auto URL indexing for whitelisted domains** — query non-hardcoded specs by URL
- **Fast SQLite indexing** with FTS5 for instant queries
- **Algorithm and IDL extraction** with rendered markdown content
- **LSP server** for inline spec hovers and step validation in your editor
- **LLM-friendly** `--help` output — automatically detected when run inside Claude Code, Codex, Gemini CLI, or OpenCode

## Installation

```bash
cargo binstall webspec-index
```

Or build from source:

```bash
cargo install webspec-index
```

## Quick Start

```bash
# Look up a spec section (algorithm, definition, heading, IDL)
webspec-index query "HTML#navigate"
webspec-index query "https://html.spec.whatwg.org/#navigate"
webspec-index query "https://w3c.github.io/webappsec-permissions-policy/#permissions-policy-header"
webspec-index query "DOM#concept-tree" --format markdown

# Full-text search
webspec-index search "tree order" --spec DOM

# Check if an anchor exists (exit code 0 = found, 1 = not found)
webspec-index exists "HTML#navigate"

# Find anchors by glob pattern
webspec-index anchors "*-tree" --spec DOM

# List all headings in a spec
webspec-index list HTML

# Cross-references (exact or shorthand)
webspec-index refs "HTML#navigate" --direction incoming
webspec-index refs "Window.navigation" --limit 5

# Graph traversal
webspec-index graph "HTML#navigate" --max-depth 2 --graph-format mermaid
webspec-index graph "HTML#navigate" --graph-format dot
webspec-index graph "HTML#navigate" --same-spec-only
webspec-index graph "HTML#navigate" --include "*concept-*" --exclude "re:^URL#"

# Query dedicated WebIDL definitions
webspec-index idl "HTML#dom-window-navigation"
webspec-index idl "Window.navigation"
webspec-index idl "Window.open()" --spec HTML

# Update specs to latest versions
webspec-index update
```

### PR Previews

Query spec sections as they would look after a PR is merged. Two providers are supported:

- **WHATWG specs** — previews are lazily fetched from [whatpr.org](https://whatpr.org).
- **TC39 proposals** — resolved via the GitHub API; the PR's committed `index.html` build is fetched from the head repo, and the merge base from the base repo. (Reflects the *committed* `index.html`, so a PR that edits `spec.emu` without rebuilding will preview the stale build.)

Both are cached locally.

```bash
# Query a section from a PR preview (falls back to merge base for unchanged sections)
webspec-index query "HTML#navigate" --pr 12345
webspec-index query "proposal-defer-import-eval#sec-IsModuleSCCEvaluated" --pr 85 --format markdown

# Diff: see what sections the PR adds or modifies vs the merge base.
# With --diff the #anchor is optional — a bare spec name previews the whole PR.
webspec-index query "HTML#navigate" --pr 12345 --diff --format markdown
webspec-index query proposal-defer-import-eval --pr 85 --diff --format markdown

# Force re-fetch (e.g. after the PR is updated)
webspec-index query "HTML#navigate" --pr 12345 --force-update

# Other commands also support --pr
webspec-index exists "HTML#navigate" --pr 12345
webspec-index list HTML --pr 12345
webspec-index refs "HTML#navigate" --pr 12345
webspec-index search "OpaqueRange" --spec HTML --pr 12345
webspec-index anchors "*opaquerange*" --spec HTML --pr 12345

# Manage cached PR data (each PR caches rendered pages + merge base)
webspec-index clear-pr                          # list cached PRs
webspec-index clear-pr --spec HTML --pr 12345   # remove one
webspec-index clear-pr --all                    # remove all
```

All commands support `--format json` (default) or `--format markdown`.

Spec data is fetched and cached locally on first query — no setup needed.

Spec refreshes are freshness-based: once checked, a spec is considered fresh for 24h.
When refreshed, the CLI fetches live HTML and re-indexes only if content changed.

## AI Agent Integration

### Skill files

Drop [skills/webspec-index/](skills/webspec-index/SKILL.md) into your repo to teach the agent how to use the CLI.

[skills/spec-trace/](skills/spec-trace/SKILL.md) builds on it: given a question about what a spec
specifies, it traces the algorithm call chain with `paths` and produces a reviewable verdict.

## Editor Integration

The **webspec-lens** extension provides inline spec hovers, step validation, and coverage tracking. Available for VS Code and any LSP-compatible editor.

See [editors/vscode/](editors/vscode/) and [editors/zed/](editors/zed/) for details.

## How It Works

1. **Fetches** spec HTML from WHATWG/W3C/TC39 spec URLs
2. **Parses** sections, algorithms, IDL definitions, and cross-references
3. **Indexes** in SQLite with FTS5 for fast full-text search
4. **Refreshes snapshots** on a 24h cadence with content-hash change detection
5. **PR previews** fetch the PR build and its merge base (WHATWG: rendered pages from whatpr.org + commit-snapshots; TC39 proposals: committed `index.html` from the head/base repos via the GitHub API), storing both as separate snapshots for querying and diffing

## Development

```bash
cargo test
cargo clippy        # lint
cargo fmt --check   # format check
```

## License

MIT

## Links

- [GitHub Repository](https://github.com/jnjaeschke/webspec-index)
- [Issue Tracker](https://github.com/jnjaeschke/webspec-index/issues)
- [VS Code Extension](https://marketplace.visualstudio.com/items?itemName=jnjaeschke.webspec-lens)