# emet 1.4.1 — Internal collector registry release

## Release summary

emet 1.4.1 adds a read-only, internal collector layer for social/platform data sources — Hacker News, V2EX, GitHub, RSS/Atom feeds, and YouTube. Collectors are not exposed as new MCP tools; they are internal capabilities usable by the existing `emet` tool and visible via `emet doctor`.

This release implements the architecture prepared in 1.4.0: public agent tools stay `emet` + `web_fetch`, while platform-specific data lives behind an internal registry.

## What changed

### 1. Internal no-auth collector registry

New `lib/collectors/` module with a lazy Map-based registry:

- **Base contract** (`collector.js`): `SocialCollector` base class, `fetchWithTimeout` primitive, shared output schema with `emptyResult()` and `normalizeResult()` factories.
- **Registry** (`index.js`): Lazy instantiation, `getCollector(name)`, `listCollectors()`, `runCollectorDoctor()`.
- **Hacker News** (`hn.js`): Algolia API search — no auth needed, 10k req/h/IP.
- **V2EX** (`v2ex.js`): Public v1 `/api/topics/hot.json` — no auth, 120 req/h/IP, client-side filter for search.
- **GitHub** (`github-collector.js`): Public REST search — no auth (60 req/h), repos/code/issues types.
- **RSS/Atom** (`rss.js`): Zero-dependency regex parser for RSS 2.0 `<item>` and Atom `<entry>` elements.
- **YouTube** (`youtube.js`): `yt-dlp`-based metadata extraction — degrades gracefully when yt-dlp is missing.

### 2. `emet doctor` shows collector status

`emet doctor` now appends collector availability checks:

```
ok   collector:hn:       Hacker News (Algolia)
ok   collector:v2ex:     V2EX (public API)
ok   collector:github:   GitHub (public REST)
ok   collector:rss:      RSS/Atom feeds
warn collector:youtube:  YouTube (via yt-dlp): yt-dlp not found
  fix: Install yt-dlp: brew install yt-dlp or pip install yt-dlp
```

Missing optional runtimes (yt-dlp) produce warnings, not failures.

### 3. No new dependencies, no new MCP tools

- Zero new entries in `package.json` dependencies — all collectors use built-in `fetch` and `node:child_process`.
- No new MCP tools registered: `tools/list` returns only `emet` and `web_fetch`.
- Collectors are internal modules; agent-facing integration belongs in follow-up issues (#19).

## Tool surface policy (unchanged)

Public agent tools stay minimal:

1. `emet` — research, citations, policy, synthesis, future collector-backed orchestration
2. `web_fetch` — raw text for one known URL, no synthesis

## Compatibility notes

- All existing `emet` and `web_fetch` calls work unchanged.
- `emet doctor` gains new collector checks, but `runDoctor()` return value stays backwards-compatible.
- Plan file at `docs/plans/2026-06-22-issue-18-collector-registry-plan.md` documents design decisions.
- No behavioral changes to any existing exports.

## Verification before release

- `npm test` — 228/228 passing (18 new collector tests + 210 existing)
- All existing tests pass unchanged (cli.test.js, web-research.test.js, etc.)
- `node bin/emet.js doctor` — reports all 5 collectors with availability

## Suggested GitHub release text

```markdown
## emet 1.4.1 — Internal collector registry release

This release adds a read-only, internal collector layer for platform data sources, 
implementing the architecture prepared in 1.4.0. No new MCP tools, no new dependencies.

### Added
- Internal no-auth collector registry: HN (Algolia), V2EX (public API), GitHub (REST search),
  RSS/Atom feeds (zero-dependency parser), YouTube (yt-dlp, optional).
- `emet doctor` now reports collector availability (ok/warn per platform).

### Changed
- `runDoctor()` includes collector availability checks.
- Package version and all plugin/manifest metadata bumped to 1.4.1.

### Compatibility
- All existing `emet` and `web_fetch` calls work unchanged.
- No new MCP tools — collectors are internal modules.
- No new dependencies.

### Verification
- `npm test`: 228/228 passing
```