# CrowdListen

Both transports now use one MCP implementation. See [the consolidation and compatibility note](docs/MCP_CONSOLIDATION.md) for the removed internal dispatcher, consistent revision-conflict errors and local verification.

> Give your AI agents shared context — structured harnesses compiled from your organization's conversations, feedback, and decisions.

![CrowdListen — Give your agent evidence, not guesses](docs/images/hero.png)


## The Problem

AI agents don't share context. Every session starts from scratch — no awareness of prior decisions, no access to customer feedback, no understanding of what's been tried. You end up pasting context into every prompt and watching agents build without the one thing that matters: shared organizational understanding.

CrowdListen fixes this with a four-step loop:

1. **Ingest** — connect Slack, Discord, Reddit, support tools, internal discussions
2. **Compile** — AI distills signal into structured harnesses: themes, evidence, severity, trends
3. **Execute** — agents inherit harnesses as context, building with full organizational knowledge
4. **Learn** — agents write back observations, enriching harnesses for the next build

Any agent — Claude Code, Cursor, Gemini CLI, Codex — reads and writes harnesses. Context compounds across sessions and across agents. That's how you align builds at scale.

## Get Started

One command. Your browser opens, you sign in, and your agents are configured automatically:

```bash
npx -y -p @crowdlisten/harness crowdlisten-harness login
```

Auto-configures MCP for **Claude Code, Cursor, Gemini CLI, Codex, and Amp**. No env vars, no JSON editing, no API keys to manage. Restart your agent after login.

Verify the signed-in session with `recall({mode: "connection"})`. After choosing an entity or project, repeat the check with its ID to verify current permissions and record reads. The check starts no research. Browser access, writes and analysis still need execution checks. Login and MCP startup no longer create a separate API key; existing script keys remain unchanged.

Setup pins generated MCP entries to the package version performing setup and preserves custom commands. Running clients use refreshed session tokens and save rotation for reconnects. Logging out or signing in again invalidates the prior local connection; restart it after login. Publish the matching package before distributing its setup instructions.

Local clients now coordinate on-demand refresh and conditional credential writes across processes. Upgrade and restart all existing clients together; older writers do not use these locks. See [session behavior and recovery](docs/AUTH_SESSIONS.md), including the distinction between local logout and server token revocation.

### Manual Setup

Add to your agent's MCP config:

```json
{
  "mcpServers": {
    "crowdlisten": {
      "command": "npx",
      "args": ["-y", "-p", "@crowdlisten/harness", "crowdlisten-harness"]
    }
  }
}
```

For a local Streamable HTTP transport, run `crowdlisten-harness serve` and connect with a valid CrowdListen session token:

```json
{
  "mcpServers": {
    "crowdlisten": {
      "url": "http://localhost:3848/mcp",
      "headers": {
        "Authorization": "Bearer YOUR_SESSION_TOKEN"
      }
    }
  }
}
```

## What You Can Do

**Preserved project history:** `recall({mode: "projects"})` discovers accessible projects; add a project ID to read saved research, documents, notes, writing, canvas history and original source records. The human page uses the same scoped reader. Requires the matching database migration. [Contract and local verification](docs/PROJECT_RECORDS.md).

**Versioned shared knowledge:** connected agents can now save grounded syntheses, retrieve original evidence and historical revisions, and safely extend another agent's work. Use `recall` modes `knowledge`, `knowledge_source`, `knowledge_changes`, and `ingest` destination `insight`. These require the matching backend migration/deployment. [Record contract and examples](docs/KNOWLEDGE_RECORDS.md).

| Capability | What it does | How it works |
|---|---|---|
| **Search everything** | Semantic search across your knowledge base, entities, workspaces, Insights, Actions, and raw content | `recall` includes these discovery/search modes: `search` (semantic KB search), `entities` (list tracked companies/products), `workspaces` (list workspaces + entities), `insights` (pre-computed audience intelligence with evidence), `actions` (saved Product or Marketing Actions), `intelligence` (coverage, market volume, competitor gaps, tracked-account opinions, and owned-content performance), and `content` (raw social posts/articles). |
| **Research product questions** | Save sources and an unreviewed synthesis with explicit coverage | `analyze` searches stored content or available live public sources, captures originals and returns a saved analysis with coverage/failure receipts. Supports bounded `brief` and `deep` acquisition. Provider summaries, promotion and possible feedback remain distinct; source counts are not customer prevalence. |
| **Ingest context and outcomes** | Save notes, evidence, feedback, or an Action-linked execution result | `ingest` keeps the three-tool surface while routing curated context, evidence enrichment, observations, and explicitly approved execution lifecycle records to their canonical stores. |
| **Pre-computed insights** | Retrieve clustered, scored insights per entity with evidence chains and suggested actions | `recall({ mode: "insights", entity_name: "Acme" })` returns pain points, feature requests, praise, churn risks, competitive gaps, and trends with urgency/impact scores and source evidence. |
| **Actions** | Retrieve continuously enriched Product or Marketing Actions | `recall({ mode: "actions", entity_id: "..." })` returns Actions with evidence, workflow state, organization execution plans, and projected reach and impact without re-running source analysis. |
| **Decision-ready intelligence** | Compare self with linked competitors, inspect required-channel readiness, track specific accounts, and measure employee/advocate content | `recall({ mode: "intelligence", entity_name: "Acme", window_days: 30 })` returns an evidence-backed readout and labels sampled discovery separately from exhaustive collector coverage. |
| **Expert workflows** | 8 SKILL.md packs deliver domain expertise — no extra tools, just methodology | Audience, competitive, Reddit, Hacker News, Instagram, TikTok, Twitter, and YouTube workflows compose the same 3 tools. |

## How It Works

![CrowdListen Pipeline — Raw Crowd Signals to Agent Delivery](docs/images/pipeline.jpg)

Your agent gets **3 tools** — `recall`, `analyze`, `ingest` — always active, no pack activation needed. Nine SKILL.md workflow packs layer expert methodology on top, teaching agents how to compose the 3 tools into domain-specific workflows.

**Structured Errors** — Every tool failure returns `{ error, suggestion, docs }` so your agent knows what happened, what to do next, and where to find help.

### Tools (3)

| Tool | What it does |
|------|---|
| `recall` | Search entities, workspaces, insights with evidence, saved action objects, decision-ready intelligence, KB pages, and raw content |
| `analyze` | Queue durable research, observe saved results and inspect job status |
| `ingest` | Save curated context, submit feedback, enrich approved evidence, or create/update an approved Action execution and reviewed outcome |

### Skills (10 SKILL.md Workflow Packs)

Skills deliver expert methodology — not tools. Each skill teaches the agent a structured workflow composing `recall`, `analyze`, and `ingest`.

| Skill | What it does |
|-------|---|
| `audience-research` | Full-cycle crowd intelligence — research question to compiled truth |
| `competitive-analysis` | Competitor positioning, messaging comparison, content gaps, battlecards |
| `collect-sources` | Browser-agent collection of originals/comments with product/competitor scope and coverage |
| `hackernews-research` | Research Hacker News discussions and evidence |
| `instagram-research` | Research Instagram audience signals |
| `operate-from-customer-context` | Turn company evidence into safe, reusable execution workflows |
| `reddit-research` | Research Reddit posts and comments |
| `tiktok-analyzer` | Analyze TikTok content and engagement evidence |
| `twitter-research` | Research Twitter/X conversations |
| `youtube-research` | Research YouTube videos and comments |

Full tool reference: **[docs/TOOLS.md](docs/TOOLS.md)**

### Knowledge Base

Agent work compounds in two intentionally different layers: curated briefs live in `pages`, while approved entity evidence enters `content_store` and is enriched into reusable insights and Actions.

```
 ingest(context)  `pages`          semantic recall
 ingest(evidence) `content_store`  opinions → insights → Actions
 recall()         reads both layers without re-fetching the source
 recall(search)     semantic search     falls back to keyword
 recall(insights)   pre-computed intel  with source evidence
 recall(entities)   tracked companies   with workspace mapping
```

**How data flows:**

1. **Curate** — `ingest({ title, content, tags })` writes authored context to `pages`.
2. **Compound evidence** — `ingest({ destination: "evidence", entity_id, title, content, source_platform })` writes normalized evidence to `content_store` and returns the terminal persisted enrichment sweep.
3. **Recall** — `recall({ query })` performs semantic search with keyword fallback. Filter by project or tags.
4. **Analyze** — `analyze({ question, request_id })` queues durable analysis and observes its saved result. Keep the returned analysis/job IDs; disconnects do not cancel the job. Repeat unchanged inputs with the same request ID after a lost acknowledgment, or inspect `analyze({ action: "status", job_id })`. `action: "start"` keeps the older entity enrichment producer on that same job ledger.
5. **Observe** — `ingest({ observations: [...] })` submits feedback signals that auto-classify into themes.
6. **Execute and learn** — `ingest({ destination: "execution", ... })` records an approved Action handoff, step results, measured outcome, owner review, and explicit internal/external/training portability scopes.

After research, use `ingest(destination: "insight", ...)` to save versioned findings with exact citations and limitations. `recall` reads those shared findings and their sources for the next agent. Streamed analyses save their own research records and coverage receipts; automatic promotion into the versioned finding store is still being consolidated.

### Platforms

When using `analyze({ search_mode: "multi_source" })` or durable `analyze({ action: "start", ... })`, the backend searches across social platforms. Available platforms depend on your setup:

| Platform | Setup | Notes |
|---|---|---|
| Reddit | Provider-dependent | A current production probe returns upstream 403; treat it as unavailable until repaired |
| TikTok | `npx playwright install chromium` | Browser-based extraction |
| Instagram | `npx playwright install chromium` | Browser-based extraction |
| Twitter/X | `TWITTER_USERNAME` + `TWITTER_PASSWORD` in `.env` | Public content works without credentials |
| YouTube | `YOUTUBE_API_KEY` in `.env` | API key required |
| Hacker News | None | Works immediately |

When a platform is unavailable, `analyze` may return partial results. Preserve provider failures and do not describe an empty source set as a valid no-signal finding.

### Supported Agents

**Auto-configured on login:** Claude Code, Cursor, Gemini CLI, Codex, Amp

**OpenClaw:** explicit workspace setup through the pinned bridge; see the local verification and hosted release limits below.

Other MCP-capable clients can use manual configuration; their native-client compatibility has not been verified here.

## CLI

```bash
npx -y -p @crowdlisten/harness crowdlisten-harness login          # Sign in + auto-configure agents
npx -y -p @crowdlisten/harness crowdlisten-harness logout         # Clear saved credentials
npx -y -p @crowdlisten/harness crowdlisten-harness whoami         # Show current user
npx -y -p @crowdlisten/harness crowdlisten-harness setup          # Re-run auto-configure
npx -y -p @crowdlisten/harness@2.2.1 crowdlisten-harness setup openclaw /absolute/path/to/workspace
npx -y -p @crowdlisten/harness crowdlisten-harness serve          # Start HTTP server on :3848
npx -y -p @crowdlisten/harness crowdlisten-harness context        # Launch skill pack dashboard (port 3847)
npx -y -p @crowdlisten/harness crowdlisten-harness context file   # Process a file through context pipeline
```

## Privacy

- Browser login stores the user session in a mode-0600 local file
- CrowdListen data access is scoped to the authenticated user and database policies
- Canonical tool calls use authenticated CrowdListen API boundaries
- Provider failures remain explicit instead of becoming simulated research success
- MIT open-source and inspectable

## Development

```bash
git clone https://github.com/Crowdlisten/crowdlisten_harness.git
cd crowdlisten_harness
npm install && npm run build
npm test    # 95 current tests via Vitest
```

For agent-readable capability descriptions and example workflows, see [AGENTS.md](AGENTS.md).

## OpenClaw setup

OpenClaw uses a workspace skill and a pinned mcporter bridge. After browser login, run `crowdlisten-harness setup openclaw /absolute/path/to/workspace`, start a new OpenClaw session and follow the installed connection check. Setup preserves other tools and refuses to overwrite customized files. See [OpenClaw setup and release requirements](docs/OPENCLAW.md). A configured skill does not prove browser collection or a hosted release.

## Contributing

Highest-value contributions: new platform adapters (Threads, Bluesky, Hacker News, Product Hunt, Mastodon) and extraction fixes.

## License

MIT — [crowdlisten.com](https://crowdlisten.com)