# Moderation

> Moderate user content before it commits — keyword denylist or AI provider, block or flag via rpc interceptor, plus an in-handler redact helper.



---

<!-- source: en/plugins/moderation.md -->
## Moderation

_Moderate user content before it commits — keyword denylist or AI provider, block or flag via rpc interceptor, plus an in-handler redact helper._

`@voltro/plugin-moderation` screens user-submitted content. A rule matches an rpc tag + fields; the configured provider classifies the text; the interceptor **blocks** (typed `ContentRejected`, the write never commits) or **flags** (annotates, lets it through). Because the interceptor can't rewrite input, **redact** is an in-handler helper.

## Wiring

```ts
// app.config.ts
import { moderationPlugin, keywordProvider, aiProvider } from '@voltro/plugin-moderation'

export default {
  type: 'api' as const,
  name: 'api',
  plugins: [
    moderationPlugin({
      provider: keywordProvider(['spam', 'scam', 'http://badsite']),  // or aiProvider() — uses @voltro/ai
      rules: [
        { match: 'posts.create', fields: ['title', 'body'], action: 'block' },
        { match: /^comments\./, fields: ['text'], action: 'flag' },
      ],
    }),
  ],
}
```

## Providers

- **`keywordProvider(terms)`** — pure denylist; zero deps, deterministic, instant. Best for a known set of banned terms / URLs.
- **`aiProvider(opts?)`** — lazy `@voltro/ai` classification (`@voltro/ai` is an optional dep). **Fails open**: a provider error resolves to `CLEAN` so an AI outage can't block all writes. Use for nuanced toxicity/abuse detection.
- Custom — any function with the provider signature `(text) => Effect.Effect<ModerationVerdict>` plugs into the same `provider` slot. The verdict is `{ flagged, categories?, score?, reason? }`.

## Block vs flag vs redact

- **`block`** — interceptor fails the call with typed `ContentRejected` (`{ tag, categories, reason }`, merged into the wire-error union). The write never commits.
- **`flag`** — lets the write through; surfaces the classification for downstream review (e.g. a moderation queue).
- **Redact** — the interceptor can't mutate input, so there is no `redact` action. Instead, call the in-handler `moderate(text)` helper: it **classifies** the text against the configured provider (returning a `ModerationVerdict` — it does NOT return redacted text) and the handler decides what to store:

```ts
import { moderate } from '@voltro/plugin-moderation'

export default (input: { body: string }, ctx) => Effect.gen(function* () {
  const verdict = yield* moderate(input.body)   // classifies via the configured provider
  const body = verdict.flagged ? '[redacted]' : input.body
  return yield* Effect.tryPromise(() => ctx.store.insert('posts', { body }))
})
```

## Dashboard panel

Both dashboards ship a **Moderation** panel (api apps): block/flag counts + the **review queue** of flagged content (tag, categories, reason, subject), each pending item with **Confirm** (real violation) / **Dismiss** (false positive) actions. Resolving gates on the `canModerateContent` capability. Backed by `/_voltro/inspect/plugins/moderation/{flagged,resolve}`.

## Permissions

`rpc:intercept:mutation` + `rpc:intercept:action` (moderation gates both writes and external-I/O actions) + `inspect:read` (dashboard panel).