# Plugins > How Voltro plugins compose into the runtime, what they can intercept, the catalogue, and writing your own. --- ## Overview _How Voltro plugins compose into the runtime, what they can intercept, the catalogue, and writing your own._ A Voltro plugin is a server-side extension that hooks into the runtime. Plugins can intercept mutations / queries / actions, contribute schema mixins + tables + migrations, mount raw-HTTP and inspect routes, contribute an Effect service layer, and run install / activate / deactivate lifecycles. The framework ships some plugins; you write your own; the contract is small enough to learn in a single read. ## What's in this section - [The plugin contract](/docs/plugins/contract) — `definePlugin`, lifecycle hooks, rpc interceptors, framework-version compatibility - [plugin-audit](/docs/plugins/audit) — mutation audit log + `audit()` mixin - [plugin-auth](/docs/plugins/auth) — full auth suite: password (rehash-on-verify) + sessions (multi-key rotation + sliding-window) + magic-link/reset + email verification + tenant invitations + user impersonation + passkeys (atomic clone detection, BYO multi-replica challenge store) + CSRF + session revocation + memberships/switch-tenant + TOTP/MFA (sign-in enforcement + recovery codes), mounted by `authRoutesPlugin()` (see also the [Authentication section](/docs/authentication/overview)) - [plugin-multitenancy](/docs/plugins/multitenancy) — `tenant()` schema mixin + `assertOwnTenant` write-guard + `TenantMismatch` - [plugin-soft-delete](/docs/plugins/soft-delete) — `softDelete()` schema mixin (hide on delete, `hardDelete()` bypass) - [plugin-rbac](/docs/plugins/rbac) — roles + permissions + the `permission()` guard - [plugin-ratelimit](/docs/plugins/ratelimit) — per-endpoint / per-subject / per-tenant request limits - [plugin-billing](/docs/plugins/billing) — subscriptions, plans, entitlements + usage metering (Stripe + mock provider); seat-based billing on Stripe's own proration, retries, tax and checkout; money as integer minor units - [plugin-licensing](/docs/plugins/licensing) — offline-verified EdDSA license keys + cloud-issued entitlement snapshots that feed plugin-billing; plan entitlements + pricing decided server-side, never baked into a published version - [plugin-mail](/docs/plugins/mail) — transactional email (Resend / Postmark / SendGrid / SES / Mailgun / SMTP, templates, suppression, scheduling, batch, idempotency) - [plugin-storage](/docs/plugins/storage) — file storage: public (CDN-direct) + private (access policy + per-object grants), S3 / MinIO (R2 and GCS via their S3 interop) / Azure / database / filesystem - [plugin-ai-flows](/docs/plugins/ai-flows) — durable multi-step AI pipelines (deterministic + agentic) with human-in-the-loop, chaining, and cadence; code-first `defineFlow` or data-driven rows - [plugin-postgis](/docs/plugins/postgis) — postgres-native `geography` / `geometry` columns + spatial operators - [plugin-broadcast](/docs/plugins/broadcast) — cross-replica reactivity over a pub/sub bus (Redis / NATS) for non-postgres dialects - [plugin-webhooks](/docs/plugins/webhooks) — durable incoming + outgoing webhooks (HMAC signing, retries, idempotency) - [plugin-atlassian](/docs/plugins/atlassian) — `JiraService` + `ConfluenceService` over the Atlassian APIs - [plugin-deactivation](/docs/plugins/deactivation) — `deactivation()` schema mixin (visible, can't log in) - [plugin-prometheus](/docs/plugins/prometheus) — Prometheus exporter at `GET /metrics`; scrapes the unified Metrics-API (the same source the dashboard Metrics panel reads) - [plugin-datadog](/docs/plugins/datadog) — deep Datadog integration; agentless metrics + opt-in logs + traces (OTLP→Agent) + profiler, trace-correlated - [plugin-sentry](/docs/plugins/sentry) — deep Sentry integration; trace-correlated errors + breadcrumbs from the log sink + opt-in performance traces - [plugin-flags](/docs/plugins/flags) — feature flags: per-subject / per-tenant targeting, deterministic % rollouts, kill-switch, declarative rpc gating + client UI gating - [plugin-notifications](/docs/plugins/notifications) — unified notifications: one send API across email / Slack / SMS / push (first-class APNs/FCM factory) / in-app, channel preferences + in-app inbox, digest/batching, quiet hours, broadcast/topics, durable DataStore-backed store by default - [plugin-logship](/docs/plugins/logship) — ship structured logs to Better Stack / Axiom / Loki / any HTTP sink; batched, redacted, fail-soft - [plugin-moderation](/docs/plugins/moderation) — moderate user content before commit: keyword or AI provider, block / flag via interceptor + in-handler redact - [plugin-search](/docs/plugins/search) — keep an external index (Typesense / Meilisearch / Algolia) in sync via the ChangeEvent tap; tenant-scoped `search.query` + hook - [plugin-cdc-out](/docs/plugins/cdc-out) — declarative reverse-ETL: mirror table changes outward to a webhook sink (or any custom `CdcSink`) through a durable outbox; ordered per pipe, at-least-once from enqueue, dead-lettered - [plugin-governance](/docs/plugins/governance) — data governance: retention TTL sweep, GDPR export + erasure, consent ledger, field encryption - [plugin-openapi](/docs/plugins/openapi) — OpenAPI 3.1 spec + Swagger-UI docs generated from your `defineRestRoute` descriptors and (opt-in) rpc procedures - [plugin-row-history](/docs/plugins/row-history) — full row history + time-travel (`rowHistory` / `rowAsOf` / `restoreAsOf` / `diffVersions`); what-changed-to-what on every write - [plugin-presence](/docs/plugins/presence) — ephemeral realtime presence: heartbeat roster per channel + `usePresence` / `useTyping` hooks, held in memory; cross-instance with [plugin-broadcast](/docs/plugins/broadcast) - [plugin-auth-social](/docs/plugins/auth-social) — first-party Sign in with Google / GitHub / Apple: mandatory PKCE + state, JWKS-verified ID tokens, a deliberate account-linking policy, sessions issued through plugin-auth - [plugin-scim](/docs/plugins/scim) — SCIM 2.0 provisioning (Users + Groups at `/scim/v2`) so an enterprise IdP can create/deactivate users - [plugin-sso-saml](/docs/plugins/sso-saml) — enterprise SAML 2.0 SSO: SP-initiated login + Single Logout, ACS, metadata (+ IdP-metadata-URL auto cert rotation, encrypted assertions, SP request signing); mints a framework session - [API keys](/docs/configuration/api-keys) — **first-class** (not a plugin): `apiKeys: true` enables Bearer-key auth + admin-gated issue/list/revoke - [Analytics & warehouse sinks](/docs/plugins/analytics) — `AnalyticsSink` contract + five first-party sink plugins (postgres-lite, DuckDB, ClickHouse, Tinybird, PostHog) - [External identity providers](/docs/authentication/external-idp) — the six auth-adapter packages (WorkOS, Kinde, Clerk, Auth0, Supabase, generic OIDC) ## The catalogue at a glance Status legend: ✓ shipped · ◐ partial · — planned. | Plugin | Status | What it does | |---|---|---| | `@voltro/plugin-audit` | ✓ | Mutation audit log + `audit()` mixin | | `@voltro/plugin-auth` | ✓ | Full auth suite via `authRoutesPlugin()`: password (rehash-on-verify), sessions (multi-key rotation + sliding-window), magic-link + password-reset, email verification (off/soft/strict policy), tenant invitations (addressed, single-use, role chosen by the inviter), user impersonation (marked, time-bounded, escalation-proof), passkeys/WebAuthn (atomic clone detection, BYO multi-replica challenge store), CSRF, session enumeration + revocation, memberships + switch-tenant, TOTP/MFA (sign-in enforcement + recovery codes); `authTables` schemas | | `@voltro/plugin-multitenancy` | ✓ | `tenant()` schema mixin (read-scope + write-fill) + `assertOwnTenant` guard + typed `TenantMismatch` | | `@voltro/plugin-soft-delete` | ✓ | `softDelete()` schema mixin — `deletedAt` / `deletedBy`; `delete` → UPDATE, `hardDelete()` bypass | | `@voltro/plugin-rbac` | ✓ | Roles compile to scopes + the `permission()` handler guard + typed `ScopeError` | | `@voltro/plugin-ratelimit` | ✓ | Per-endpoint / per-subject / per-tenant limits; sliding-window / fixed-window / token-bucket; memory / postgres / redis stores | | `@voltro/plugin-billing` | ✓ | Subscriptions, plans, entitlements + usage metering over a pluggable provider (Stripe + mock); seat-based billing; proration, failed-payment retries, tax and the checkout seat stepper are Stripe's, via the official SDK; `requireEntitlement()` guard + `enforce` interceptor; `/billing/webhook` via plugin-webhooks; money as integer minor units | | `@voltro/plugin-licensing` | ✓ | Offline-verified EdDSA license keys + cloud-issued entitlement snapshots that feed plugin-billing; plan entitlements + pricing decided server-side, never baked into a published version. [→ details](/docs/plugins/licensing) | | `@voltro/plugin-ai-flows` | ✓ | Durable multi-step AI pipelines — deterministic or agentic, with human-in-the-loop, chaining and cadence; author flows in code (`defineFlow`) or as data (visual-editor rows), one engine runs both. [→ details](/docs/plugins/ai-flows) | | `@voltro/plugin-mail` | ✓ | Transactional email — Resend / Postmark / SendGrid / SES / Mailgun / SMTP, *.email.tsx templates, per-tenant suppression, send-time scheduling, bulk/batch send, per-send idempotency, durable via workflows | | `@voltro/plugin-storage` | ✓ | File storage — public (CDN-direct) + private (access policy + per-object grants), S3 / MinIO (R2 and GCS via their S3 interop) / Azure / database / filesystem providers, presigned URLs, `listRefs` browse/search, HTTP Range (206) serving, dashboard browser | | `@voltro/plugin-postgis` | ✓ | Postgres-native `geography` / `geometry` columns + spatial predicates (`ST_DWithin`, `ST_Contains`, `ST_Intersects`); GiST indexes via `.expressionIndex(..., { kind: 'gist' })`. No `ST_Distance` projection yet. Postgres-only by design (fails loud elsewhere). [→ details](/docs/plugins/postgis) | | `@voltro/plugin-broadcast` | ✓ | Cross-replica reactivity — fans out app-mutation change events to every replica over a pub/sub bus (Redis / NATS). Closes the single-instance gap for every non-postgres dialect. [→ details](/docs/plugins/broadcast) | | `@voltro/plugin-webhooks` | ✓ | Incoming + outgoing webhooks — `defineIncomingWebhook` (signature verify + idempotency, Stripe/GitHub/Slack presets) and `defineEvent` (durable delivery workflow, HMAC signing, retries, filters). [→ details](/docs/plugins/webhooks) | | `@voltro/plugin-auth-social` | ✓ | First-party social login — Sign in with Google / GitHub / Apple with no identity vendor: authorize URL + code exchange + JWKS-verified ID tokens, mandatory PKCE (S256) and `state`, an explicit account-linking policy (`never` by default), Apple's signed-JWT client secret / one-time name / private-relay email all handled; sessions via `issueUserSession`. [→ details](/docs/plugins/auth-social) | | `@voltro/plugin-auth-{workos,kinde,clerk,auth0,supabase,oidc}` | ✓ | Six IdP adapters over the shared `jwtBearerStrategy` — JWKS verify + claims→tenant mapping; WorkOS additionally ships hosted-login OAuth primitives (`workosAuthorizationUrl` / `workosAuthenticateWithCode`) for a redirect-based SSO login flow. [→ details](/docs/authentication/external-idp) | | `@voltro/plugin-analytics-postgres` | ✓ | First-party lite — events on the main DataStore, cross-dialect (postgres / mysql / mariadb / mssql / sqlite / turso). [→ details](/docs/plugins/analytics#voltroplugin-analytics-postgres) | | `@voltro/plugin-duckdb` | ✓ | Embedded DuckDB sidecar — real OLAP performance, no external service. [→ details](/docs/plugins/analytics#voltroplugin-duckdb) | | `@voltro/plugin-clickhouse` | ✓ | Production OLAP via the official ClickHouse client. [→ details](/docs/plugins/analytics#voltroplugin-clickhouse) | | `@voltro/plugin-tinybird` | ✓ | Hosted ClickHouse via Events API + Pipes. [→ details](/docs/plugins/analytics#voltroplugin-tinybird) | | `@voltro/plugin-posthog` | ✓ | Product analytics — track-only; compose with another sink for reads. [→ details](/docs/plugins/analytics#voltroplugin-posthog) | | `@voltro/plugin-atlassian` | ✓ | `JiraService` + `ConfluenceService` over the Atlassian REST / Greenhopper / Agile APIs — PAT **or** OAuth 2.0 (3LO) auth, transient retry, SSRF guard, comment-write, signature-verified inbound webhooks, avatar proxy, per-tenant cache. [→ details](/docs/plugins/atlassian) | | `@voltro/plugin-deactivation` | ✓ | `deactivation()` schema mixin — `deactivatedAt` + `deactivatedBy` (→ Actor); subject can't log in but data stays visible. [→ details](/docs/plugins/deactivation) | | `@voltro/plugin-prometheus` | ✓ | Prometheus exporter — `GET /metrics` in text exposition format over the unified Metrics-API (Effect `MetricRegistry`); counters / histograms / gauges + custom metrics, optional bearer gate + node process metrics. [→ details](/docs/plugins/prometheus) | | `@voltro/plugin-datadog` | ✓ | Deep Datadog integration — agentless metrics push to `/api/v2/series` + opt-in logs (`/api/v2/logs`, `dd.trace_id`-correlated) + traces (OTLP→Agent) + dd-trace profiler; `DD_API_KEY`/`DD_SITE` + unified service tagging, fail-soft. [→ details](/docs/plugins/datadog) | | `@voltro/plugin-sentry` | ✓ | Deep Sentry integration — mutation/query/action errors reported correlated to the active trace (trace_id + span_id) + breadcrumbs from the framework log sink; opt-in performance traces (`SentrySpanProcessor`, OTel-consumer mode) + profiler. `@sentry/*` optional + lazy. [→ details](/docs/plugins/sentry) | | `@voltro/plugin-flags` | ✓ | Feature flags — per-subject / per-tenant targeting + deterministic % rollout (FNV-1a bucket) + kill-switch; multivariate variant flags + scheduled / ramping rollouts + a durable kill-switch audit trail; declarative `gatedBy` (typed `FlagDisabled`) + `requireFlag` guard + `flags.evaluate` / `flags.variants` routes + `useFlags`/`useFlag`/`useVariant` hooks; memory / postgres store. [→ details](/docs/plugins/flags) | | `@voltro/plugin-notifications` | ✓ | Unified notifications — one `send` across email / Slack / SMS / push (first-class `pushChannel` APNs/FCM factory) / in-app channels, per-user channel preferences, in-app inbox + unread count + delivery records; digest/batching rollup, per-subject quiet hours (DND), broadcast/topic fan-out; `NotificationService` + `useInbox`/`useUnreadCount`/`useMarkRead`/`useTopicSubscription`/`useQuietHours` hooks; durable DataStore-backed store by default (auto-migrated `notification_*` tables). [→ details](/docs/plugins/notifications) | | `@voltro/plugin-logship` | ✓ | Ship structured logs to Better Stack / Axiom / Loki / any HTTP sink — rides the log-sink hook, batched + redacted + fail-soft, trace-correlated. [→ details](/docs/plugins/logship) | | `@voltro/plugin-moderation` | ✓ | Content moderation — keyword denylist or AI provider (fails open), block (typed `ContentRejected`) / flag via rpc interceptor + in-handler `moderate()` redact helper. [→ details](/docs/plugins/moderation) | | `@voltro/plugin-search` | ✓ | External search index sync — rides the ChangeEvent tap to mirror tables into Typesense / Meilisearch / Algolia (memory default), tenant-scoped `search.query` action (facets · highlighting · fuzziness · range/negation filters · engine-param passthrough) + `useSearch` hook + `backfillIndex` + durable cross-replica sync stats. [→ details](/docs/plugins/search) | | `@voltro/plugin-cdc-out` | ◐ | Declarative reverse-ETL — mirror table changes outward to external sinks (webhook, plus a `CdcSink` interface for custom sinks) through a durable outbox; ordered per pipe, at-least-once from enqueue, retried with backoff, dead-lettered. Engine + memory/webhook sinks shipped; anything else implements the `CdcSink` interface. | | `@voltro/plugin-governance` | ✓ | Data governance — retention TTL sweep (delete / anonymise), GDPR subject export + erasure (admin-gated routes + `GovernanceService`), consent ledger, field encryption. [→ details](/docs/plugins/governance) | | `@voltro/plugin-openapi` | ✓ | OpenAPI 3.1 spec (`GET /openapi.json`) + Swagger-UI (`GET /docs`) generated from `defineRestRoute` descriptors AND (opt-in) rpc procedures (queries/mutations/actions/streams → `POST /rpc/`) — input/output/error Schemas via `JSONSchema.make`. [→ details](/docs/plugins/openapi) | | `@voltro/plugin-row-history` | ✓ | Full row history + time-travel — value snapshot of every insert/update/delete (every table by default; narrow with include/exclude) into `_voltro_row_history` (rides the ChangeEvent tap); `rowHistory` / `rowAsOf` queries + `restoreAsOf` / `diffVersions`; TTL + per-row cap retention. [→ details](/docs/plugins/row-history) | | `@voltro/plugin-presence` | ✓ | Ephemeral realtime presence — heartbeat roster per channel (`presence.heartbeat`/`list`/`leave` + `usePresence`), a `useTyping` typing indicator. Held **in memory**, owner-partitioned — no table is written; cross-instance requires [`@voltro/plugin-broadcast`](/docs/plugins/broadcast), and without a broker each replica sees only its own clients. [→ details](/docs/plugins/presence) | | `@voltro/plugin-scim` | ✓ | SCIM 2.0 provisioning — Users + Groups REST at `/scim/v2` (bearer-gated) incl. group-membership PATCH/PUT + the RFC 7644 discovery trio (ServiceProviderConfig/Schemas/ResourceTypes), `userName`/`externalId`/`displayName eq` filters, pagination, unique `userName`, `active:false` deactivation; `_voltro_scim_users`/`_voltro_scim_groups`. [→ details](/docs/plugins/scim) | | `@voltro/plugin-sso-saml` | ✓ | Enterprise SAML 2.0 SSO — SP-initiated login + Single Logout (both directions) + ACS + SP metadata under `/saml`; IdP-metadata-URL auto cert rotation, encrypted assertions, clock-skew, SP request signing. Signature verify via `@node-saml/node-saml` (optional+lazy), mints a framework session. [→ details](/docs/plugins/sso-saml) | API keys are **first-class** (not a plugin): `apiKeys: true` in `app.config.ts` → Bearer-key auth + admin-gated `/v1/api-keys` management, hash-only storage. [→ details](/docs/configuration/api-keys) Every plugin carries a design doc in the framework's `plans/` directory before it ships. ## Configuration shape ```ts // app.config.ts import { auditPlugin } from '@voltro/plugin-audit' import { rateLimitPlugin } from '@voltro/plugin-ratelimit' export default { type: 'api' as const, name: 'api', plugins: [ rateLimitPlugin({ default: { limit: 60, window: '1m' } }), auditPlugin({ sink: 'console' }), ], } ``` Order matters: the framework composes outer→inner, so the rate-limit interceptor runs *before* the audit interceptor sees the request. Rejected requests never enter the audit log. ## What plugins can do | Surface | What it lets you do | |---|---| | `interceptMutation` / `interceptQuery` / `interceptAction` | Wrap every mutation / query-setup / action — gate, audit, transform input/output. | | `extendSchema` | Contribute tables + custom SQL migrations (tracked in `_voltro_plugin_migrations`). | | `services` | Provide an Effect `Layer` whose Tags every handler can `yield*` (e.g. `MailService`, `StorageService`). | | `routes` | Register plugin-owned rpc queries / mutations / actions (alias-prefixed tags). | | `httpRoutes` | Serve public raw-HTTP endpoints on the framework listener (e.g. `GET /_voltro/storage/:id`). The request carries `store` — the app's DataStore — for a route that must read or write (a login endpoint minting a session row cannot be an rpc mutation), and `remoteAddr`, the client address already resolved through `security.trustedProxies` (use it instead of `x-forwarded-for`). Not tenant-scoped: raw HTTP has no resolved Subject, so scope it yourself. A state-changing route is [origin-checked](/docs/security/overview#cross-site-requests-are-refused) unless it declares `originGuard: 'exempt'`. | | `inspectEndpoints` | Mount tooling under `/_voltro/inspect/plugins//…`. | | `onScheduleFire` / `onWorkflowStep` / `onHttpRequest` | Wrap every cron firing, every workflow `step()`, every pre-auth HTTP request. | | `onInstall` / `onActivate` / `onDeactivate` / `onUninstall` | Lifecycle hooks at first-install, boot, shutdown, and removal. | | Schema mixins (`defineMixin`) | The OTHER plugin shape — `audit()`, `tenant()`, `softDelete()` — declared in `@voltro/database`, not via the runtime contract. | ## Pointing YOUR table at a plugin's row Every table-carrying plugin exports its table handles, so a column in your schema can reference one exactly like it references your own: ```ts import { aiFlowsTable } from '@voltro/plugin-ai-flows' import { id, reference, table, text } from '@voltro/database' export const flowFavourites = table('flow_favourites', { id: id({ prefix: 'fav' }), employeeId: reference(() => employees, { onDelete: 'cascade' }), // A real foreign key across the plugin boundary. Deleting the flow removes // the favourite; the DATABASE enforces it, so no cleanup subscriber exists to // forget. flowId: reference(() => aiFlowsTable, { onDelete: 'cascade' }), note: text().nullable(), }) ``` This is not a special primitive — it is `reference()`, with the same `onDelete` semantics and the same index defaults. `plugin-storage`'s `assetRef()` has always been exactly this under the hood: a `reference(() => _voltroStorageRefsTable, { onDelete: 'setNull' })`. **Referencing the table as a VALUE rather than its name as a string is what makes this safe across a plugin's own migrations.** When ten plugin tables moved into the `_voltro_` namespace in 0.22.0, a `reference(() => table)` followed the rename (catalog-only, the constraint travels with the table); a hand-written `text()` column holding ids would not have told you anything had changed. **`fk: false`-style decoupling is still available** — declare a plain `text()` column instead. Choose it when you deliberately want the app schema independent of the plugin's, and accept that nothing then enforces the link. What you should NOT do is reach for it by default. **When you DO want the decoupling, `pluginRef` gives you the rule without the key.** It is a plain typed id column — no constraint, no cross-schema dependency — plus a declared orphan policy the framework runs on the post-commit change channel: ```ts import { pluginRef } from '@voltro/database' flowId: pluginRef(aiFlowsTable, { orphanPolicy: 'delete' }) sharedFlow: pluginRef(aiFlowsTable, { orphanPolicy: 'null' }).nullable() ``` That closes the gap named above: an unenforced id column plus a hand-written `defineSubscriber` that cleans up on delete is referential integrity re-implemented per app, and it is silently wrong the first time somebody forgets it. The declaration is the same one line either way — the difference is that the framework performs it. **Prefer `reference()` when you want a real key.** `pluginRef` is for the case where you have deliberately chosen not to have one; it does not make the database enforce anything. The tenant boundary fails closed, soft deletes are opt-in (`onSoftDelete`), and a `pluginRef` at a table no installed plugin registers refuses at boot rather than sitting there looking enforced. **`orphanPolicy` is not part of this.** It is migration metadata — how existing orphan rows are cleaned up *before* the FK constraint is added — and has no runtime semantics. Runtime behaviour comes from `onDelete`. ## Composing with a plugin's namespace Sharing a namespace with a plugin already works: the collision check compares FULL tags, so `notifications.list` of yours beside the plugin's `notifications.inbox` is not a clash. Only an identical name is — two handlers behind one tag is not something a caller can reason about. To REPLACE one deliberately, declare it: ```ts // Adopt the plugin's namespace, add your own leaves beside it… defineQuery({ name: 'notifications.archive', guards: [{ scope: 'notifications:read' }], … }) // …and REPLACE just the one you need to behave differently. Your replacement is // YOUR procedure, so it needs its own access decision — the plugin's does not // carry over with the tag. defineMutation({ name: 'notifications.markRead', overridesPlugin: true, guards: [{ scope: 'notifications:write' }], … }) ``` The plugin's route is dropped, not merely permitted alongside yours — permitting both would leave two handlers bound, which is the state the check exists to prevent. The boot logs which routes were replaced. **Explicit, never inferred.** Letting your route win silently would mean a plugin upgrade that adds a route could shadow one of yours with no diff to read. It is also why the two obvious alternatives are worse: renaming your procedure, or `alias`ing the whole plugin away, both move the split from a domain boundary to "who built it" — for whoever calls the api, the worst possible partition. ## When NOT to write a plugin - **One-off side effect** — just call it from the mutation directly. - **App-specific behaviour** — keep it in app code, not a reusable plugin. - **Anything cross-cutting that only affects ONE mutation** — a single `await ctx.audit.log(...)` call beats a plugin's hook. Plugins are for cross-cutting concerns. Audit-log every write, rate-limit every mutation, send a `user.created` event from every sign-up: that's plugin territory. ## Where to read next - [The plugin contract](/docs/plugins/contract) — write your own - [plugin-audit](/docs/plugins/audit) — most complete reference implementation --- ## Plugin contract _definePlugin, lifecycle hooks, rpc interceptors, framework-version compatibility — the surface every Voltro plugin implements._ A **plugin** is the unit of cross-cutting framework extension. Examples in the wild: - `@voltro/plugin-audit` records every mutation invocation to a sink. - `@voltro/plugin-multitenancy` adds the `tenant()` schema mixin + write-guard. - `@voltro/plugin-webhooks` ships incoming + outgoing webhook tables + workflow. - `@voltro/plugin-auth-workos` (and 5 siblings) plug an external IdP into the auth chain. Plugins live as npm packages, get listed in `app.config.ts`'s `plugins:` array, and the framework wires their hooks at boot. The contract is intentionally narrow — a plugin is NOT a full app extension; it's a focused cross-cutting concern that pairs with the existing query / mutation / action / workflow primitives. ## The shape ```ts import { definePlugin } from '@voltro/protocol' export const myPlugin = (options: MyOptions) => definePlugin({ name: '@vendor/my-plugin', // stable id — surfaced in boot logs + dashboard description: 'rate-limits outbound HTTP per tenant', framework: '^1.0.0', // semver range — soft-checked at boot interceptMutation: async (next, ctx) => { /* wraps every mutation */ }, interceptAction: async (next, ctx) => { /* wraps every action */ }, onActivate: (lifecycle) => { /* one-shot boot: open pools, register metrics */ }, onDeactivate: (lifecycle) => { /* graceful shutdown */ }, }) ``` `definePlugin` is an identity function with type-level enforcement — it returns the input unchanged at runtime, but the type-checker catches missing required fields, misnamed hooks, and excess properties at the declaration site. Always use it over plain object literals. ## Hooks ### `interceptMutation` / `interceptAction` (Effect-native) Wrap every mutation or action call respectively. **Both are Effect-based** — the framework is Effect end-to-end and the plugin boundary preserves that shape so tracing, interruption, and the typed error channel flow through without round-tripping to Promise: ```ts type RpcInterceptor = ( next: Effect.Effect, context: { readonly tag: string // 'todos.create', 'support.ping', … readonly kind: 'mutation' | 'action' // discriminator readonly input: unknown // the validated payload readonly subject: Subject // resolved by AuthMiddleware readonly traceId: string // matches OTel spans + logs }, ) => Effect.Effect ``` The interceptor MUST flow `next` through somehow — yield it, pipe through it, or return it. Whatever the returned Effect produces becomes the final result UNLESS the interceptor substitutes a different one (cache hits, etc). Failing the returned Effect skips the rest of the chain and surfaces the error to the rpc layer. ```ts // Pre-only: short-circuit before the executor runs. const guard: RpcInterceptor = (next, ctx) => ctx.tag.startsWith('admin.') && ctx.subject.type !== 'user' ? Effect.fail(new ScopeError({ required: 'user', message: ` is admin-only` })) : next // Post-only: tap the success/failure channels. const log: RpcInterceptor = (next, ctx) => next.pipe( Effect.tap((result) => recordAudit(ctx.tag, ctx.input, result, ctx.subject)), Effect.tapError((err) => recordAuditError(ctx.tag, err)), ) // Tracing: spans nest automatically. const trace: RpcInterceptor = (next, ctx) => next.pipe(Effect.withSpan(`plugin.${ctx.tag}`)) ``` **Composition.** When multiple plugins each install `interceptMutation`, the framework composes them in declaration order: the first plugin in `app.config.ts`'s `plugins:` array is the OUTERMOST wrapper, the last is closest to the executor. That mirrors how middleware composition normally reads top-to-bottom in user code. **Why three hooks instead of one.** A plugin that only wants to govern outbound HTTP (rate limiting, tenant-scoped IO quotas) installs `interceptAction` without touching mutations. A plugin that records writes (audit log) installs `interceptMutation` without touching reads or actions. The split lets plugins opt in narrowly. `interceptQuery` wraps subscription **setup** — the one-time call that produces the `QueryDescriptor`. Use it for pre-setup authz (deny before the subscription opens), per-tenant subscription-rate-limiting, or subscription-open audit logging. **It does NOT wrap every snapshot/delta delivery** — per-delta observability flows through the framework's OTel spans (`subscription..snapshot` / `.delta`); wrapping every delivery would add per-event latency the streaming model is specifically designed to avoid. ```ts interceptQuery: (next, ctx) => ctx.subject.type === 'anonymous' && ctx.tag.startsWith('admin.') ? Effect.fail(new Unauthenticated({ reason: 'admin queries require authentication' })) : next, ``` A failure inside an `interceptQuery` aborts the subscription open; the rpc layer surfaces it to the client as a subscription error. Substituting the return Effect is supported but rare — usually interceptors do pre-only or post-only side effects. **Per-plugin observability is automatic.** Every interceptor is wrapped at compose time with `Effect.withSpan('plugin..intercept-')` + a `plugin.*` metric sample — no opt-in. The DevTools / cloud dashboard's Plugins panel filters `/_voltro/inspect/metrics` by the `plugin.*` prefix to render per-plugin latency + count buckets, and the trace waterfall shows the plugin layer as a nested span inside the handler boundary. ### `onChangeEvent` — the post-commit ChangeEvent tap ```ts onChangeEvent: (event: PluginChangeEvent) => Effect.Effect // PluginChangeEvent = { // table, op: 'insert'|'update'|'delete', new: Row | null, old: Row | null, // origin?: 'inline' | 'injected', changeScope: 'local' | 'fleet', // traceId?, subjectId?, procedure?, // oversized?: 'rehydrated' | 'tombstone' | 'unrecovered', // } ``` A tap on the store's post-commit change stream — the plugin sees every committed insert/update/delete. It returns an `Effect` that the runtime **supervises**: it forks the Effect under the plugin's supervision scope (so the tap is non-blocking and can never back-pressure the change stream or the writing mutation) and routes the Effect's failure channel to the plugin-scoped logger. That gives a change-tap a real, typed error channel — `Effect.retry`, `Effect.timeout`, `Effect.catchTag`, a durable enqueue — instead of fire-and-forget glue. ```ts onChangeEvent: (event) => event.table !== 'orders' ? Effect.void // not my table — no-op : Effect.tryPromise(() => mirror(event)).pipe( // failure is logged by the runtime Effect.retry({ times: 3 }), ) ``` **Check `event.oversized` before you treat an image as a snapshot.** It is absent on an ordinary event, and set when the transport could not carry the row and the images were reconstructed — a row over postgres' 8000-byte NOTIFY cap. `'rehydrated'` means `new` is the row re-read from the database (correct to index or forward, but the row as it is NOW rather than the image at commit); `'tombstone'` means a delete whose `old` is the primary key and nothing else (enough to REMOVE the row, never a record of what it held); `'unrecovered'` means both images are null and the content is gone. A tap that stores history must not write a tombstone as a snapshot. Full guarantee: [postgres — oversized rows](/docs/database/dialects/postgres). Runs under BOTH `voltro dev` and `voltro serve` (the prod serve path fans out the same way). Requires the `'store:changes:read'` permission. It is NOT durable at the framework layer — a crash between commit and the fork loses the event; build durability INSIDE the Effect (insert into an outbox and retry against the typed error channel, the way `@voltro/plugin-cdc-out` does). Exactly-once / change-scope semantics are unchanged: read `event.origin` + `event.changeScope` inside the Effect to act once per change fleet-wide (skip `origin: 'injected'` on `'local'` scope; elect one worker on `'fleet'`). Used by `@voltro/plugin-search` to mirror rows into an external index. For in-transaction reactions use a mutation; for best-effort per-table reactions in app code prefer a `*.subscribe.ts` — `onChangeEvent` is the plugin-level equivalent. ### `onInstall` / `onUninstall` / `onActivate` / `onDeactivate` lifecycle Four lifecycle hooks, all accepting Effect / Promise / sync return values. The framework awaits each in turn — sync returns settle synchronously, Promises and Effects are awaited. ```ts interface PluginLifecycleContext { readonly app: { readonly name: string readonly type: 'api' | 'web' readonly voltroVersion: string } readonly logger: { info; warn; error; debug } // scoped to the plugin's name readonly env: NodeJS.ProcessEnv readonly config: unknown // validated against configSchema if declared } ``` - **`onInstall`** runs ONCE per process per `plugin@version` — the FIRST time the host sees this plugin. Use for schema migrations + resource provisioning that survive across deactivate/activate cycles. In v1 the install marker is in-memory; persisted state (`_voltro_plugin_installs` table) lands with the migration runner. Failure aborts boot. - **`onUninstall`** is the teardown mirror of `onInstall` — declare it to drop tables, delete queues, etc. when the plugin is removed from the app. Keep it idempotent (re-running is a no-op once torn down). - **`onActivate`** runs every boot. Use for warming caches, opening connection pools, registering metrics sinks. The framework awaits each hook sequentially in declaration order. **Throwing/failing aborts boot** — better to surface a misconfigured plugin loudly than start the rpc server with half-wired plugins. - **`onDeactivate`** runs in **REVERSE declaration order** (mirror of activate — the most-recently-activated plugin tears down first). Each hook has a **5-second per-plugin grace window** and the whole sequence is capped at **30 seconds total**. A plugin that throws or exceeds its grace window logs a warning + the sequence moves on; shutdown never blocks on a stuck plugin. **Idempotent** — repeated SIGTERM doesn't re-run hooks. Hookup covers SIGINT + SIGTERM in both `voltro dev` and `voltro start`. ### `framework` compatibility ```ts definePlugin({ name: '@vendor/x', framework: '^1.0.0', // accepts 1.x, rejects 0.x and 2.x // OR: '~1.2.0', '>=1.0.0 <2.0.0', '1.2.3' (exact), '*' (any) }) ``` At boot the framework runs `checkFrameworkCompat(plugin.name, plugin.framework, runningVoltroVersion)`. Mismatch logs a `WARN` with the constraint + the running version, and **boot continues by default** — the operator decides whether to pin a different version or upgrade. Set `VOLTRO_STRICT_PLUGIN_COMPAT=1` to make a mismatch a hard boot-abort instead (the plugin throws at boot) — for CI or regulated deployments that must refuse to run a plugin built against a different framework version. Absent `framework` field → no compat check. Suitable for in-tree plugins that ship lockstep with the framework. ## What plugins CAN'T do (initial policy) - **Mutate other plugins' state.** Plugins don't talk to each other directly. If two plugins need to coordinate, it's via the rpc layer (one plugin's interceptor sees the other's `subject.metadata`, for example). - **Bypass tenant scoping.** Interceptors run AFTER the runtime's tenant predicate merge. A mutation interceptor can't query rows from another tenant by manipulating `subject.tenantId`. - **Access raw secrets directly.** Plugins get config via their own factory function's options object. `lifecycle.env` exposes `process.env` but only the plugin's own factory chooses which env vars to read. - **Modify the core schema DSL or query builder.** Schema extension happens through schema mixins (`defineMixin` from `@voltro/database`); not through plugin runtime hooks. These boundaries hold for v1; some may relax for verified plugins once the marketplace ships. ## Stacking ```ts // app.config.ts import { auditPlugin } from '@voltro/plugin-audit' import { rateLimitPlugin } from '@vendor/rate-limit' import { metricsPlugin } from '@vendor/metrics' export default { type: 'api' as const, name: 'myApi', plugins: [ metricsPlugin({ sink: 'datadog' }), // OUTERMOST — sees every request first rateLimitPlugin({ perTenant: 100 }), auditPlugin({ sink: 'console' }), // INNERMOST — closest to executor ], } ``` The first plugin's `interceptMutation` wraps the second's, which wraps the third's, which wraps the executor. Same composition for `interceptAction`. The chain runs in deterministic order independent of import order or filesystem walk — only `plugins:` array order matters. ### `services: Layer` — contribute Tags into the per-request runtime A plugin can provide an Effect Layer whose Tags become available to EVERY handler in the app. Handlers `yield* MyTag` to read; plugins own the implementation; consumers don't import the plugin directly. ```ts import { Effect, Layer } from 'effect' import { definePlugin, definePluginService } from '@voltro/protocol' interface AuditService { readonly record: (event: { tag: string; actor: string }) => Effect.Effect } const { Tag: Audit, Live: AuditLive } = definePluginService( '@vendor/audit/Service', { record: (event) => Effect.sync(() => console.log('AUDIT', event)), }, ) export const auditPlugin = (): VoltroPlugin => definePlugin({ name: '@vendor/audit', services: AuditLive, }) // In any handler: // import { Audit } from '@vendor/audit' // const audit = yield* Audit // yield* audit.record({ tag: 'todos.create', actor: 'u_42' }) ``` Layer composition is independent of declaration order across plugins (Effect's `mergeAll` is commutative on Tag identity); collisions on the SAME Tag resolve to the LAST layer in the merge list, so a user layer in `apiConfig.layers` can override a plugin layer with the same Tag. ### `routes: PluginRpcRoute[]` — plugins ship their own RPC endpoints Plugins can register their own queries / mutations / actions alongside user-authored queries. Same wire protocol, same dashboard surface, same interceptor + tracing wiring. Query tags carry a plugin-derived prefix so plugin queries never collide with user queries: ```ts import { definePlugin, definePluginRoute } from '@voltro/protocol' import { Effect, Schema } from 'effect' export const auditPlugin = (): VoltroPlugin => definePlugin({ name: '@voltro/plugin-audit', routes: [ definePluginRoute({ kind: 'query', name: 'list', // tag = 'audit.list' (plugin alias prepended) input: Schema.Struct({ limit: Schema.optional(Schema.Number) }), output: Schema.Array(AuditEventSchema), execute: (input) => Effect.gen(function* () { const buffer = yield* AuditBuffer // service Tag from plugin's `services` layer return buffer.read(input.limit ?? 100) }), }), definePluginRoute({ kind: 'mutation', name: 'clear', // tag = 'audit.clear' input: Schema.Struct({}), output: Schema.Struct({ cleared: Schema.Number }), execute: () => Effect.sync(() => { // clear logic — runs inside the framework's transactional wrap return { cleared: 0 } }), }), ], }) ``` Tag derivation: `.`. The plugin alias strips `@scope/` + the `plugin-` prefix and kebab→camelCase: | Plugin name | Alias | |---|---| | `@voltro/plugin-audit` | `audit` | | `@scope/plugin-rateLimit` | `rateLimit` | | `@scope/plugin-rate-limit` | `rateLimit` | | `plain-name` | `plainName` | | `@voltro/audit` | `audit` (no `plugin-` to strip) | A `query.name` that already contains a dot is handled by whether it names the plugin's OWN namespace: - `'notifications.inbox'` on a plugin whose canonical name is `@voltro/plugin-notifications` is re-namespaced — under `alias: 'inbox'` it becomes `inbox.inbox`, not `notifications.inbox`. A deeper path keeps its depth: `'audit.admin.events'` under `alias: 'trail'` becomes `trail.admin.events`. - `'acme.legacyBridge'` — a namespace that is not the plugin's own — passes through untouched. That is the escape hatch, and it is the only case that still bypasses the alias. The re-namespacing needs the plugin to declare `baseName` (its canonical name, before any app-supplied `alias`); a plugin that omits it keeps the older behaviour where any dotted name passes through. ### Naming a plugin: `alias` vs `name` Two different app-side problems land on a plugin's name, so first-party plugins that carry tables or routes accept two separate options: | Option | Question it answers | Effect | |---|---|---| | `alias` | "your namespace collides with mine" | replaces the namespace — tags become `.`, the inspect mount becomes `/_voltro/inspect/plugins//…` | | `name` | "I want two of these" | appends a `#suffix` discriminator so two installs never register the same tag | ```ts notificationsPlugin({ alias: 'alerts' }) // alerts.inbox notificationsPlugin({ name: 'ops' }) // notifications#ops.inbox notificationsPlugin({ alias: 'alerts', name: 'ops' }) // alerts#ops.inbox ``` `alias` exists to escape a tag collision, which is fatal at codegen. Two costs are worth knowing before you reach for it: - the local and cloud dashboards fetch a plugin's inspect panel at its DEFAULT slug, so an aliased plugin keeps serving its inspect endpoints while its dashboard panel stops resolving; - the plugin-migration ledger key is `__`, so aliasing a plugin that ships `extendSchema.migrations` makes its already-applied migrations look unapplied. Choose the alias before first boot, not after. ### `tables: false` — keeping your own tables Plugins whose tables carry no authorization or safety decision accept `tables: false`, which stops them contributing DDL through `extendSchema` so an app can keep equivalent tables it already has. Everything else — routes, inspect, interceptors — is unchanged. ```ts notificationsPlugin({ tables: false }) // you declare the six notification tables ``` It is offered on `@voltro/plugin-rbac`, `@voltro/plugin-audit`, `@voltro/plugin-notifications` and `@voltro/plugin-ai-flows`. The plugin still writes to those tables BY NAME, so you take over declaring each one with the shape the package exports, and a missing or mis-shaped table fails at the first write rather than at boot. It is deliberately NOT offered on plugins whose tables carry a guarantee — the SAML assertion replay cache, SCIM provisioning state, billing's usage counters, cdc-out's delivery outbox, the governance consent ledger, search's tenant-scoped index rows. A `tables: false` there would disable a security or correctness decision with no signal to the app that it now owns it. Boot fails with a clear error on tag collisions (between two plugins, or with a user-authored tag). #### Reactive plugin queries — `source` A plugin query can be **push-driven** instead of poll-only. Add a `source` table (or tables) to a `kind: 'query'` route and the framework re-runs the executor and pushes a fresh result over the SAME subscription/WS transport app reactive queries use — every time one of those tables changes. No new push system: it reuses the framework's computed-reactive-query machinery, so the client just subscribes and receives live updates. ```ts definePluginRoute({ kind: 'query', name: 'list', source: '_voltro_presence', // ← reactive: re-run + push on any change to this table input: Schema.Struct({ channel: Schema.String }), output: Schema.Array(MemberSchema), // The executor returns the query's VALUE (the same shape it returns when // polled); the framework recomputes + pushes it on change. execute: (input, ctx) => Effect.gen(function* () { // read the table, shape the roster, return it return roster }), }) ``` `source` must name a table that is reactive — every table is, unless it opted out with `.nonReactive()`. Omit `source` for a plain poll-only plugin query, a mutation, or an action. This is exactly how `@voltro/plugin-presence` makes `presence.list` push-driven — the `usePresence` roster updates live with no client polling. ### `permissions: PluginPermission[]` + `configSchema: Schema` — manifest fields ```ts definePlugin({ name: '@vendor/audit', version: '1.2.3', permissions: [ 'rpc:intercept:mutation', // matches interceptMutation 'rpc:intercept:query', // matches interceptQuery 'secrets:read:audit:*', // pattern-perm (runtime check) ], configSchema: Schema.Struct({ sink: Schema.Literal('console', 'memory', 'postgres'), verbose: Schema.optional(Schema.Boolean), }), }) ``` `permissions` is a **typed enum**. The framework audits every plugin's declared set against its hook surfaces at boot — a plugin that ships `onHttpRequest` without declaring `'http:intercept'` (or any other hook-without-perm combination) fails boot with the specific scope name. No silent strip; no advisory warnings; the operator either grants the perm or the plugin doesn't run. The static surface is a union of well-known scopes; pattern perms (`'secrets:read:foo:*'`, `'network:outbound:api.stripe.com'`, `'plugin:hook:'`) fall through the template-literal arm. Runtime resource checks use `permissionMatches(declared, required)` — a declared `'secrets:read:auth0:*'` covers a required `'secrets:read:auth0:clientSecret'`. | Hook surface | Required permission | |---|---| | `interceptMutation` | `'rpc:intercept:mutation'` | | `interceptQuery` | `'rpc:intercept:query'` | | `interceptAction` | `'rpc:intercept:action'` | | `onScheduleFire` | `'schedule:fire'` | | `onWorkflowStep` | `'workflow:step'` | | `onHttpRequest` | `'http:intercept'` | | `onChangeEvent` | `'store:changes:read'` | | `inspectEndpoints` (non-empty) | `'inspect:read'` | | `dashboard` (non-empty) | `'dashboard:mount'` | | `extendSchema.tables` (non-empty) | `'store:write'` | | `extendSchema.migrations` (non-empty) | `'migration:run'` | `configSchema` decodes the operator's user-supplied config payload at boot. Decode failures abort boot with a typed error pointing at the plugin. The decoded value lands in `PluginLifecycleContext.config` so lifecycle hooks consume the already-validated shape. ## Writing a plugin — minimal example ```ts // packages/plugin-rate-limit/src/index.ts import { Effect } from 'effect' import { definePlugin, type VoltroPlugin } from '@voltro/protocol' interface RateLimitOptions { readonly perTenant: number // requests/minute } const buckets = new Map() export const rateLimitPlugin = (options: RateLimitOptions): VoltroPlugin => definePlugin({ name: '@vendor/rate-limit', framework: '^1.0.0', interceptMutation: (next, ctx) => { if (ctx.subject.type === 'anonymous') return next // no per-tenant limit const key = ctx.subject.tenantId const now = Date.now() const bucket = buckets.get(key) ?? { count: 0, windowStart: now } if (now - bucket.windowStart > 60_000) { bucket.count = 0 bucket.windowStart = now } bucket.count++ buckets.set(key, bucket) if (bucket.count > options.perTenant) { return Effect.fail(new Error(`rate limit exceeded for tenant ${key}`)) } return next }, onActivate: ({ logger }) => { logger.info('rate-limit plugin armed', { perTenant: options.perTenant }) }, }) ``` That's the whole plugin. Drop it in `app.config.ts`, `voltro dev` picks it up, every mutation goes through the rate limiter. ## Schema mixins — the OTHER plugin shape Not every cross-cutting concern needs runtime hooks. Schema extensions live in their own surface — `defineMixin({...})` from `@voltro/database` — and don't go through the `VoltroPlugin` contract at all. See [database/mixins](/docs/database/mixins) for that pattern. A package can ship BOTH a schema mixin AND a runtime plugin (`@voltro/plugin-audit` does — the `audit()` mixin adds columns; the `auditPlugin()` factory installs the mutation interceptor). They're independent — apps that want the schema but not the interceptor `.with(audit())` without `plugins: [auditPlugin()]`, and vice versa. ## More extension surfaces The plugin contract carries the additional surfaces below, beyond the interceptors + lifecycle + manifest fields covered above: ### `inspectEndpoints: PluginInspectEndpoint[]` — plugin-mounted HTTP endpoints ```ts import { definePlugin } from '@voltro/protocol' import { Effect } from 'effect' export const auditDebugPlugin = (): VoltroPlugin => definePlugin({ name: '@voltro/plugin-audit-debug', inspectEndpoints: [ { method: 'GET', path: 'buffer', // → /_voltro/inspect/plugins/auditDebug/buffer description: 'Returns the last 1000 audit events', handler: () => Effect.sync(() => ({ kind: 'json', data: { events: readAuditBuffer() }, })), }, { method: 'POST', path: 'clear', handler: () => Effect.sync(() => { clearAuditBuffer() return { kind: 'json' as const, data: { cleared: true } } }), }, ], }) ``` Plugin endpoints inherit the framework's auth resolver — plugins can't bypass `VOLTRO_INSPECT_TOKEN`. Path: `/_voltro/inspect/plugins//`. Boot fails loudly on collisions (two plugins claiming the same path+method). ### `onScheduleFire: ScheduleFireInterceptor` — wrap every cron firing ```ts definePlugin({ name: '@vendor/schedule-gate', onScheduleFire: async (next, ctx) => { // Suppress all firings of `nightlyBilling` when the kill-switch is on. if (ctx.name === 'nightlyBilling' && (await isKilled('billing'))) { return // skip — handler doesn't run } await next() }, }) ``` Composed across plugins in declaration order; runs INSIDE the schedule's `maxRuntimeMs` watchdog. Per-plugin span (`plugin..schedule-fire`) + metric auto-emitted. ### `onWorkflowStep: WorkflowStepInterceptor` — wrap every `step()` ```ts import { Effect } from 'effect' definePlugin({ name: '@vendor/step-observer', onWorkflowStep: (next, ctx) => next.pipe( Effect.withSpan(`vendor.workflow.${ctx.stepName}`, { attributes: { 'workflow.attempt': ctx.attempt }, }), Effect.tap((output) => Effect.sync(() => recordStep(ctx, output))), ), }) ``` Wraps every `step()` (== `Activity.make`) inside a workflow body. Effect- native — the interceptor sees `next` as an Effect and can pipe through `tap`/`retry`/`withSpan`. **Bypassed on workflow REPLAY** — the interceptor only fires on the first execution per step; on resume after a crash, the cached Activity output is replayed without re-running the user effect or the plugin interceptor. ### `codegen: PluginCodegen` — emit typed bindings into `rpcGroup.generated.ts` ```ts definePlugin({ name: '@voltro/plugin-audit', codegen: (ctx) => ` // Typed accessor for the audit plugin's inspect endpoint export const useAuditBuffer = () => fetch('/_voltro/inspect/plugins/${ctx.pluginAlias}/buffer') .then((r) => r.json() as Promise<{ events: ReadonlyArray }>) `, }) ``` The framework emits the returned string into `rpcGroup.generated.ts` between `// ` / `// ` markers. The plugin sees the api name + the list of discovered user-query rpc tags so it can emit per-rpc bindings. Returning `null` contributes nothing. ### `templates: PluginTemplate[]` — ship scaffolding templates ```ts definePlugin({ name: '@vendor/plugin-stripe-webhooks', templates: [ { id: 'stripe-receiver', title: 'Stripe webhook receiver', description: 'Verified Stripe-webhook endpoint with replay protection', kind: 'api', sourcePath: 'templates/stripe-receiver', postInstallSteps: [ 'Set VOLTRO_WEBHOOK_SECRET_STRIPE in your env', 'Configure the endpoint URL in your Stripe dashboard', ], }, ], }) ``` Templates surface under the plugin alias in `voltro list-templates`. The registration shape + manifest are part of the public contract, and a plugin-provided template is scaffolded exactly like a built-in one — `scaffoldFromTemplate` copies the declared tree and substitutes the `{{…}}` tokens in both file content AND file/directory names. ### `httpRoutes: PluginHttpRoute[]` — raw HTTP endpoints on the framework listener A plugin can mount plain HTTP routes beside the rpc surface (`@voltro/plugin-storage`'s upload/download routes, an IdP callback). The request/response shape is transport-honest, and four properties are worth pinning: - **The full method union is first-class.** `method` is `'*' | 'GET' | 'POST' | 'PUT' | 'PATCH' | 'DELETE' | 'HEAD' | 'OPTIONS'`. **HEAD is admitted wherever GET is** (RFC 9110) — the GET handler runs and the transport drops the body; you never mount a second route for it. A wrong method stays a precise `405` with an `Allow:` header, including when several routes share one path. - **The body read is capped** — 8 MiB by default, the same cap as every other surface (`http.maxBodyBytes` in `app.config.ts`, env `VOLTRO_MAX_BODY_BYTES`), and the read is binary-clean. A route that takes more declares its own `maxBodyBytes`; routes **sharing a path share one body read**, so the widest override in the group applies to the group. Oversize answers `413` for both `Content-Length` and chunked requests. - **Binary streaming responses** return `byteStream` on the `PluginHttpRouteResult` — a `ReadableStream` (or lazy thunk) with optional `contentLength` / `contentDisposition`, piped without buffering and never compressed. It is the plugin-route spelling of the REST surface's [`bytes()`](/docs/data/rest-routes#binary-downloads--bytes). - **Buffered responses are compression-negotiated** (brotli/gzip, compressible types only) by the listener — nothing to declare; see [Security → compression](/docs/security/overview). A state-changing plugin route is origin-checked unless it declares `originGuard: 'exempt'`, and `req.remoteAddr` is the trusted-proxy-resolved client address — both covered with examples in [Security](/docs/security/overview#routes-that-a-third-party-legitimately-posts-to). ### `onHttpRequest: HttpRequestInterceptor` — pre-auth HTTP-pipeline hook Fires at the **very top** of every HTTP request — BEFORE auth resolution, BEFORE rpc routing, BEFORE inspect. Composed across plugins in declaration order (first listed = outermost). Returning a `HttpInterceptResponse` short-circuits the pipeline; returning `null` (or calling `next()` and returning its result) lets the request flow through to the framework's normal handlers. This is **not** an auth replacement — `AuthMiddleware` still runs on the rpc path. Use this hook for concerns that need to fire BEFORE auth: rate-limit, geo-block, bot detection, header injection for downstream observability. Requires the `'http:intercept'` permission. ```ts import { definePlugin } from '@voltro/protocol' export const rateLimitPlugin = (opts: { perMinute: number }) => definePlugin({ name: '@vendor/plugin-rate-limit', permissions: ['http:intercept'], onHttpRequest: async (next, ctx) => { // `ctx.remoteAddr` is ALREADY resolved through the app's // `security.trustedProxies` policy — the same value the framework's own // rate limiter, geo-block and audit rows use. Never read // `ctx.headers['x-forwarded-for']`: it is a request header, so any // caller can write it, and a limiter keyed on it is bypassed by one // extra header. `undefined` only when the socket address is unavailable. const clientAddr = ctx.remoteAddr ?? 'unknown' if (buckets.consume(clientAddr, opts.perMinute) === 'exhausted') { return { status: 429, headers: { 'retry-after': '60', 'content-type': 'application/json' }, body: JSON.stringify({ error: 'rate_limit_exceeded' }), } } return next() // let the request continue }, }) ``` **`ctx.remoteAddr` is the client address — `x-forwarded-for` is not.** The framework resolves `remoteAddr` through the app's `security.trustedProxies` policy (`VOLTRO_TRUSTED_PROXIES`) before it hands you the context: with no trusted proxy declared the header is ignored entirely and the socket address wins, and with one declared only the hops that are actually a configured proxy are believed. Reading `ctx.headers['x-forwarded-for']` yourself throws that away and keys your limiter on a string the caller typed — one extra header and every request looks like a new client. The same rule applies to a plugin's raw HTTP routes, where the resolved value arrives as `req.remoteAddr`; see [Trusted proxies](/docs/security/overview#the-same-address-reaches-your-plugin-routes). A per-plugin `plugin..http-intercept` metric is auto-emitted so the dashboard's Plugins panel surfaces HTTP-intercept latency next to RPC-intercept latency. **Runs on both boot paths.** The chain is composed and installed identically by `voltro dev` and `voltro serve` — this is a production capability, and for a pre-auth shield production is the point. There is exactly ONE exemption, and it is deliberate: `GET /internal/liveness` and `GET /internal/readiness` are answered before the interceptor, so a rate-limit or geo-block plugin cannot 503 a Kubernetes probe and take the replica out of rotation. **The chain is FAIL-CLOSED.** An interceptor that throws is a `500` plus a log line — the request does NOT continue. It used to: the failure was swallowed and the request flowed on, which meant a crashed security gate was an open one. That polarity puts a decision on every interceptor author: if your hook is a GATE (geo-block, bot detection), let a failure propagate — refusing is the correct degraded behaviour. If it is protection with a DEPENDENCY (a rate-limit counter in Redis), catch your own failure inside the hook and **degrade loudly** — `@voltro/plugin-ratelimit`'s `httpShield` does exactly that: a Redis outage means unlimited-with-a-warning, never a self-inflicted API outage. What no interceptor gets to do anymore is fail silently and stay in the chain. ### `extendSchema: { tables, migrations }` — contribute schema + migrations A plugin contributes BOTH declarative table descriptors AND custom SQL migrations. Tables merge into the user's schema and flow through the same idempotent `applySchema()` path. Migrations run after `applySchema` against the live `SqlClient` and are tracked in `_voltro_plugin_migrations` so each runs exactly once per app database. The ledger key is `__` so two plugins can each ship `'001-init'` without collision. Failure aborts boot; re-runs are no-ops. **Which commands run them:** `voltro dev`'s boot auto-migrate, `voltro db apply` (bare and `--plan`) and `voltro migrate --create-only`. NOT `voltro serve` — serve never applies a schema, so a plugin's steps land in the pre-deploy job alongside the schema, which is where they belong. Until 0.34.0 only the `voltro dev` boot ran them, so a plugin's SQL steps executed on every developer machine and on no deployed database; if you ship migrations, verify against a deployed database rather than a dev boot. ```ts import { Effect, Schema } from 'effect' import { definePlugin } from '@voltro/protocol' import { table, id, text, timestamp } from '@voltro/database' const auditLogs = table('audit_logs', { id: id({ prefix: 'audit' }), actorId: text(), action: text(), payload: text(), createdAt: timestamp().default('now'), }) export const auditPlugin = () => definePlugin({ name: '@voltro/plugin-audit', permissions: ['store:write', 'migration:run', 'rpc:intercept:mutation'], extendSchema: { tables: [auditLogs], migrations: [ { id: '001-pgcrypto-extension', description: 'enable pgcrypto for hashing actor ids', up: (sql) => Effect.gen(function* () { yield* sql`CREATE EXTENSION IF NOT EXISTS pgcrypto;`.pipe(Effect.orDie) }), }, ], }, }) ``` `tables` requires `'store:write'`; `migrations` requires `'migration:run'`. Either one missing → boot fails with the specific permission name. ### `dashboard: PluginDashboardMount[]` — remote-mounted dashboard surfaces Plugins surface UI in the cloud / devtools dashboard by declaring remote-loaded ESM modules. The host runtime fetches the bundle URL at mount time, dynamic-imports it, and renders the exported component inline (NOT iframe — true in-process mount). Trade-offs vs iframe: - The plugin's bundle MUST be ESM with React/Effect as peerDeps; the host pins versions. Mismatch logs a warning at mount time but still mounts. - No CSS isolation. Plugin authors are expected to use scoped Tailwind classes (the dashboard ships the framework token set) or CSS modules. - Plugin code shares the host JS realm — honor-system sandboxing. The `dashboard:mount` permission gate keeps the operator audit trail explicit; signing/marketplace verification is the longer-term answer. ```ts definePlugin({ name: '@vendor/plugin-audit', permissions: ['dashboard:mount'], dashboard: [ { id: 'overview', slot: 'page', // 'page' | 'widget' | 'nav' route: 'overview', // mounted at /plugins/audit/overview label: 'Audit log', icon: 'shield', // optional lucide-react icon name bundleUrl: 'https://cdn.example.com/audit/v1/dashboard.mjs', exportName: 'AuditOverview', // optional; defaults to 'default' dashboardVersion: '^1.0.0', // optional compat range }, { id: 'recent-events', slot: 'widget', // card on the dashboard home label: 'Recent audit events', bundleUrl: 'https://cdn.example.com/audit/v1/widget.mjs', }, ], }) ``` The registry surfaces at `/_voltro/inspect/plugins/dashboard-mounts`: ```bash curl http://localhost:4000/_voltro/inspect/plugins/dashboard-mounts | jq # { "mounts": [...], "capturedAt": 1717142400000 } ``` Boot validates within-plugin id uniqueness AND that `slot: 'page'` mounts carry a `route`. Cross-plugin ids are namespaced as `:` (e.g. `audit:overview`). The actual dashboard-host runtime — the React shell that dynamic-imports + mounts the component — lives in `voltro-cloud-dashboard` + `voltro-devtools`. The framework owns the registration shape + manifest; the consumers own the renderer. ### `bindDataStore(store, ctx)` — bind DB-backed resources + framework handles `onActivate` runs BEFORE the app's `DataStore` exists (the framework activates plugins, THEN opens the store). `bindDataStore` is the post-store hook: the framework calls it once, after the store + pool are open, so a plugin can bind a resource it couldn't build from static config — e.g. swap an in-memory ref store for a `DataStore`-backed one. The second argument, `ctx: PluginBindContext`, hands the plugin the framework's ALREADY-OPEN handles so it never rebuilds them: - **`ctx.sql`** — the framework's live `SqlClient` (from `@effect/sql`), the SAME pool the app's store uses. Run raw SQL through it instead of standing up your OWN `ManagedRuntime` + pool from env. `undefined` on the in-memory store (no SQL engine) — guard with `if (ctx.sql)`. - **`ctx.scheduleCoordinated(name, intervalMs, effect)`** — run a periodic task on ONLY ONE replica per tick, cluster-coordinated via the same claim-table exactly-once gate the cron scheduler uses. Replaces the hand-rolled `setInterval` a plugin would otherwise run inside `bindDataStore` — which fires on EVERY replica, so a 10-pod deployment runs the same full-table sweep 10× per interval. On a single-process / memory / sqlite deployment it simply runs every tick locally (correct — one process needs no fan-out dedup). Returns a handle whose `stop()` cancels the task; the framework also stops every armed task at shutdown. Each tick claims one row in `_voltro_schedule_claims`, so the interval you pick is also a write rate: `250` is four rows per second, per task, fleet-wide. **The interval is a floor, not a cadence — if you report idleness.** Return `{ idle: true }` from the effect on a tick that found nothing to do, and the runner backs off toward a 30 s ceiling (`VOLTRO_POLL_CEILING_MS`) instead of ticking at your interval forever. Return nothing and the tick keeps its fixed interval, which is the safe default for a task that cannot tell. **Or stop it entirely.** Pass `{ disarmWhenIdle: true }` as a fourth argument and an idle tick with nothing pending stops the timer altogether — the task runs once at startup and then only when `wake()` says so. That is the difference between a cheaper poller and no poller, and it is what the framework's own two background tasks do. Only pass it when an arrival is **guaranteed** to call `wake()`. That is a claim about the deployment, not the task: with Postgres LISTEN/NOTIFY or a broadcast broker every replica sees every write, so it holds. Without either, a peer replica's enqueue produces no local event and a disarmed task would sleep through it — the backoff ceiling is the correct choice there. Two more fields make that affordable rather than a latency tax: - `handle.wake()` runs a tick **now**. Wire it to whatever announces work — a `store.onChange` on the queue table your plugin drains — and the work starts on arrival rather than up to one interval later. Calls are coalesced to at most one extra tick per interval, so calling it per row is fine. - `{ idle: true, nextDueInMs }` caps the backoff. Return it when the task is idle right now but already knows something is coming (a window closing, a lease expiring); the next tick lands on that instant instead of on the ceiling. ```ts const outbox = ctx?.scheduleCoordinated('vendor.outbox', 1_000, async () => { const sent = await drainOutbox() return { idle: sent === 0 } }, { disarmWhenIdle: true }) store.onChange((event) => { if (event.table === 'vendor_outbox' && event.op === 'insert') outbox?.wake() }) ``` Why this exists: two framework tasks polled permanently-empty queues on a deployment's deployment and wrote **2 506 claim rows an hour** between them, against 18 from the app's own eight schedules. A fixed interval has no way to learn that a queue is empty. Without a `wake()` source the ceiling is your worst-case latency, so a task whose arrivals cannot announce themselves should keep reporting nothing. ```ts import { definePlugin } from '@voltro/protocol' definePlugin({ name: '@vendor/plugin-presence', permissions: ['store:write'], bindDataStore: (store, ctx) => { // Reuse the framework's open pool — no connFromEnv, no second pool. if (ctx?.sql) { // ctx.sql is the app's live SqlClient (Effect-native). } // One coordinated sweep fleet-wide, not one setInterval per replica. ctx?.scheduleCoordinated('presence.sweep', 60_000, async () => { await store.deleteMany('_voltro_presence', { where: { column: 'lastSeen', op: 'lt', value: Date.now() - 120_000 }, }) }) }, }) ``` `store` is typed as the framework `DataStore`; `ctx` is optional in the type (the framework always passes it) so an older single-argument `bindDataStore(store)` still compiles. Requires no extra permission beyond whatever the store operations themselves need (`store:write` for writes). ## Inspecting plugins at runtime Every running app exposes its plugin manifest at `/_voltro/inspect/plugins`: ```bash curl http://localhost:4000/_voltro/inspect/plugins | jq ``` Returns one entry per plugin in `app.config.ts`'s `plugins:` array with: - `name`, `version`, `description`, `framework` semver range - `permissions: string[]` — declared scopes - `hooks` — which interceptor + lifecycle hooks the plugin installed - `queries` — `kind:name` tuples for every plugin-contributed query - `services: boolean` — whether the plugin contributes a service layer - `hasConfigSchema: boolean` — whether `configSchema` is declared - `activated`, `installed` — runtime state The DevTools + cloud dashboard's Plugins panel consumes this; AI agents grep it to answer "what is this deployment running, with what permissions". Per-plugin metrics surface under the SAME `/_voltro/inspect/metrics` endpoint that carries `rpc.*` / `request.*` buckets — plugin buckets have `kind: 'plugin'` and tag `plugin..intercept-`. Filter client-side by the `plugin.` prefix to render the per-plugin latency table. ## Building integration plugins — `@voltro/integration-http` A plugin that talks to a third-party REST API (Jira, GitHub, GitLab, …) needs the same server-side transport concerns every time: token auth, transient retry with backoff, per-request timeout, an SSRF host guard, and typed errors. `@voltro/integration-http` is that transport core — one implementation shared by `@voltro/plugin-atlassian` and any integration plugin you write, so you don't re-invent it. `makeHttpClient` binds a base URL, an auth strategy, a retry/timeout policy, and a `fetch` impl once, then hands you typed request methods. Every method returns `Effect.Effect` — failures land on **your** error channel via the `makeError` you supply, so each integration keeps its own `Schema.TaggedError`: ```ts import { makeHttpClient, type FetchLike } from '@voltro/integration-http' import { Schema } from 'effect' class GithubError extends Schema.TaggedError()('GithubError', { message: Schema.String, transient: Schema.Boolean, status: Schema.optional(Schema.Number), code: Schema.optional(Schema.Literal('session_expired')), }) {} const github = makeHttpClient({ baseUrl: 'https://api.github.com', auth: () => ({ authorization: `Bearer ${process.env.GITHUB_TOKEN}`, accept: 'application/vnd.github+json', }), makeError: (a) => new GithubError(a), fetchImpl: fetch as unknown as FetchLike, }) // Each method returns Effect.Effect<…, GithubError>. const repo = yield* github.getJson('/repos/acme/widgets') const created = yield* github.postJson('/repos/acme/widgets/issues', { body: { title: 'bug', body: 'it broke' }, }) ``` What the core does for you: retries `408/425/429/5xx` with capped exponential backoff (jittered, honouring `Retry-After` within the `maxDelayMs` ceiling), fails a hanging request as transient after the timeout, pins every request to the base URL's host (an off-host or malformed URL is rejected **before** any fetch, fail-closed), maps a `401` to a non-transient `session_expired` (re-auth, never retried), and wraps each request in an `integration-http.request` span (method + host only — never a token). When the token is resolved per request (per-subject credentials), give the client a context type and pass `ctx` on each call: ```ts const client = makeHttpClient({ baseUrl: 'https://api.github.com', auth: (ctx) => ({ authorization: `Bearer ${ctx.token}` }), makeError: (a) => new GithubError(a), fetchImpl: fetch as unknown as FetchLike, }) const me = yield* client.getJson('/user', { ctx: { token: perSubjectToken } }) ``` > **Server-side only.** This module performs outbound network I/O — never > import it (or a module that imports it) from a browser-loaded query / > mutation / action / workflow descriptor. ## Related - [Schema mixins](/docs/database/mixins) — the OTHER plugin shape (schema-only, no runtime hooks). - [Auth strategies](/docs/authentication/strategies) — a specialised plugin pattern for identity providers. --- ## Analytics & warehouse sinks _AnalyticsSink contract — narrow cross-provider API for track / aggregate / timeseries / topN — plus the five first-party sink plugins (postgres-lite, DuckDB, ClickHouse, Tinybird, PostHog) and composeAnalytics for dual-write._ Voltro ships a narrow, cross-provider analytics contract (`AnalyticsSink`) and five first-party plugins that implement it. Mirror of the auth-strategy pattern: one typed interface, multiple providers, swap them in `app.config.ts` without touching handler code. The intent is **"go very far before you reach for an external tool"** — but make external tools a one-line install when you do. The first-party lite plugin (`postgres-analytics`) works on day 1 with zero external services and carries you up to ~10M events/day before queries get slow. ## The contract — `AnalyticsSink` Four methods. Effect-typed throughout. Every sink implements all four; sinks that genuinely can't support a capability return `AnalyticsCapabilityNotSupported` on the typed error channel rather than throwing. ```ts import { useAnalytics } from '@voltro/runtime' export default (input, ctx) => Effect.gen(function* () { const analytics = yield* useAnalytics() yield* analytics.track({ name: 'match_completed', subjectId: ctx.request.subject.id, properties: { mapId: 'dust2', durationSec: 1284, mvp: 'player_abc' }, }) // Total over a window const total = yield* analytics.aggregate({ event: 'match_completed', metric: 'count', range: { from: hoursAgo(24) }, }) // Time-bucketed series const daily = yield* analytics.timeseries({ event: 'match_completed', metric: 'count', bucket: 'day', range: { from: daysAgo(30) }, }) // Top-N by a property const topMaps = yield* analytics.topN({ event: 'match_completed', groupBy: 'mapId', metric: 'count', n: 10, range: { from: daysAgo(30) }, }) }) ``` The contract is deliberately narrow. **No raw SQL, no funnels, no cohorts, no custom dashboards.** Anything provider-specific lives outside the contract — call the provider's API/client directly where you need it (the plugins expose no raw-client escape hatch). ## Plugins at a glance | Plugin | Type | Best for | Ceiling | |---|---|---|---| | [`@voltro/plugin-analytics-postgres`](#voltroplugin-analytics-postgres) | Lite | Day-1 zero-setup, dev + early production | ~10M events/day | | [`@voltro/plugin-duckdb`](#voltroplugin-duckdb) | Embedded OLAP | Real column-store performance, no external service | Vertical scale: ~hundreds of GB in one process | | [`@voltro/plugin-clickhouse`](#voltroplugin-clickhouse) | External OLAP | Production-scale analytics, self-hosted or ClickHouse Cloud | Billions of events comfortably | | [`@voltro/plugin-tinybird`](#voltroplugin-tinybird) | Hosted ClickHouse | Pay-as-you-go without operating ClickHouse | Tinybird's own limits | | [`@voltro/plugin-posthog`](#voltroplugin-posthog) | Product analytics | Sessions, feature flags, funnels in PostHog's UI | `track()` only — compose with another sink for reads | ## Picking one Decision rubric in order: 1. **First service you don't want to operate?** → `postgres-analytics`. Works against the main DataStore. Cross-dialect. 2. **First-party experience but real OLAP performance?** → `duckdb`. Embedded sidecar, no external service to run. 3. **Past 10M events/day OR want billions-of-rows queries to stay under a second?** → `clickhouse` (self-hosted or Cloud) or `tinybird` (hosted). 4. **Already invested in PostHog for product analytics?** → `posthog` for ingest (alongside a primary sink for aggregates). Switching providers later is a one-line config change. Handlers stay identical because they consume the same `AnalyticsSink` contract. ## `@voltro/plugin-analytics-postgres` First-party lite. Stores events in `_voltro_events` on the main DataStore. Cross-dialect: works on postgres / mysql / mariadb / mssql / sqlite / turso via `sql.onDialectOrElse`. ```ts // app.config.ts import { postgresAnalytics } from '@voltro/plugin-analytics-postgres' export default defineApi({ name: 'myApi', store: 'postgres', analytics: postgresAnalytics(), }) ``` What the plugin owns: - The `_voltro_events` table — auto-created on first boot (numeric `id`, `name`, `subject_id`, `properties` JSONB, `occurred_at`). Two composite indexes (`name+occurred_at`, `subject_id+occurred_at`) for the hot read paths. - All four methods: `track` writes a row; `aggregate` / `timeseries` / `topN` compile to `date_trunc` / JSON-extract SQL that runs across all six SQL dialects. **Honest ceiling**: by ~10M events/day, time-range aggregates over 30 days take >5s on Postgres. The `_voltro_events` table is bounded by a retention sweep (`VOLTRO_EVENTS_TTL_HOURS`, default 365 days) so it never grows without limit; past ~10M events/day, swap to a real OLAP sink before queries get slow. **Tenant-scoped reads**: `useAnalytics()` stamps the caller's `subject.tenantId` onto every `track` and every `aggregate` / `timeseries` / `topN`, so a handler can't write or read across tenants. A system/background context with no subject gets the raw (cross-tenant) sink; set an explicit `tenantId` on the event/query only for a deliberate cross-tenant op. ## `@voltro/plugin-duckdb` Embedded DuckDB sidecar via `@duckdb/node-api`. Real column-store + vectorized execution, in-process — no external service. ```ts import { duckdbAnalytics } from '@voltro/plugin-duckdb' analytics: duckdbAnalytics({ path: '.voltro/analytics.duckdb', // or omit / ':memory:' for ephemeral }), ``` DuckDB is the quiet win for ~80% of growth-stage apps: real OLAP performance with zero external service to deploy. The plugin owns its own `voltro_events` table inside the DuckDB instance and queries against it. Scope: - **Events + opt-in CDC-mirror.** `track()` writes to DuckDB; queries read from DuckDB. Pass `mirrorTables` to stream the main DataStore's reactive-table changes into `voltro_mirror_` tables inside DuckDB so analytical queries can JOIN events against live user data (see [CDC-mirror](#cdc-mirror-of-reactive-tables) below). Without `mirrorTables` the sink is events-only. - **Single-process.** DuckDB can't open the same file from multiple workers. For multi-instance deployments either pin analytics traffic to one replica or use `clickhouse` instead. ## `@voltro/plugin-clickhouse` Production OLAP via the official `@clickhouse/client`. Self-hosted ClickHouse OR ClickHouse Cloud. ```ts import { clickhouseAnalytics } from '@voltro/plugin-clickhouse' analytics: clickhouseAnalytics({ url: process.env.CLICKHOUSE_URL!, database: 'voltro_events', username: process.env.CLICKHOUSE_USER, password: process.env.CLICKHOUSE_PASSWORD, }), ``` The plugin owns the events table schema (MergeTree engine, `ORDER BY (name, occurred_at, id)`, `LowCardinality(String)` on the event name, ZSTD-compressed properties JSON). First boot creates it; the plugin pings the cluster to fail-fast on bad config. **Opt-in batching.** By default every `track()` is one immediate HTTP insert. Pass `batch: { maxSize?, flushIntervalMs? }` to buffer rows and flush them in ONE multi-row insert by size (default 1000), on a timer (default 5000 ms), and on shutdown (a graceful drain before the client closes). This trades per-event delivery confirmation for far fewer round-trips under load — with batching a successful `track()` means "buffered", and a later flush failure is logged + the batch dropped (best-effort), so leave `batch` unset when you need per-event delivery confirmation. Retention (a `TTL` on the events table) stays the operator's job. There is no raw-client escape hatch: HyperLogLog, dictionaries, materialised views — the things you actually picked ClickHouse for — live outside the cross-provider contract, and the plugin exposes no handle to the raw `@clickhouse/client`. Where you need them, query ClickHouse with your own client instance against the same tables. Stay on `useAnalytics()` for code that should remain provider-portable. ## `@voltro/plugin-tinybird` Hosted ClickHouse via Tinybird's Events API + Pipes. ```ts import { tinybirdAnalytics } from '@voltro/plugin-tinybird' analytics: tinybirdAnalytics({ token: process.env.TINYBIRD_TOKEN!, region: 'eu', // 'eu' | 'us-east' | 'us-west' | 'asia-southeast' datasource: 'voltro_events', // pipes: { aggregate: 'my_agg', timeseries: 'my_ts', topN: 'my_top' }, }), ``` **The plugin expects three canonical pipes to exist in your workspace:** - `events_aggregate` — receives `event`, `from`, `to`, `metric_expr`, `filter_sql` - `events_timeseries` — same + `bucket_fn` - `events_topn` — same + `group_expr`, `limit` We don't synthesize pipes from the framework — Tinybird's `.pipe` DSL is too rich to generate from a generic spec. Push the three canonical pipes once via the `tb` CLI; override names with `pipes: { ... }` if your team uses a different convention. ## `@voltro/plugin-posthog` PostHog forwards `track()` only. Funnels / cohorts / sessions / feature flags live in PostHog's own UI and SQL — the plugin's `aggregate` / `timeseries` / `topN` return `AnalyticsCapabilityNotSupported` so callers fall back cleanly. ```ts import { posthogAnalytics } from '@voltro/plugin-posthog' analytics: posthogAnalytics({ apiKey: process.env.POSTHOG_KEY!, host: 'https://eu.posthog.com', // optional, default app.posthog.com }), ``` Typical wiring is **compose** with a primary sink that handles reads (see below) — PostHog mirrors every event for product-analytics insights while your primary sink owns the typed aggregate queries. ## composeAnalytics — multi-sink For dual-write (e.g. postgres-lite for typed reads + PostHog for product analytics): ```ts import { composeAnalytics } from '@voltro/runtime' import { postgresAnalytics } from '@voltro/plugin-analytics-postgres' import { posthogAnalytics } from '@voltro/plugin-posthog' export default defineApi({ name: 'myApi', store: 'postgres', analytics: composeAnalytics([ postgresAnalytics(), // primary — handles reads posthogAnalytics({ apiKey: process.env.POSTHOG_KEY! }), // mirrors every track() ]), }) ``` Semantics: - **`track()`** fans out to every sink in parallel. Failures are isolated per sink — one provider returning HTTP 503 doesn't fail the postgres insert. - **`aggregate` / `timeseries` / `topN`** query to the **first sink that supports the op**. PostHog doesn't support these → composition queries to the next sink (postgres-lite). - If no sink supports a read op, the composite returns `AnalyticsCapabilityNotSupported({ provider: 'compose' })`. ## Capability-not-supported error handling The cross-provider contract intentionally surfaces capability gaps on the typed error channel. Catch them when you want graceful fallback: ```ts import { Effect } from 'effect' const result = yield* analytics.aggregate({ event: 'match_completed', metric: 'count', range: { from: daysAgo(7) }, }).pipe( Effect.catchTag('AnalyticsCapabilityNotSupported', () => Effect.succeed(0)), Effect.catchTag('AnalyticsError', (err) => Effect.gen(function* () { yield* Effect.logWarning('analytics aggregate failed', { provider: err.provider, cause: err.cause }) return 0 })), ) ``` The two typed errors: - **`AnalyticsCapabilityNotSupported`** — the sink doesn't implement the operation (e.g. PostHog returning this for `aggregate`). - **`AnalyticsError`** — the underlying provider raised something (HTTP failure, query syntax, connection lost). ## CDC-mirror of reactive tables By default a sink stores only the events you `track()` — the main DataStore's reactive tables aren't in the warehouse, so analytical queries can't JOIN events against user data. Opt in with `mirrorTables`: the framework subscribes to the store's change stream and upserts/deletes the changed rows into a corresponding warehouse table, idempotently (keyed on the primary key). ```ts import { duckdbAnalytics } from '@voltro/plugin-duckdb' analytics: duckdbAnalytics({ path: '.voltro/analytics.duckdb', mirrorTables: ['users', 'teams'], // reactive tables to mirror // mirrorPrimaryKey: 'id', // default 'id' }), ``` The postgres-lite (`postgresAnalytics({ mirrorTables: [...] })`) and ClickHouse (`clickhouseAnalytics({ url, mirrorTables: [...] })`) sinks take the same options. Each mirrored table lands as `_voltro_mirror_
` (`voltro_mirror_
` on DuckDB / ClickHouse) holding `{ id, data, version, is_deleted }` — `id` is the source row's primary key, `data` is the full row as JSON, `version` orders the writes (see below) and `is_deleted` marks a tombstone. Analytical queries JOIN events against the mirror and filter tombstones out: ```sql -- DuckDB: events per user tier SELECT json_extract_string(m.data, '$.tier'), COUNT(*) FROM voltro_events e JOIN voltro_mirror_users m ON m.id = e.subject_id AND m.is_deleted = false GROUP BY 1 ``` The mirror is **opt-in** (omit `mirrorTables` → events-only) and **idempotent** — inserts/updates upsert by primary key, deletes write a tombstone by key, so a re-delivered change (e.g. after a reconnect) is a no-op-equivalent overwrite. ### Delivery guarantee **At-least-once for the lifetime of the process, ordered per row.** Read that sentence literally — every clause is a promise the framework keeps, and the sentence stops where the implementation does: - **Retried, not dropped.** A failing mirror write is retried with exponential backoff (`retryAttempts`, default 5). A write that outlives its retries is queued for **repair**: a timer re-reads the row's *current* state from your database and re-applies it, so the mirror converges on the truth rather than on a stale change that happened to be in flight. - **Ordered per row.** Writes for the same primary key are applied one at a time, in commit order, and every write carries a **version** stamped when the change left the store — never a clock read inside the warehouse client. A change that arrives late therefore *loses*: ClickHouse's `ReplacingMergeTree(version)` keeps the highest version, and the DuckDB / postgres mirrors apply the update only when the incoming version is newer. Different rows are still mirrored concurrently. - **A delete is a tombstone, not a row removal.** Filter `is_deleted = false` (`is_deleted = 0` / `FINAL` on ClickHouse). A physical delete would leave nothing for a late, stale insert of the same key to lose against — the row would silently come back. The tombstone also keeps its version, which is what lets a replacement replica seed a key's numbering after a delete. - **N replicas converge to ONE row per change.** Under postgres CDC / mysql binlog every replica observes every change and mirrors it, so a change is written N times — but the version is derived from the CHANGE (the warehouse's own high-water mark for the key, plus the change's position in the fleet stream), never from a replica's clock. The N duplicates are byte-identical, and the sink's version guard collapses them for free: no leader election, and no clock-skew window in which an older image could out-version a newer one. A replica that boots mid-stream reads each key's high-water mark from the warehouse before its first write, so it continues the fleet's numbering instead of restarting it. - **Never surfaces to the request.** The OLTP write that produced the change has already committed; a warehouse outage is isolated into the log channel and the metrics below. - **A graceful shutdown is not a crash.** On SIGINT / SIGTERM the mirror stops taking new changes and then **settles** what is already queued and in flight, before the sink itself is disposed. That drain is bounded (3 s of the teardown budget you set with `VOLTRO_SHUTDOWN_GRACE_MS`, default 10 s): a warehouse that has stopped answering cannot hold the process open until the orchestrator's SIGKILL, which would lose strictly more. The two outcomes log differently — `analytics mirror drained` at info, or a `warn` naming what was still pending when the deadline cut it, because that is the moment those counters can still be read. - **Not durable across a crash.** The repair queue lives in memory. A change still awaiting repair when the process dies — SIGKILL, an OOM, a host failure — is lost, as is one evicted after `repairQueueLimit`. Both are logged at error level and counted by `voltro_analytics_mirror_dropped_total` — if that counter is non-zero, the affected tables need a re-seed. Metrics: `voltro_analytics_mirror_forwarded_total`, `..._retries_total`, `..._repair_queued_total`, `..._dropped_total`. ### Tuning Every number the mirror picks on your behalf has a default and an environment override: | Env var | Default | Meaning | |---|---|---| | `VOLTRO_ANALYTICS_MIRROR_RETRY_ATTEMPTS` | `5` | Total attempts per mirror write (`1` = no retry). | | `VOLTRO_ANALYTICS_MIRROR_RETRY_BASE_MS` | `100` | First backoff delay; doubles per attempt. | | `VOLTRO_ANALYTICS_MIRROR_RETRY_MAX_MS` | `30000` | Ceiling for the doubling backoff. | | `VOLTRO_ANALYTICS_MIRROR_REPAIR_INTERVAL_MS` | `60000` | How often the repair loop re-drives exhausted changes (`0` disables it). | | `VOLTRO_ANALYTICS_MIRROR_REPAIR_QUEUE_LIMIT` | `10000` | Maximum keys held for repair before the oldest is dropped and counted. | | `VOLTRO_ANALYTICS_MIRROR_VERSION_STATE_LIMIT` | `100000` | Maximum keys whose fleet-scope version state stays in memory; an evicted key re-seeds from the warehouse on its next change. | ## Default — no sink configured If `app.config.ts` doesn't set `analytics`, the framework provides a no-op sink. `track()` calls drop silently (logged once at boot so operators notice); `aggregate` / `timeseries` / `topN` fail with `AnalyticsCapabilityNotSupported({ provider: 'noop' })`. Apps without analytics setup never crash on `useAnalytics().track(...)`. ## Anti-patterns - **Don't bypass `useAnalytics()` and call provider clients directly** unless you specifically need a provider-locked feature. Going through the contract keeps handler code provider-portable. - **Don't compose two reading-capable sinks expecting both to be queried.** Reads query to the first capable sink — that's by design (different sinks may hold different views of the data). If you genuinely need cross-sink reads, query each provider directly with its own client. - **Don't dump millions of properties per event.** Sink storage is JSON; cardinality at the property level slows JSON-extract queries. If you have unbounded dimensions, restructure into a separate events table that the analytical query reads through joins. - **Don't expect aggregate queries to reflect new tracks instantly on every sink.** By default ClickHouse and Tinybird ingest each `track()` as an immediate HTTP request, but the providers themselves make rows visible asynchronously (Tinybird's Events API acknowledges before the row is queryable). Two sinks add an opt-in client-side `batch` option (`clickhouseAnalytics({ batch })`, `posthogAnalytics({ batch })`) that buffers events and flushes them in one request by size / on a timer / on shutdown — a successful `track()` then means "buffered", not "delivered". Postgres-lite reflects writes immediately because they're synchronous to the main DB. ## Related - The cross-cutting [aggregate convention](/docs/data/aggregates) covers `*.aggregate.ts` files — pre-defined queries that materialise their result on a schedule. An aggregate's build function gets `ctx.analytics` (the configured sink) alongside `ctx.store`, so it can query the warehouse (`ctx.analytics.topN(...)`) for the "reduce 10B-row warehouse to a 100-row leaderboard" pattern. See the [cross-plan section](/docs/data/aggregates#cross-plan-querying-the-warehouse-from-a-build-function). - [`*.subscribe.ts`](/docs/data/subscribers) covers best-effort per-row reactive callbacks that fire after every commit. Different shape from `track()` (subscribe fires from CDC; track is explicit ingest).