# Webhooks > First-class outgoing + incoming webhooks — declared events with a webhook: block, defineIncomingWebhook in *.webhook.tsx files, runtime subscriptions, durable delivery via @effect/workflow, HMAC signing, provider presets, and idempotency. --- ## Webhooks _First-class outgoing + incoming webhooks — declared events with a webhook: block, defineIncomingWebhook in *.webhook.tsx files, runtime subscriptions, durable delivery via @effect/workflow, HMAC signing, provider presets, and idempotency._ `@voltro/plugin-webhooks` adds first-class outgoing **and** incoming webhooks. Both directions are declared in `*.webhook.tsx` files: outgoing events the app emits, and incoming HTTP endpoints the app exposes to third parties. Delivery is durable — each outgoing delivery runs as an `@effect/workflow`, so signing, retries, rate limits, and `Retry-After` honouring all happen inside a crash-safe workflow body. The framework discovers `*.webhook.tsx` files and wires the queries + codegen. The managed tables (`_voltro_webhook_targets`, `_voltro_webhook_deliveries`, and the rate-limit counter table `_voltro_webhook_rate_windows`) are created by the bootstrap migration; you can register them explicitly in your schema via the `webhookTables()` mixin from `@voltro/plugin-webhooks/mixin`. The targets + deliveries tables carry `tenant()` (the rate-window table holds only operational counters — no tenant data), so webhook subscriptions are **per-tenant**: a target subscribed under tenant A is invisible to tenant B, and `emit()` from a tenant fans out only to that tenant's targets. The runtime stamps `tenantId` from the caller's subject on subscribe and carries it through to delivery — you don't pass it by hand. ## Outgoing — a declared event with a `webhook:` block Declare an event the app emits. External systems subscribe to it at runtime via `ctx.webhooks.subscribe(...)`. Each subscribed target is a row in `_voltro_webhook_targets`; each delivery attempt is a row in `_voltro_webhook_deliveries`. ```ts // events/orders.event.ts import { defineEvent } from '@voltro/protocol' import { Schema } from 'effect' export const orderCompleted = defineEvent({ name: 'order.completed', key: Schema.Struct({}), payload: Schema.Struct({ orderId: Schema.String, tenantId: Schema.String, total: Schema.Number, items: Schema.Array(Schema.Struct({ sku: Schema.String, qty: Schema.Number })), }), webhook: { description: 'Order moved into "fulfilled" — payment captured, items shipped.', version: 1, }, }) ``` Recipients receive `{ event, eventId, occurredAt, payload }` with `payload` matching the schema — `emit()` decodes every payload against it and rejects a mismatch with the typed `WebhookPayloadInvalid` before any delivery is created. `version` defaults to `1`; increment it whenever the payload shape changes in a way subscribers must adapt to — new subscriptions pin to the event's current version. Optional `defaultRetry` and `defaultSigning` apply to new subscriptions when the subscriber doesn't pass `retry` / `signing` explicitly (explicit input wins, then the event default, then the package default). Optional `globalRateLimit: { perMinute }` caps ALL deliveries of this event combined: the delivery workflow claims a slot in a fixed one-minute window counter at the shared store (so the cap holds across replicas) before every POST; over-limit deliveries are deferred — parked as `status='pending'` rows and durable-slept until the next window opens — never dropped. ### Two audiences, one publish — and one direct door A declared event with a `webhook:` block has **two** audiences: in-app subscribers (`useEvent`) and subscribed HTTP targets. `ctx.events.publish(...)` reaches **both**, from one call, on the commit boundary: ```ts await ctx.events.publish(orderCompleted, {}, { orderId, tenantId, total, items }) ``` Inside a mutation the publish is deferred to commit, so a rollback discards every audience together — a partner is never told about an order the database rolled back. `ctx.webhooks.emit(descriptor, payload)` is the direct door, for an app that wants the HTTP audience *only* (no in-app subscribers, no workflow triggers). It is also post-commit inside a mutation, and it returns `{ deferred: true }` with an empty `deliveries` list in that case — the deliveries do not exist until the transaction commits, and an unmarked empty list would read as "no endpoint wanted it". **Prefer `publish`.** One declaration, every audience, and the boot's wiring check sees it. Reaching for `emit` because you assumed `publish` did not reach HTTP targets is a mistake a reporter made — they grepped `@voltro/runtime`, where the fan-out is not; it lives where `ctx.events` is assembled. ### The fan-out is tenant-scoped, and the filter is the *second* boundary `ctx.webhooks.emit` delivers only to targets of the acting subject's tenant. That was not true before: the service is built once at boot with the app-level store and no subject, so the `tenant()` mixin on `_voltro_webhook_targets` had nothing to scope by, and confinement rested entirely on each target's own `filter`. It looked safe because a filter usually predicates on a globally unique app id — a cross-tenant match was impossible *by accident*. It stops being accidental the moment you introduce a value deliberately equal across tenants. A **system** emit — a schedule, a replay, a startup task — has no tenant to be confined to and stays unscoped. Pass `{ tenantId }` explicitly to fan out across tenants from a request: a deliberate cross-tenant delivery has to say so. ```ts await ctx.webhooks.emit(orderCompleted, payload) // this tenant await ctx.webhooks.emit(orderCompleted, payload, { tenantId: null }) // every tenant, deliberately ``` ### Subscribing targets Subscribers register at runtime — from a settings page, an admin mutation, or a seed. Targets persist; the dashboard surfaces them under a "Webhooks" tab. ```ts // mutations/webhooks.subscribe.mutation.server.ts const execute = async (input: { url: string; secret?: string }, ctx: AppContext) => { const target = await ctx.webhooks.subscribe({ event: 'order.completed', url: input.url, // Optional — the server generates a 32-byte hex secret if absent. // The caller receives it ONCE in the response; subsequent reads // never expose it again. secret: input.secret, retry: { // overrides the event's defaultRetry strategy: 'exponential', maxAttempts: 8, initialDelay: '5s', maxDelay: '1h', retryOn: [408, 429, 500, 502, 503, 504], jitter: 'full', }, filter: { 'payload.total': { gt: 1000 } }, // optional predicate filter headers: { 'X-Tenant': 'acme' }, // optional per-target headers rateLimitPerMinute: 60, // optional — excess deliveries defer to the next minute window format: 'json', // optional wire format: 'json' (default) | 'form' | 'xml' autoDisableAfter: 20, // optional — auto-pause after N consecutive terminal failures (off by default) }) return { targetId: target.id, secret: target.secret } } ``` `rateLimitPerMinute` caps wire POSTs to this target (retries count too): the delivery workflow claims a fixed-window slot at the shared store before each POST and defers over-limit deliveries to the next window — queued as `status='pending'` rows, never dropped. #### Delivery format — `json` (default) · `form` · `xml` `format` decides the bytes the delivery workflow signs and POSTs. The producer side always stringifies the payload to JSON; the workflow **re-encodes** it into the target's format at delivery time and **signs the re-encoded bytes** (never the source JSON). If a format can't represent the payload shape the delivery is marked `failed` with a typed `WebhookPayloadUnrepresentable` reason *before* any wire POST — a mis-formatted target never silently ships JSON under a wrong content-type. - **`json`** (default) — `content-type: application/json`, the JSON string verbatim. Represents any JSON value. - **`form`** — `content-type: application/x-www-form-urlencoded`. The payload MUST be a JSON **object** (a top-level array/scalar is unrepresentable). Nested structure flattens to **bracketed keys** (the PHP/Rails convention): `{ a: 1, b: { c: 2 }, items: [{ sku: 'x' }, 'y'] }` → `a=1&b[c]=2&items[0][sku]=x&items[1]=y`. Leaf primitives become their string form; `null` becomes an empty value (`key=`); keys/values are percent-encoded. - **`xml`** — `content-type: application/xml`. The payload is wrapped in a single `` root: object keys → child elements (`…`), arrays → repeated `…` elements under the array's key, primitives → escaped text, `null` → an empty element. Prefixed with the XML declaration. An object key that isn't a valid XML element name is unrepresentable. #### Auto-disable — dead-letter guard for failing endpoints `autoDisableAfter` (off by default; a positive integer when set) auto-pauses a target after that many **consecutive terminal** delivery failures. The delivery workflow tracks a per-target streak at the shared store (multi-replica-correct): each terminal `failed` delivery increments it, any `succeeded` resets it to zero. When the streak reaches the threshold the target flips to `active:false` (stamped with `autoDisabledAt` + the failure reason, surfaced on the inspect panel) and a `warn` log fires. Auto-disable reuses the **same pause machinery** as a manual `pauseTarget`: while disabled, subsequent emits **queue** as `status='pending'` rows (nothing is dropped — see the pause/flush section below). A manual `resumeTarget` re-activates the target, **flushes** the accumulated queue, and **clears the streak** so the next failure starts a fresh count. ### The routing filter — all six operators ```ts filter: { 'payload.teamId': 'team_7' } // equality filter: { 'payload.teamId': { in: ['team_7', 'team_9'] } } // one of filter: { 'payload.total': { gte: 100, lt: 1000 } } // a range ``` Dotted paths INTO the emitted envelope, whose root is `{ payload }`. A bare value means equality; an object names an operator. `eq`, `in`, `gt`, `gte`, `lt`, `lte` — those six and nothing else. **An unsupported operator falls through to `===` and therefore matches nothing.** **This was typed `Readonly>` and it cost a deployment a feature for a year.** They wrote in a code comment that the filter was key-path equality and could not express "id is one of these", refused that capability in their own API with a typed error, and shipped it — while `in` had been supported the whole time. One grep would have corrected it; the type did not. It is the one place where being wrong is silent in **both** directions: a predicate matching nothing reads as "no endpoint wanted it", and one matching everything reads as working. A path that does not resolve is `undefined`, so a typo routes nothing, forever, without an error. **Operators inside one object are ANDed, and the range above did not used to be.** The matcher returned on the first operator it found, so `{ gte: 100, lt: 1000 }` evaluated `gte` and ignored `lt` — a `total` of 5000 satisfied a filter declared as 100–1000. That is the worse direction of wrong: an under-matching filter delivers nothing and gets noticed, an over-matching one posts a partner data their own filter says they must not receive, and nothing anywhere reports it. Fixed in 0.31.0; the example on this page is the one that was broken. **`gt` / `gte` / `lt` / `lte` compare strings too**, lexicographically — which is what makes an ISO timestamp work: ```ts filter: { 'payload.at': { lt: '2026-01-01' } } // before a date ``` They are typed `number | string` and used to compare only when both sides were numbers, so a string bound type-checked, subscribed, stored, and matched nothing. Mixed types (a number bound against a string value) still match nothing, deliberately: `'10' < 9` depends on which side JavaScript converts, and a routing rule resting on that is worse than one that does not fire. An operator object with **no** recognised key — `{ gtE: 5 }`, or `{}` — matches nothing rather than everything, for the same reason a typo'd path does. ### Emitting Emit from any handler that imports the plugin: ```ts import { useWebhooks } from '@voltro/plugin-webhooks' const execute = async (input: { orderId: string }, ctx: AppContext) => { // ... do the mutation ... const webhooks = useWebhooks(ctx) await webhooks.emit('order.completed', { orderId: newOrder.id, tenantId: newOrder.tenantId, total: newOrder.total, items: newOrder.items, }) } ``` `useWebhooks(ctx)` returns the typed `WebhooksServiceShape` the framework attached to `AppContext`; it throws a clear message if the plugin isn't active. `emit()` accepts the event id or the declared event itself — `emit(orderCompleted, payload)` types `payload` against the event schema at the call site. Either way the payload is decoded against the schema and a mismatch throws `WebhookPayloadInvalid`. It returns `{ eventId, deliveries }` — one entry per matched target, each `'dispatched'` (workflow kicked off) or `'queued'` (the target is paused — see below), for log correlation. The actual HTTP POSTs happen in the background delivery workflow; the call returns as soon as the workflows are kicked off, not when they complete. The service also exposes `replay(deliveryId)`, `listTargets(event?)`, `pauseTarget` / `resumeTarget`, `deleteTarget`, `rotateSecret`, and `updateTargetPayloadVersion` — consumed by the dashboard's Webhooks panel. While a target is **paused** — whether by a manual `pauseTarget` or by the `autoDisableAfter` dead-letter guard above — emits against it queue as `status='pending'` rows in `_voltro_webhook_deliveries` instead of POSTing — nothing is dropped. `resumeTarget` re-activates the target and flushes its queue through the normal delivery workflow in emit order (`createdAt` ascending, millisecond granularity) per target, and clears any auto-disable streak. `deleteTarget` removes the target's queued pending rows along with it. An auto-disabled target is distinguishable from a manually-paused one by its non-null `autoDisabledAt` / `autoDisableReason`. ### Typed errors — `@voltro/plugin-webhooks/errors` The outgoing `WebhooksService` ops throw `Schema.TaggedError`s, not bare `Error`s — catch them with `Effect.catchTag` server-side, or declare them on a mutation/action descriptor's `error:` union so the client decodes them typed: - **`WebhookSubscribeInvalid`** (`{ field, reason }`) — `subscribe` input failed a policy check (a non-http(s) URL, a secret below the minimum length, a degenerate retry policy, or an out-of-range rate limit). - **`WebhookDeliveryNotFound`** (`{ deliveryId }`) — `replay` found no delivery row for that id. - **`WebhookPayloadVersionInvalid`** (`{ version }`) — `updateTargetPayloadVersion` got a non-positive-integer version. - **`WebhookPayloadInvalid`** (`{ event, issues }`) — `emit` got a payload that doesn't decode against the event's `payload` schema; rejected before any delivery row or workflow run exists. - **`WebhookPayloadUnrepresentable`** (`{ format, reason }`) — the target's `form`/`xml` encoder can't represent the payload shape (e.g. a `form` target whose payload is a top-level array/scalar, or an `xml` object key that isn't a valid element name). The delivery is marked `failed` with this reason before any wire POST. Import these from the **browser-safe** `@voltro/plugin-webhooks/errors` subpath when a descriptor (`*.mutation.ts` / `*.action.ts`) names one in its `error:` schema. The package root re-exports them too but is server-only (it also re-exports `node:crypto` / `@voltro/database` modules), so a descriptor importing from the root would pull server code into the browser bundle — the same rule as `@voltro/plugin-multitenancy/guard`. ```ts // mutations/webhooks.subscribe.mutation.ts — DESCRIPTOR (browser-safe) import { defineMutation } from '@voltro/protocol' import { WebhookSubscribeInvalid } from '@voltro/plugin-webhooks/errors' import { Schema } from 'effect' export const subscribeWebhook = defineMutation({ name: 'webhooks.subscribe', // Registers a URL this server will POST your events to — never openAccess. guards: [{ scope: 'webhooks:manage' }], input: Schema.Struct({ url: Schema.String }), output: Schema.Struct({ targetId: Schema.String }), error: WebhookSubscribeInvalid, }) ``` ## Incoming — `defineIncomingWebhook(...)` Declare a public HTTP endpoint the app exposes to receive webhooks from a third party (Stripe, GitHub, Slack, a custom partner). The framework's incoming middleware verifies the signature, decodes the body, claims an idempotency key, and only then calls your handler. ```ts // webhooks/stripe.webhook.tsx import { defineIncomingWebhook } from '@voltro/plugin-webhooks' import { stripeProvider } from '@voltro/plugin-webhooks/providers' import { Schema } from 'effect' export default defineIncomingWebhook({ id: 'stripe', provider: stripeProvider(), payload: Schema.Struct({ id: Schema.String, type: Schema.String, // Stripe payloads vary by event type — narrow per handler via ctx.body.type. data: Schema.Any, }), handler: async (ctx) => { // ctx.body is the validated payload. // ctx.idempotencyKey is the Stripe event id (from the provider preset). // ctx.headers carries the original request headers (lowercased keys). // ctx.rawBody is the raw bytes, if you need to recompute a MAC. if (ctx.body.type === 'invoice.payment_succeeded') { await store.update('invoices', ctx.body.data.object.metadata.invoiceId, { paidAt: new Date() }) } }, }) ``` The endpoint mounts at `/webhooks/` by default; override with `path: '/integrations/stripe/v1'`. The signing secret comes from the env var `VOLTRO_WEBHOOK_SECRET_` — for `id: 'stripe'`, that's `VOLTRO_WEBHOOK_SECRET_STRIPE` (non-alphanumeric characters in the id become `_`). ### Verification is not optional An incoming webhook is a **public POST that runs your application code**, so the framework refuses to mount one that has made no decision about who may call it. A descriptor with no `signature` and no `provider` fails the boot, naming the endpoint. Four ways to satisfy it: | Declaration | Means | |---|---| | `provider: stripeProvider()` | a preset brings the scheme, replay window and idempotency key | | `signature: { _tag: 'hmac', … }` | a hand-declared scheme for a sender with its own convention | | `verification: 'provider'` | your handler verifies with the provider's own SDK | | `verification: 'none'` | deliberately public — a gateway or IP allow-list owns the boundary | ```ts export default defineIncomingWebhook({ id: 'internal.reindex', verification: 'none', // behind the cluster gateway; nothing else may reach it payload: Schema.Struct({ index: Schema.String }), handler: async (ctx) => { /* … */ }, }) ``` `verification: 'none'` logs a warning at every boot, on purpose — an open endpoint should stay visible. **A declared signature with no configured secret answers 503**, on every delivery, naming the variable to set. It does not skip the check: that fallback meant one missing env var silently turned a verified webhook into an open one. The framework never generates the secret — the sender holds the other half of it. ### Provider presets Built-in presets configure the signature scheme, idempotency extraction, and body type for the common integrations. Import them from `@voltro/plugin-webhooks/providers`: | Preset | Signature header | Idempotency key | |---------------------|-------------------------|----------------------------| | `stripeProvider()` | `Stripe-Signature` | `body.id` | | `githubProvider()` | `X-Hub-Signature-256` | `X-GitHub-Delivery` header | | `slackProvider()` | `X-Slack-Signature` | `X-Slack-Request-Timestamp`| | `genericProvider()` | `X-Webhook-Signature` | `Idempotency-Key` header | Explicit fields on `defineIncomingWebhook` always override the preset. ### Custom providers — `defineWebhookProvider(...)` For partners not covered by the built-ins, define your own preset in a `*.webhook.tsx` file: ```ts import { defineWebhookProvider } from '@voltro/plugin-webhooks' export default defineWebhookProvider({ id: 'acme.custom', name: 'Acme Custom', signature: { _tag: 'hmac', algorithm: 'hmacSha256', header: 'X-Acme-Sig', includeTimestamp: true, replayWindowSeconds: 600, encoding: 'hex', versionPrefix: 'v1=', }, idempotency: { from: 'x-acme-delivery-id', ttl: '7d' }, bodyType: 'json', }) ``` Then reference it from another `*.webhook.tsx`: ```ts import { defineIncomingWebhook } from '@voltro/plugin-webhooks' import acmeProvider from '../providers/acme.webhook' import { Schema } from 'effect' export default defineIncomingWebhook({ id: 'acme', provider: acmeProvider, payload: Schema.Struct({ id: Schema.String }), handler: async (ctx) => { /* ... */ }, }) ``` ## Signing — Standard Webhooks v1.0.0 Outbound deliveries are signed to the [Standard Webhooks](https://github.com/standard-webhooks/standard-webhooks) spec by default. That is the point of a spec: every conformant consumer library verifies your deliveries with no code specific to you. ```http POST /your/hook webhook-id: 3f6a… ← the delivery id — USE THIS AS THE IDEMPOTENCY KEY webhook-timestamp: 1786503000 ← unix seconds webhook-signature: v1,K5oZfzN95Z…= ← base64 HMAC-SHA256, space-delimited during a rotation x-voltro-event: orders.paid x-voltro-attempt: 1 content-type: application/json ``` The signed content is `..`. The key is `whsec_` + base64 of 24–64 random bytes, and the HMAC runs over the **decoded** bytes — `subscribe()` mints the right shape automatically. Passing your own hex secret for a spec-signed target is REFUSED at subscribe rather than producing signatures no consumer accepts: a hex string is also valid base64, so a lenient decoder would key the HMAC on bytes that round-trip against itself and fail against everyone else. Implemented: the symmetric `v1` scheme, multi-signature rotation, the `.`-delimiter constraint on the id (asserted, not assumed), a constant-time compare, a 300s replay tolerance (the spec requires a tolerance and names no number — this one is ours, and tunable). NOT implemented: the asymmetric half (ed25519 / `v1a` / `whsk_`). A `v1a`-only signature is rejected by name rather than reported as a generic mismatch. Other schemes stay available as explicit choices — `genericHmacSignature()` (the previous house format, `X-Webhook-Signature: t=…,v1=`), `stripeSignature()`, `githubSignature()`, `slackSignature()`: ```ts subscribe({ event: 'orders.paid', url, signing: genericHmacSignature() }) ``` ### Delivery semantics the spec dictates | response | what happens | |---|---| | `2xx` | success | | `3xx` | **failure. The redirect is not followed** — the target URL is the one we validated, and following one walks past that check | | `410 Gone` | the target is disabled **immediately**, whatever `autoDisableAfter` says. The receiver answered the question | | `429` / `5xx` | retried with backoff; `Retry-After` is honoured | Each attempt has a **30s wire timeout** (`VOLTRO_WEBHOOK_TIMEOUT_MS`, or `timeoutMs` on the delivery workflow) — the top of the spec's recommended 15–30s band. ## `voltro webhooks consumer` — the package your subscribers install Your app already knows every event a subscriber can register for, every payload's shape, and the exact scheme it signs with. So generate the verification package rather than making the receiving team write it: ``` voltro webhooks consumer --out ../partner-sdk voltro webhooks events --json # what a subscriber can register for ``` It emits `index.js` + `index.d.ts` + `package.json` + a README, with **no runtime dependencies at all** — it imports `node:crypto` and nothing else. That constraint is the design, not an optimisation: the package runs in your subscriber's service, which is usually a different codebase and often not a Voltro app, and a dependency list is where "npm i and paste this in" stops being true. ```js import { createVerifier, WebhookVerificationError } from 'acme-webhooks' const verifier = createVerifier({ secret: process.env.WEBHOOK_SECRET }) app.post('/webhooks/acme', (req, res) => { let delivery try { delivery = verifier.verify(req.rawBody, req.headers) // ← RAW bytes } catch (err) { if (err instanceof WebhookVerificationError) return res.status(401).send(err.reason) throw err } if (alreadyProcessed(delivery.id)) return res.sendStatus(200) // webhook-id switch (delivery.event) { /* … */ } res.sendStatus(200) }) ``` Notes worth knowing before you hand it over: - **The raw body is the whole trap.** The signature covers the exact bytes that arrived; a JSON body-parser re-serialises and the signature then never matches. The generated README carries the per-framework recipe (Express / Fastify / Next / Hono). - **Node 18+**, deliberately. Web Crypto's HMAC is async, which would make `verify()` return a Promise and force every handler using it to be async too. - **Payload types only.** They are generated from each event's Schema; no decoder ships. The types describe what you send — the signature is what proves you sent it. - Pass an array as `secret` during a rotation; each is tried. ## Managing targets ```ts // Scope a target to your own dimension — opaque to the framework. subscribe({ event: 'order.completed', url, scope: { teamId: 'q970abc' } }) listTargets('order.completed', { teamId: 'q970abc' }) // Edit in place. Absent key = leave alone; explicit null = clear. updateTarget(id, { url: 'https://new.example/hook', headers: null }) // Send one delivery to ONE target — the first button in every webhook UI. testTarget(id) testTarget(id, { hello: 'world' }) // Read the delivery log without touching the framework table. listDeliveries({ targetId, status: 'failed', since, limit: 50 }) getDelivery(id) // adds payload + responseBody ``` **`scope` is yours and opaque.** The framework stores and returns it verbatim and never interprets it; `listTargets` filters on equality against the keys you pass, so a target scoped `{ teamId, projectId }` still matches `{ teamId }`. It exists because `.with(tenant())` is one level too coarse for most deployments — endpoints are commonly scoped to a team, a project or a workspace, and a tenant has many of those. **`updateTarget` writes only the keys present.** That distinction is the point: without it, "I did not mention headers" and "delete the headers" are the same call, which is why editing a URL used to mean delete + re-subscribe — rotating the secret and orphaning the delivery history. `event` and `secret` stay unpatchable: a different event is a different subscription, and the secret has `rotateSecret`. **`testTarget` bypasses fan-out and the filter, but not `active`.** A filter excluding your probe payload would make a healthy endpoint look dead; a paused target, on the other hand, queues exactly as an emit would, so the test tells the truth about what production will do. **`listDeliveries` omits `payload` and `responseBody`** so a list view over 200 rows does not pull every response body — `getDelivery` adds them. Timestamps come back as ISO strings whatever the dialect returned, and an unparseable payload is returned verbatim rather than throwing: a management view has to render a malformed row, not 500 on it. ## `emit` inside a mutation rides the commit — durably An emit inside a mutation does not POST when you await it. It writes a **transactional outbox row**, and the delivery goes out after the transaction commits. A mutation that emits and then throws rolls its rows back and no request ever leaves — the subscriber is never told about a change that did not happen. ```ts await ctx.store.insert(order) await ctx.webhooks.emit(orderCreated, payload) throw new Error('boom') // rows rolled back, outbox row rolled back with them, no POST ``` The enqueue writes through `ctx.store`, which inside a mutation *is* the transactional view — so the intent to deliver commits with your data or not at all. There is no window in which the row landed and the delivery was lost, and a crash between commit and dispatch is a retry rather than a loss. Delivery itself stays at-least-once, which is the strongest guarantee available without distributed transactions into the receiver: **your receiver must be idempotent**, and the signature it verifies makes a duplicate cheap to recognise. Two consequences worth knowing before you meet them: - **The result carries no deliveries.** A deferred emit returns `{ event, deliveries: [], deferred: true }` — the rows do not exist yet and cannot, since the emit may still be rolled back. If you want the subscriber count, ask `listTargets(event)`; that is a question about configuration, not about this delivery. - **Outside a mutation nothing defers.** A query, an action, a schedule or a workflow step has no transaction to ride, so the emit dispatches immediately. That is correct rather than a gap. For the cases that genuinely want the request on the wire now — a diagnostic ping, an emit whose receiver you are about to poll — say so: ```ts await ctx.webhooks.emit(pingEvent, payload, { immediate: true }) ``` This does **not** make the emit safe: a rollback after it still tells a subscriber about a change that did not happen. The point of the flag is that the trade is written at the call site. The deferral is durable whether or not your app declares an `*.outbox.ts` of its own — the framework registers its own delivery handler for webhook emits, so `_voltro_outbox` is a framework table every app carries. The in-memory after-commit callback is only what an app WITHOUT a webhook surface would have used, and such an app has no emit to defer. ## "Subscribed, never emitted" — the two columns, and why there are two The Webhooks panel's Events tab reports **delivery** and **emit** separately, because one cannot answer for the other. `everDelivered: false` conflates three different facts: 1. no `emit(...)` call site exists, or none ever ran — **the defect** 2. it ran, but nobody was subscribed yet 3. it ran, but every target's filter excluded the payload, or every target was paused Only the first is a bug. It is also the one that is expensive to find by hand: a team shipped a create dialog offering eleven event checkboxes of which four were wired, and ticking an unwired one returned 200, showed the endpoint enabled and healthy, and delivered nothing, forever. So `_voltro_webhook_event_stats` records every emit **regardless of whether any target matched** — the axis delivery history structurally cannot see. Their disagreement is the useful signal: *emitted, never delivered* means every target is paused, filtered out, or failing. The table is deliberately not tenant-scoped (the question is whether the code has a live call site, not whether one tenant has triggered it) and deliberately not retention-swept (an event that fires quarterly must not read as dead). An unreadable stats table renders as **unknown**, never as "never" — "we did not look" and "it never fired" are different answers, and only one of them is a finding. ## Failure semantics - **Signature mismatch** → `401`, handler never runs. - **Schema validation failure** → `422`, handler never runs. - **Duplicate idempotency key** (within TTL) → `200` + `{ duplicate: true }`, handler never runs. - **Concurrent in-flight delivery** with the same idempotency key → `409`, the provider retries later. - **Handler exception** → `500`, the idempotency claim is released so the provider's retry can re-process. Common providers (Stripe, GitHub, Slack) retry `5xx` automatically. ## A subscription is a set of events ```ts // One URL, a list of events — one subscription, N rows, ONE secret. subscribe({ events: ['user.created', 'user.updated'], url, scope: { teamId } }) // Address the GROUP, not the rows. pauseTarget({ scope: { teamId } }) updateTarget({ scope: { teamId } }, { url: 'https://new.example/hook' }) rotateSecret({ scope: { teamId } }) // → one new secret for every row listDeliveries({ scope: { teamId } }) // → the endpoint's whole history ``` A row is one event, but a subscription — as every webhook UI models it, ours included — is one URL with a list of event checkboxes. `events` creates the rows in one call, and a **`{ scope }` selector addresses them as a group** anywhere a target id is accepted. **The shared secret is the reason this is correctness, not ergonomics.** The receiver verifies one signature for one URL, so N rows for one endpoint must sign identically. `subscribe` mints ONE secret for the whole set, and `rotateSecret({ scope })` rotates every row to the same new value — which also replaces the delete-and-re-subscribe that used to be the only way, minting new target ids and orphaning the delivery history. `secret` is deliberately **not** patchable via `updateTarget`. Making a live credential app-writable would close the same gap by weakening the invariant that it is surfaced once and never again. A scope that matches no row is an error, not a no-op: "pause the endpoint" that pauses nothing and reports success is the failure this selector exists to avoid. **Adding an event to an existing endpoint inherits its secret.** Pass the same `scope` and `subscribe` reuses the group's secret instead of minting a new one — the last reason an app had to read a plugin column. It refuses a scope whose rows do not all share one secret: that is not one endpoint, and signing it as one would re-sign half a group with a key the receiver does not hold. ```ts await ctx.webhooks.subscribe({ url, scope, events: ['user.deleted'] }) // same secret ``` ## Anti-patterns - **Inventing an `events.emit` accessor or a plugin-list entry to enable webhooks.** Neither exists. Emit via `useWebhooks(ctx).emit(...)`; declare events and endpoints in `*.webhook.tsx` files (the framework discovers them). - **Trusting webhook input without signature verification.** The framework rejects unsigned incoming requests by default. If you genuinely need to accept unsigned traffic, set `signature: undefined` explicitly and gate the query at the network boundary (IP allow-list, VPC peering). - **Mutating domain state synchronously in the outgoing emit path.** Outbound delivery is async by design. Mutate state in your mutation, THEN emit. - **Using the same idempotency-key TTL as the provider's retry window.** Pick a TTL of at least 2× the provider's max retry window so the dedup catches the slowest retry.