# Interoperability contracts An **interoperability contract** is the transformation step between a raw inbound row (CSV, JSON, etc.) and a typed datalake resource. Each contract carries: - a `resource_type` (the target shape — `patient`, `customer`, `payment_account`, `legal_entity`, `generic_table`, etc.) - a `template_config` (the Liquid template that renders the inbound row into the target shape) - an `mdm_input_config` (the Liquid template that renders the matching subject identity for entity resolution — see `mdm.md` for the full MDM surface, including the `api.mdm.verify` verb invoked at workflow action time) - an optional `filter_template` (a Liquid pre-filter that decides whether a row enters the pipeline at all) SDK namespace: `api.interoperabilityContracts`. The contract is **Datalake-DB-resident** — `create()` requires the parent datalake at `status: 'ready'` (see `datalakes.md` §5). ## 1. Wire shape ```typescript import type { InteroperabilityContractRequestWritable, InteroperabilityContractResponse, } from '@alvera-ai/platform-sdk' const { data: created } = await api.interoperabilityContracts.create( tenantSlug, datalakeSlug, { name: 'Inbound CRM → Legal Entity', description: 'Vendor CRM rows → the legal entity they identify', resource_type: 'legal_entity', filter_template: undefined, // optional template_config: { type: 'custom', body: liquidBody }, // The template IS the subject here, so no entity resolution // runs. A contract that writes a generic table instead pairs // with this one and carries a real mdm_input_config emitting // the SAME (uri, external_id) — see §2 "The contract pair". mdm_input_config: { type: 'null' }, generic_table_id: null, // see §2 }, ) // created.id, created.slug — server-derived // created.type — mirrors template_config.type // created.resource_type === body.resource_type ``` Both polymorphic embeds use the SAME shared `TemplateConfig` shape with discriminator `type`: ``` template_config.type 'system' | 'custom' | 'identity' | 'null' mdm_input_config.type 'system' | 'custom' | 'identity' | 'null' ``` Default for both is `'null'`. The wire accepts all four values on each surface; semantic narrowing is a runtime concern, not a schema one: - `custom` — caller-supplied `body` string (Liquid template). Pair with `body`; leave `path` null. - `system` — references a platform-shipped template by filesystem path. Pair with `path` (required); leave `body` null. See `templates.md`. - `identity` — pass-through. The inbound row is forwarded unchanged. Used by the auto-created identity contracts for generic tables. Leave both `body` and `path` null. - `null` — disabled / "row IS the subject" on the mdm surface. Leave both `body` and `path` null. ### TemplateConfig embed fields Both `template_config` and `mdm_input_config` are instances of the shared `TemplateConfig` embed: ``` type required enum — 'system' | 'custom' | 'identity' | 'null' (default 'null') body optional string — Liquid template body; REQUIRED iff type === 'custom' path optional string — filesystem path; REQUIRED iff type === 'system' output_schema server-injected — JSON Schema { type: 'object' } for this surface; not caller-set ``` Server validation enforces the body/path pairing — supplying `body` with `type: 'system'`, or `path` with `type: 'custom'`, returns 422. ## 2. Rules the type cannot encode ### `generic_table_id` is conditionally required When `resource_type === 'generic_table'`, the body MUST include a non-null `generic_table_id` identifying the target table. For every other `resource_type`, the field must be `null`. The TypeScript type permits either value; the validator and server enforce the conditional rule. ### `resource_type` is a closed platform set GH-859 removed the per-industry vocabularies. Every datalake now accepts the same set: ``` action_log beneficial_owner document generic_table legal_entity message ``` The subject-specific types an industry used to bring — `patient`, `customer`, `payment_account`, `compliance_screening` — are gone. Model those as a **generic table** and pair it with a `legal_entity` contract (see "The contract pair" below). On the wire `resource_type` is a plain `string`; the set is enforced server-side, so an invalid value is a `/resource_type` 422 at create rather than a compile error. `datalakes.md` §5 `.systemDatasets` remains the runtime authority. ### The contract pair One inbound row usually becomes two things, so it needs two contracts: | | `resource_type` | `mdm_input_config` | writes | |---|---|---|---| | **A** | `legal_entity` | `{ type: 'null' }` | the subject | | **B** | `generic_table` (+ `generic_table_id`) | a real template | the row | Both emit the **same** `(uri, external_id)` identifier pair, so they converge on one legal entity and the platform stamps `legal_entity_id` onto the generic-table row. **Never declare `legal_entity_id` as a column on that table.** The name is reserved, so declaring it does not shadow the stamp — the create is REJECTED with 422 (`contain reserved system column names: legal_entity_id`). You lose the table, not the linkage. Drop it from your definition; the platform adds the column itself and you read the stamp back off the row. ### `filter_template` has INVERTED semantics This is the single highest-value gotcha on this surface. A `filter_template` is a Liquid template that decides whether a row enters the pipeline: - **Empty render → PASS** (row enters the pipeline). - **Non-empty render → SKIP** (row is dropped). This is the opposite of most filter DSLs ("truthy = keep"). A correctly-shaped filter looks like: ```liquid {% if msg.row.should_drop == 'Y' %}drop{% endif %} ``` Renders `"drop"` when the condition fires (SKIP); renders empty otherwise (PASS). Inverting `{% unless %}...{% endunless %}` is equivalent and stylistically common — but always anchor on the empty-render-passes rule, not on the Liquid keyword. If you omit `filter_template`, every row passes through. ### `template_config` body is Liquid, with row data on `msg` Templates receive the inbound row on the `msg` variable. The exact accessor depends on the source shape — CSV-shaped sources typically use `msg.` (with `msg.row.` available for nested cases); JSON-shaped sources mirror the inbound JSON under `msg.`. The render output is JSON — the platform parses the rendered string as JSON and treats it as the target resource body. A render that produces invalid JSON fails at sandbox-run with an HTTP 422 (`title: 'transform_failed'`, surfaced as an `AlveraApiError`) — not a `stage` value (see §5). Always sandbox-run a contract before binding it to a live data activation client. ## 3. Field ownership **Server-derived (Response-only).** Universal set from `type_naming.md`, plus: ``` type string — mirrors template_config.type at create time ``` **Caller-supplied (round-trip).** ``` name required string — agent-facing label description optional string — narrative resource_type required string — target shape (closed set, §2) filter_template optional string — Liquid pre-filter (inverted semantics) template_config required embed — { type, body?, path? } mdm_input_config optional embed — { type, body?, path? }; omit equals { type: 'null' } generic_table_id required UUID|null — required iff resource_type='generic_table' ``` **Write-only (Request-only).** None — every caller-supplied field round-trips on the response. ### What `mdm_input_config` must render — and the field that is easy to miss The body renders a JSON object describing the **subject** the row belongs to, so MDM can resolve it. The one part with a hard requirement that is not obvious from the shape is `identifiers`: ``` identifiers[] uri REQUIRED — the identifier's namespace type REQUIRED — what kind of identifier it is value REQUIRED — the identifier itself system accepted, and NORMALISES ONTO `uri` country optional — issuing country ``` **`type` is required and has no default.** An identifier with no type cannot be matched against anything, so MDM rejects the whole row: ``` MDM input validation failed reject_reason: %{identifiers: [%{type: ["can't be blank"]}]} ``` That rejection fails the contract at the `:mdm` stage, which means **the row never persists and the ingest looks silently empty** — you poll the dataset table and see zero rows, with nothing in the response to say why. Read the contract-stage failure to see the reason. `system` is accepted as an alias and normalises onto `uri`, so `{system, type, value}` and `{uri, type, value}` are both fine. What is never fine is omitting `type`. A worked example, from a green cookbook: ```liquid { "legal_entity_type": "individual", "identifiers": [ {"system": "{{ p.source_uri }}", "value": "{{ handle }}", "type": "digital_identifier"} ] } ``` **Do not confuse this with `identifications`,** which some `legal_entity` templates render into the record itself. That is a different embed with different keys (`id_type`, `uri`, `id_number`) — the two look alike and are not interchangeable. Two related shapes worth keeping straight: - **`mdm.verify`** (see `mdm.md`) asks "is this really you?" about an already-known subject. Its identifier entries carry `system` and `value`, both required, with `id_type` and `country` optional — the same on every datalake. Its example is not a template for this one. ## 4. Error envelopes Standard JSON:API envelopes per `errors.md`. Common rejections: | `source.pointer` | Cause | |------------------------------|--------------------------------------------------| | `/resource_type` | value not valid for the parent datalake's domain | | `/template_config/type` | enum mismatch | | `/template_config/body` | empty when `type === 'custom'` | | `/mdm_input_config/type` | enum mismatch | | `identifiers[].type` | blank — see §3; fails at the `:mdm` stage at INGEST time, not at create, so the contract saves fine and rows silently never land | | `/generic_table_id` | missing when resource_type='generic_table', | | | or non-null when not | | `/name` | uniqueness within datalake | | `errors.base` | "System-created contracts cannot be edited" (PUT) or "...deleted" (DELETE) — see §5 | ## 5. Lifecycle ### Create Synchronous. The create response is immediately usable — no async deployment phase. Templates are compiled lazily at sandbox-run time and at activation-client fan-out time, so a `body` containing invalid Liquid syntax will surface at first use, not at create. Always sandbox-run before binding to a live data activation client. ### Sandbox run — `.run(slug, msg)` The contract's templates can be exercised against a sample row WITHOUT persisting anything. `.run` executes only the RENDER prefix of the ingestion pipeline — `run_filter → run_transform → render_mdm_input` — and returns those outputs. It deliberately stops BEFORE the pipeline's persisting tail (`mdm.resolve → upsert_into_datalake → persist_carried_document`), so no row lands in the datalake and there is nothing to search afterwards. The `upsert_into_datalake` step (the DB insert, routed by the contract's `resource_type` destination) runs only when the DAC worker ingests a row — i.e. on `dataActivationClients.ingest`, not on `.run`. To prove a row *landed*, ingest and then read the run's log rows for its `batch_id` (see `data_activation_clients.md` §6.6). ```typescript const { data: result } = await api.interoperabilityContracts.run( tenantSlug, datalakeSlug, contractSlug, { /* sample row, same shape as inbound CSV/JSON */ }, ) // result.stage === 'completed' | 'filtered' // 'completed' → passed filter + transform + mdm_input // 'filtered' → skipped by filter_template (transform/mdm_input not run) // result.filter_result === 'pass' | 'skip' // result.transformed object | null — rendered template_config body // (null when stage === 'filtered') // result.mdm_input object | null — rendered mdm_input_config body // (null when filtered, or when mdm_input_config // is null / identity) // A transform or mdm_input render error is NOT a stage — it returns // HTTP 422 with an errors[] envelope (title 'transform_failed' / // 'mdm_input_render_failed'), surfaced as an AlveraApiError. ``` Use this verb as a CI / pre-bind verification step: produce a representative sample row, run it through `.run`, and assert `stage === 'completed'`, `filter_result === 'pass'`, and that the `transformed` (and, if you resolve MDM, `mdm_input`) output shapes match the consumer's expectations. If `filter_result === 'skip'` (`stage === 'filtered'`), the row was filtered out before rendering — useful for verifying the filter semantics on a known-skip example. #### `.run` takes your payload VERBATIM — the pipeline adds two fields The sandbox renders exactly what you send it. A real ingest does not: before the templates see a row, the pipeline injects two fields the sandbox has no way to know. ``` source_uri the URI of the DATA SOURCE behind the bound data activation client — not anything in the row. A datalake's default manual-upload data source is created with `.alvera.ai`, so that is what a row ingested through it carries. timezone the datalake's timezone ``` Neither is bound during `.run` — there is no client in scope — so `{{ p.source_uri }}` renders whatever your sample row happens to carry (commonly nothing, or a placeholder like `test://crm/clients`). Both templates are faithful; the INPUT differs. The consequence bites when the identification `uri` is derived from `source_uri`: MDM stores the rendered `uri` verbatim, so the sandbox shows your placeholder and the persisted `legal_entity_identifications` row shows `.alvera.ai`. **Do not assert on a `uri` that comes from `source_uri` in `.run` output** — assert on `id_type` / `id_number`, which are row-derived and survive persistence unchanged. To verify the client-derived half, ingest through the DAC and query the persisted row. ### Generic-table contracts: auto-created identity contracts When you create a generic table (see `generic_tables.md`), the platform automatically creates an **identity contract** bound to that table: ``` new generic table (T) → auto-created contract with resource_type: 'generic_table' generic_table_id: T.id template_config.type: 'identity' ``` Discover this contract via `.list()` + filter on `generic_table_id`: ```typescript const { data: all } = await api.interoperabilityContracts.list( tenantSlug, datalakeSlug, ) const identityContract = all.data?.find( (c) => c.generic_table_id === tableId && c.type === 'identity', ) ``` Multiple contracts MAY bind to the same generic table — one identity (auto) plus any number of caller-created `custom` contracts. Distinguish by `type` when iterating. The platform enforces at most ONE identity contract per generic table; attempts to create a second `type: 'identity'` contract for the same table are rejected. The same datalake provisioning that creates the identity contract also creates a **default manual-upload data activation client** for that dataset (`" DataActivationClient"`, `is_default: true`). To ingest rows shaped by *this* contract, either `.ingest` through that default DAC or declare your own DAC and bind it via `interoperability_contract_ids` — never create a DAC from a contract's `afterCreateOrUpdate` hook (untracked, non-idempotent). A contract's own deep check should stay side-effect-free: sandbox-run it with `.run(...)` (below) and assert the mapping. See `data_activation_clients.md` §2. ### Read shapes ``` .list(tenantSlug, datalakeSlug, query?) Paged: { data, meta } (full Flop query — type_naming.md) .get(tenantSlug, datalakeSlug, id) One row, by UUID id .metadata(tenantSlug, datalakeSlug, query?) Markdown string — the datalake's whole contract catalog, one Flop page .metadataDetails(tenantSlug, datalakeSlug, id) Markdown string — one contract, by UUID id .run(tenantSlug, datalakeSlug, slug, msg) Sandbox runner (above) — keyed by SLUG ``` **CRUD is keyed by the UUID `id`, not the slug.** `.get`, `.update`, `.delete`, and `.metadataDetails` all take the contract's `id` — the show/update/delete path param is declared `format: :uuid` and the lookup is `get_interoperability_contract_for_datalake!(session, datalake, id)`. The resource DOES carry a `slug`, but the slug is used for `global_search` / `handle` Flop filtering, NOT as the CRUD key. The ONE slug-keyed verb is `.run` (backed by `get_interoperability_contract_by_slug!`). To turn a known `slug` or `name` into the `id`, `.list()` with a `handle` filter — the compound Flop field over `[slug, name]` — and read `.id` off the matched row. ### Delete `delete()` removes the contract. Contracts referenced by active data activation clients reject with a 422 naming the dependency. **System-created contracts cannot be edited or deleted.** The contract record carries a `system_created` flag (server-derived). When `system_created === true`, both `update()` and `delete()` reject with 422 on `/base` and a literal message: ``` PUT → "System-created contracts cannot be edited" DELETE → "System-created contracts cannot be deleted" ``` System-created contracts include the auto-generated identity contracts the platform provisions when a generic table is created (see §5 "Generic-table contracts: auto-created identity contracts") and the templates the platform ships with every datalake. User-created contracts of any `type` (including caller-created `type: 'identity'` rows bound to a generic table) ARE editable and deletable — the gate is `system_created`, not `type`. The flag IS exposed on the `InteroperabilityContractResponse`, so consumer agents can pre-filter before attempting a mutation: ```typescript const { data: list } = await api.interoperabilityContracts.list( tenantSlug, datalakeSlug, ) const editable = list.data.filter((c) => !c.system_created) ``` ## 6. Nesting AI agents on the contract body A contract can run AI-agent **enrichment** during ingestion. Agents are **nested directly on the contract body** — the create/update body accepts an `interoperability_contract_ai_agents` array. There is no separate attach/detach endpoint, and **no per-join checksum**: the nested agents fold into the contract's own checksum. ```typescript // Nest on create. `position` is REQUIRED. const { data: contract } = await api.interoperabilityContracts.create( tenantSlug, datalakeSlug, { name: 'MOA Enrichment Contract', resource_type: 'legal_entity', template_config: { type: 'custom', body: '{{ name }}' }, mdm_input_config: { type: 'null' }, // …other contract fields… interoperability_contract_ai_agents: [ { ai_agent_id: createdAgentId, position: 0, // ordering when multiple agents fire; REQUIRED context_mapping_config: { type: 'custom', // Liquid renders to a JSON object whose keys match the agent's // input_schema; values reference the inbound row via `msg.`. body: JSON.stringify({ name: '{{ msg.company_name }}' }), // A contract join AUTHORS output_schema — unlike a workflow // join, the server does NOT pin it here. output_schema: { type: 'object', properties: { lead: { type: 'string' } } }, }, }, ], }, ) contract.checksum // the PARENT checksum — the nested agents fold into it ``` Adding, repositioning, and removing agents are all expressed by re-supplying the full set on the contract's `update()` — `update()` replaces the whole array, and any item you omit is removed: | Operation | How | Result | |------------|--------------------------------------------------|--------| | add / nest | include in the array on create or update | contract checksum reflects the agents | | reposition | `update()` with the same agent at a new `position`| full-set replace | | detach | `update()` omitting that agent (or `[]` to clear all)| the omitted agent is removed | | drift | contract `checksum()` endpoint over the full body| the would-be contract checksum (no persist) | A contract `get()` surfaces its nested agents as `interoperability_contract_ai_agents: [...]`. The array is writable on the contract body (create + update). **Runtime.** Enrichment fires during ingestion, **not** on the sandbox `.run`. Attached agents run in `position` order: each agent's `context_mapping_config` renders against the row, the result is validated against the agent's `input_schema`, sent to the bound `llm_enrichment` tool, and the parsed response is merged under `msg.enrichment.` for the contract's `template_config` to read. So attaching an agent to a DAC-wired contract makes ingestion run enrichment per row — keep agents off contracts whose only job is a straight-through upsert. See `ai_agents.md` §3 for the agent create body and the shared attach surface, and `workflows.md` for the workflow side of the same join. ## 7. Gotchas 1. **`filter_template` is empty-passes, non-empty-skips.** Already documented in §2 — repeated here because it's the #1 source of "ingests nothing / ingests too much" silent failures. Always anchor on this rule. 2. **`mdm_input_config` is optional; omitting equals `{ type: 'null' }`.** The shared TemplateConfig embed defaults `type` to `'null'`, so a contract that omits `mdm_input_config` behaves as "the row IS the subject — no entity resolution runs". Declaring `{ type: 'null' }` explicitly is allowed for readability but not required. 3. **`resource_type` is a closed platform set.** Every datalake accepts the same six values; the retired per-industry types (`'patient'`, `'customer'`, `'payment_account'`, ...) now reject with a `/resource_type` 422 at create time. Model those as a generic table paired with a `legal_entity` contract instead. 4. **Sandbox-run is not optional in practice.** A template that compiles but renders structurally-invalid JSON, or that silently drops fields, won't surface until live ingestion — and at that point the failures are batched and harder to debug. Run `.run(slug, sampleRow)` against a representative row for every new contract before binding it to a data activation client. 5. **Generic-table identity contracts are auto-created; don't create them yourself.** Creating a generic table fires a side-effect that creates a `type: 'identity'` contract bound to the new table. Creating a second identity contract for the same table is rejected. To add custom transformation on top of a generic table, create an additional `type: 'custom'` contract bound to the same `generic_table_id`. 6. **Templates use `msg` to access the row.** Liquid receives the row under the `msg` variable. CSV columns are typically `msg.`; nested cases use `msg.row.`. JSON sources expose the inbound fields under `msg.`. Read any platform-shipped template (see `templates.md`) to learn the exact convention for your row shape. 7. **No partial updates.** `update()` replaces the whole resource — resupply the full body (see `mutations.md`). Re-supply `template_config` / `mdm_input_config` verbatim if not changing them. 8. **Custom Liquid filters are a closed allowlist of eleven.** The platform exposes exactly eleven custom filters beyond stock Liquid (`e164`, `age`, `date`, `to_json`, `json_escape`, `now`, `uuid`, `parse_date`, `convert_time`, `random`, `tz_offset`). Other filter names — including the Shopify-style `| json` — fall through to stock Liquid and produce malformed output silently. See `ai_sandbox.md` §2 for the full per-filter reference and the safety rationale for the closed set. The allowlist is the same in `template_config.body`, `mdm_input_config.body`, and `filter_template`. 9. **Dataset enum columns silently drop out-of-set rows — and sandbox-run won't catch it.** A `resource_type` maps onto a platform dataset whose columns may be `Ecto.Enum` (e.g. payments `payment_account.account_type` ∈ `{bank_account, card, wallet, crypto_wallet}`). An out-of-set value like `'checking'` PASSES `.run(slug, row)` — the sandbox renders JSON without casting the enum — but is rejected at live ingest: the batch reports `rows_ingested` yet the row never persists (`dataset_updated` stays `0`; no row points at the batch's log row). Sandbox-run (§7.4) proves the shape, not the enum domain — ground every enum value against the platform schema or a known-good fixture, never intuition. 10. **Overlay `resource_type`s resolve their MDM subject find-or-create — no pre-seed needed.** When a `resource_type` is an overlay of an identity subject (e.g. payments `payment_account` over `account_holder`), ingesting one row resolves that subject from the row's own `mdm_input` identity key (`account_number_uri` + `account_holder_number`): matched if it exists, else created. You do NOT ingest the subject first. A `:mdm_output_required` error in the platform logs means the MDM identity fields were missing or the resolve failed — not that a separate subject ingest is required.