# Tricky Handling

Every case here was found by walking all **30,784 actions** across 481 connectors and
enumerating all **6,431 distinct key paths** — not by picking dimensions in advance. If a
construct exists in the shipped corpus, it is in this file.

Load when the shape is not obvious from `connector-patterns` alone. All counts are real.

---

## 1. Step parameters — the complete list

`request` covers 95% of steps with just `url` + `method` + `args`. Everything below exists for
a case you will recognise when you hit it.

| Parameter | Count | Use |
|---|---|---|
| `url` `method` | 29,789 | — |
| `args` | 25,023 | — |
| `event` | 1,577 | `emit_event` payload |
| `dataSource` | 1,180 | input to a transform step |
| `baseUrl` | 1,103 | override the connector base URL for this step |
| `response` | 1,030 | pagination / extraction |
| `authorization` | 850 | override auth for this step |
| `customErrors` | 557 | rewrite a misleading provider status |
| `fields` | 332 | `map_fields` output shape |
| `externalSources` | 276 | pull values from outside the response body |
| `values` | 208 | `static_values` payload |
| `soapOperation` 136 · `useSoapContext` 135 · `namespaces` 125 · `soapAction` 80 · `envelope` 11 | | SOAP |
| `file` | 129 | `upload_file` |
| `iterator` | 117 | pagination cursor |
| `additionalFields` | 65 | extra multipart fields |
| `responseMode` | 51 | `stream` for large downloads |
| `followRedirects` 48 · `maxRedirects` 9 | | redirect control |
| `encoding` | 42 | `binary` for file bytes |
| `encodeQueryParams` | 28 | `false` when the provider wants raw query values |
| `binaryContent` | 18 | `upload_binary` |
| `responseType` | 12 | `arraybuffer` for binary responses |
| `contentType` 4 · `formData` 3 · `body` 4 | | explicit request shaping |
| `startUrl` `systemPrompt` `budgets` | 4 each | `browser_use` |

---

## 2. Per-step base URL — 1,103 uses

When the host differs from the connector's `baseUrl` — a regional host, a tenant host, or a
pre-signed URL on another domain:

```yaml
baseUrl: ${credentials.serverUrl}
```

Template syntax, like `url`. Common with self-hosted providers where the user supplies the
host as a credential.

---

## 3. Per-step auth override — 850 uses

```yaml
authorization:
  type: none
```

Required whenever a step must **not** carry the connector's auth: token endpoints, pre-signed
S3 URLs, public metadata. Forgetting it is why a pre-signed upload returns 403 — the extra
`Authorization` header invalidates the signature.

---

## 4. Pagination — all five shapes

**Bare array, nothing to page** — 799
```yaml
response:
  collection: true
```

**Records under a key** — 333
```yaml
response:
  dataKey: results
```

**Offset pagination** — `iterator.mode: offset`, 15
```yaml
response:
  dataKey: Resources
  nextKey: '{{$.startIndex + $.itemsPerPage <= $.totalResults ? $.startIndex + $.itemsPerPage : null}}'
iterator:
  key: startIndex
  in: query
  mode: offset
```

**Cursor with a stop condition** — `iterator.condition`, 9
```yaml
iterator:
  key: cursor
  in: query
  mode: cursor
  condition: '{{present($._links.next)}}'   # continue only while a next link exists
response:
  dataKey: results
  nextKey: '{{urlParam(($._links.base ?? "") + ($._links.next ?? ""), "cursor")}}'
```

`urlParam(url, "cursor")` extracts a query parameter from a URL the provider returned — the
usual way to turn a `next` link into the next cursor value.

**Dedupe across overlapping pages** — `response.indexField`, 5
```yaml
response:
  collection: true
  dataKey: results
  indexField: id
```

`response.nextKeyIn` (5) names where the next token lives when it is not in the body.

> **`nextKey` must eventually evaluate to `null`.** That is what ends the loop. An expression
> that always returns a value pages until the provider rate-limits you.

---

## 5. Transform steps

### `map_fields` — 757, reshape a response

```yaml
- stepId: shape
  stepFunction:
    functionName: map_fields
    parameters:
      dataSource: $.steps.fetch_classrooms.output
      externalSources:                                  # values from outside the body
        link: $.steps.fetch_classrooms.output.raw.headers.link
      fields:
        - targetFieldKey: next
          expression: '{{regexMatch($.external.link, "next=([^>]+)>", 1)}}'
          type: string
        - targetFieldKey: data
          expression: $.data
          type: string
```

`externalSources` is how you reach **response headers** — `output.raw.headers.<name>`. This is
the pattern for providers that paginate via the `Link` header rather than the body, and there
is no other route to it.

### `static_values` — 208, emit a computed constant

```yaml
- stepId: normalise
  stepFunction:
    functionName: static_values
    parameters:
      values: >-
        {{steps.fetch_team.output.data != null && steps.fetch_team.output.data.team != null
        ? steps.fetch_team.output.data.team : steps.fetch_team.output.data}}
```

Used to normalise providers that sometimes wrap a record and sometimes do not.

### `typecast` — 423, coerce types

```yaml
- stepId: cast
  stepFunction:
    functionName: typecast
    parameters:
      dataSource: $.steps.map_group_fields.output.data
```

---

## 6. File handling

### Upload via pre-signed URL — the three-step chain

Real example, `ashby · upload_candidate_resume_complete`:

```
get_presigned_url → request      ask the provider for a signed URL + form fields
upload_to_s3      → upload_file  POST the file there, auth: none
associate_resume  → request      tell the provider the upload completed
```

```yaml
- stepId: upload_to_s3
  stepFunction:
    functionName: upload_file
    parameters:
      url: $.steps.get_presigned_url.output.data.results.url
      method: post
      authorization:
        type: none                     # the signature IS the auth
      file:
        fieldName: file
        content: $.inputs.file_content
        filename: $.inputs.file_name
      additionalFields:                # signed form fields, before the file part
        - name: acl
          value: $.steps.get_presigned_url.output.data.results.fields.acl
        - name: X-Amz-Algorithm
          value: $.steps.get_presigned_url.output.data.results.fields.X-Amz-Algorithm
```

### Direct multipart upload

```yaml
functionName: upload_file
parameters:
  url: /companies/${inputs.companyId}/requests/${inputs.requestId}/attachments
  method: post
  file:
    fieldName: files
    content: $.inputs.file_content
    filename: $.inputs.file_name
```

### `formData` with a JSON part

```yaml
formData:
  - name: attributes
    value: $.inputs.attributes
    valueType: json                    # serialise this part as JSON
    condition: '{{present(inputs.attributes)}}'
```

### Download — 61

```yaml
functionName: download_file
parameters:
  url: /files/${inputs.fileId}/content
  method: get
  responseMode: stream        # do not buffer the whole file
  encoding: binary
  followRedirects: true       # storage backends redirect to a CDN
  maxRedirects: 5
```

### Raw binary body — `upload_binary`, 18

```yaml
binaryContent:
  content: $.inputs.fileContent
  contentType: $.inputs.contentType
```

### Binary response from a normal request

```yaml
responseType: arraybuffer
```

---

## 7. SOAP — 136 actions

```yaml
functionName: soap_request
parameters:
  soapOperation: ListBranches
  soapAction: http://www.monitorsoft.com/SentinelXS/ListBranches
  useSoapContext: true
  namespaces:
    - namespaceIdentifier: cls
      namespace: http://www.monitorsoft.com/SentinelXS/clsSentinelXS
    - namespaceIdentifier: xsi
      namespace: http://www.w3.org/2001/XMLSchema-instance
```

Where the provider demands a hand-built envelope (WS-Addressing, tokens), supply it literally.
`@_` prefixes an XML attribute; `#text` is the element's text content:

```yaml
envelope:
  s:Envelope:
    '@_xmlns:a': http://www.w3.org/2005/08/addressing
    '@_xmlns:s': http://www.w3.org/2003/05/soap-envelope
    s:Header:
      a:Action:
        '@_s:mustUnderstand': 1
        '#text': http://www.ultipro.com/services/.../FindCompensations
```

Use `soap_paginated_request` for paged SOAP endpoints.

---

## 8. Custom errors — 557 uses

```yaml
customErrors:
  - receivedStatus: 200
    targetStatus: 404
    message: Role not found. No permission set matches this id on the observed users page.
```

For providers that return 200 with an empty body for a missing record, or a 400 that is really
a 404. The message should tell the caller what to do, not restate the status.

---

## 9. Query encoding — 28 uses

```yaml
encodeQueryParams: false
```

Set when the provider expects raw characters in the query string — commas in a list, or a
pre-encoded token that must not be double-encoded. Symptom: the provider sees `%2C` where it
wanted `,`.

---

## 10. Expression vocabulary — what is actually used

`present` 89,072 dwarfs everything else. The rest, by real use:

| Function | Count | Use |
|---|---|---|
| `present(x)` | 89,072 | is the value set — the basis of nearly every condition |
| `regexMatch(s, pattern, group)` | 192 | pull a value out of a string or header |
| `reduce` | 131 | fold a list |
| `replace` | 124 | string substitution |
| `asString` | 111 | coerce for concatenation |
| `hmacSha256` | 65 | request signing |
| `join` | 47 | list → delimited string |
| `includes` 45 · `groupBy` 29 · `missing` 26 · `keys` 21 | | collection helpers |
| `encodeBase64` 17 · `unixToIso` 16 · `parseJson` 9 · `lower` 8 | | conversion |
| `urlParam(url, name)` | 11 | extract a query param from a returned URL |

**Operators:** ternary `?` 2,402 · `+` 1,952 · `!` 1,879 · `&&` 1,283 · `??` 1,177 (null
coalesce) · `==` 914 · `||` 723 · `!=` 638

`??` matters for pagination — `($._links.base ?? "") + ($._links.next ?? "")` avoids
concatenating `undefined` into a URL.

---

## 11. Conditions beyond `present`

```yaml
condition: '{{present(inputs.a) && present(inputs.b)}}'    # 230 — required together
condition: '{{present(inputs.a) && !present(inputs.b)}}'   # 182 — mutually exclusive
condition: '{{present(credentials.apiKey)}}'               # 787 — only when this credential exists
condition: '{{context.authenticationType == "oauth2"}}'    # 123 — differs per auth type
condition: '{{credentials.privateKey != null}}'            #  98 — explicit null check
```

`context.authenticationType` is how one action serves both an API-key and an OAuth connection
when the provider wants different parameters for each.

---

---

## 13. Event actions — 1,913 of them

An inbound webhook handler looks nothing like a request action. It makes **no HTTP call**; it
transforms the delivered payload and returns an HTTP acknowledgement.

```yaml
- actionId: webhook_alert
  actionType: event
  categories: [observability, infrastructure]
  tags: [Alert]
  label: Alert Webhook
  description: Processes every alert notification delivered to the webhook channel.
  inputs:
    - name: event                     # the whole delivered payload
      description: The full alert webhook payload.
      type: object
      in: body
      required: false                 # false — a provider test-ping may send nothing
  steps:
    - stepId: emit
      stepFunction:
        functionName: emit_event
        parameters:
          event:
            eventType: '{{present(inputs.event.entityType) ? inputs.event.entityType : inputs.event.alertName}}'
            eventId: '{{inputs.event.entityId ?? "NA"}}'
            eventDate: '{{asString(inputs.event.date)}}'
            data: $.inputs.event
  result:
    statusCode: 200                   # the response sent BACK to the provider
    body:
      status: OK
```

**`result` is an HTTP response, not data.** 1,667 of 1,913 use exactly
`{statusCode: 200, body: {status: OK}}`; 179 use an empty body; 5 use `202`. Fifteen return
XML because the provider demands it:

```yaml
  result:
    statusCode: 200
    headers:
      Content-Type: text/xml; charset=utf-8
    body: '<?xml version="1.0" encoding="utf-8"?>...'
```

**`emit_event.event` keys:** `eventType` 1,874 · `data` 1,873 · `eventId` 1,781 ·
`eventDate` 1,358.

**Input signatures vary by provider shape:** `payload:object` 307 · `event:object` 139 ·
`entity_id,event_type,timestamp,payload` 150 · `eventType,eventId,payload` 116.

`eventId` needs a fallback when the payload may omit it — `?? "NA"` is the corpus idiom.
`eventDate` usually needs `asString()` because providers send numbers or non-ISO strings.

Pair with `providerEvents: [worker.hire]` on the action, and a `router` rule in the main
file's `events` block to dispatch to it.

---

## 14. SOAP — the envelope in full

125 of 136 SOAP steps use **no `envelope`** — the framework builds it from `soapOperation`
and the args. Supply one only when the provider demands headers you cannot otherwise set:

```yaml
- stepId: find_jobs_soap
  stepFunction:
    functionName: soap_request
    parameters:
      url: /services/EmployeeJob
      method: post
      soapOperation: FindJobs
      soapAction: http://www.ultipro.com/services/employeejob/IEmployeeJob/FindJobs
      useSoapContext: false
      envelope:
        s:Envelope:
          '@_xmlns:a': http://www.w3.org/2005/08/addressing
          '@_xmlns:s': http://www.w3.org/2003/05/soap-envelope
          s:Header:
            a:Action:
              '@_s:mustUnderstand': 1
              '#text': http://www.ultipro.com/services/employeejob/IEmployeeJob/FindJobs
            UltiProToken:
              '@_xmlns': http://www.ultimatesoftware.com/foundation/authentication/ultiprotoken
              '#text': ${credentials.accessToken}      # template syntax
            ClientAccessKey:
              '@_xmlns': http://www.ultimatesoftware.com/foundation/authentication/clientaccesskey
              '#text': ${credentials.apiKey}
          s:Body:
            FindJobs:
              '@_xmlns': http://www.ultipro.com/services/employeejob
              query:
                '@_xmlns:b': http://www.ultipro.com/contracts
                b:CompanyCode: $.inputs.companyCode      # JSONPath syntax
                b:EmployeeNumber: $.inputs.employeeNumber
                b:FirstName: $.inputs.firstName
      args:
        - name: Content-Type
          value: application/soap+xml; charset=utf-8
          in: headers
  result:
    data: $.steps.find_jobs_soap.output.data
```

| Convention | Meaning |
|---|---|
| `'@_name'` | an XML attribute on that element |
| `'#text'` | the element's text content |
| `${credentials.x}` | template — used in header text |
| `$.inputs.x` | JSONPath — used for body values |

**No conditions inside an envelope.** Zero of 136 SOAP steps condition an envelope field;
unset values are simply omitted. Conditions belong on `args`, not envelope entries.

`useSoapContext` is `false` here and `true` elsewhere — follow the provider's WSDL.

---

## 15. Download — the full shape

```yaml
- stepId: download_file_request
  stepFunction:
    functionName: download_file
    parameters:
      url: /files/${inputs.file_id}/content
      method: get
      followRedirects: true      # storage backends redirect to a CDN
      responseMode: stream       # do not buffer the file into memory
      encoding: binary
      args:
        - name: range
          value: $.inputs.range
          in: headers
          condition: '{{present(inputs.range)}}'
result:
  data: $.steps.download_file_request.output.data
```

The JSON response body is **empty** — the file arrives as a downloadable artifact, not inline.
Say so in the action `description`, or callers will report the action as broken.

**Header inputs** — 636 inputs are `in: headers`, for values the *caller* supplies (a byte
`range`, a shared-link token). Distinct from the 22,121 header *args*, which are constants you
set.

---

## 16. `requiredScopes` is a string, not a list

All 12,387 uses are a bare string:

```yaml
requiredScopes: root_readonly
```

`authRequirements` (1,401 uses) is the modern replacement and is a **map keyed by auth-block
id with array values**. Do not carry the string shape across when converting.

---

---

## 17. GraphQL — 396 actions across 7 providers

There is no GraphQL step function. It is a plain `request` POST with the query and variables
as body args.

**394 of 396 hardcode the query** and take only `variables` from inputs — one action per
GraphQL operation, not a generic passthrough:

```yaml
- actionId: list_assets
  actionType: custom
  inputs:
    - name: prefix
      description: Filter assets by key prefix.
      type: string
      in: body
      required: false
      array: true
  steps:
    - stepId: query
      stepFunction:
        functionName: request
        parameters:
          url: /graphql
          method: post
          args:
            - name: Content-Type
              value: application/json
              in: headers
            - name: query
              value: >-
                query($prefix: [String!], $cursor: String, $limit: Int) {
                  assetsOrError(prefix: $prefix, cursor: $cursor, limit: $limit) {
                    __typename
                    ... on AssetConnection { nodes { id key { path } } cursor }
                  }
                }
              in: body
            - name: variables
              value: $.inputs
              in: body
  result:
    data: $.steps.query.output.data
```

The remaining 2 accept the query from an input — a deliberate escape hatch:

```yaml
    - name: query
      description: GraphQL query string. Minimum length 1.
      type: string
      in: body
      required: true
      rules:
        minLength: 1
    - name: variables
      description: Variables for the GraphQL query.
      type: object
      in: body
      required: false
```

Notes that matter:

- **Model one action per operation.** A single "run any GraphQL" action gives callers nothing
  to discover and no parameter validation.
- **GraphQL returns HTTP 200 on errors**, with an `errors` array in the body. Use
  `customErrors` to surface that as a real status where it matters.
- Some providers expose the same endpoint over GET, with `variables` as a JSON-encoded string
  in the query string rather than an object in the body.
- Pagination is per-schema — the cursor is inside the query result (`cursor` above), so it
  needs `response.dataKey` pointing into the GraphQL shape, not a generic list key.

---

## 12. Traps

| Symptom | Cause |
|---|---|
| Optional parameter rejected as null | Missing `condition: '{{present(inputs.X)}}'` — 68% of args carry one |
| `${}` in an arg value, or `$.` in a URL | URLs interpolate `${inputs.x}`; arg values are JSONPath `$.inputs.x` |
| Input silently ignored | Wrote `key:` instead of `name:`. `key` belongs to auth `setupFields`/`configFields` |
| Pre-signed upload returns 403 | Missing `authorization: { type: none }` — the extra header breaks the signature |
| Pagination never terminates | `nextKey` never evaluates to `null` |
| Duplicate records across pages | Needs `response.indexField` |
| `%2C` where a comma was wanted | Needs `encodeQueryParams: false` |
| Cannot reach a `Link` header | Needs `map_fields` + `externalSources` with `output.raw.headers.<name>` |
| Nested field rejected | `in:` belongs on the top-level input only, never inside `properties` |
| Step runs when it should not | Steps take a `condition` too, not just args — 632 uses |
| Guide shows irrelevant steps | Needs `applicableScopes` on the section — 156 uses |
| `{{...}}` breaks YAML parsing | JEXL must be single-quoted |
| GraphQL action reports success on failure | GraphQL returns 200 with an `errors` array — needs `customErrors` |
| Wrote `fieldConfigs` / `entrypointUrl` / `schemaType` | Building a unified action — out of scope. Mirror the provider's API instead |
| Schema review blocks on scopes | Corpus has `requiredScopes` on 12,383 actions but it is deprecated — write `authRequirements` |
| Large download exhausts memory | Needs `responseMode: stream` |
| Download returns an empty body | Expected — content is a separate artifact; say so in the description |
| Event action reports a failure | `result` must be an HTTP ack (`statusCode` + `body`), not `data` |
| SOAP field always sent | Envelope entries take no `condition`; unset values are omitted |
| `eventDate` rejected | Needs `asString()` — providers send numbers or non-ISO strings |
| `method: head` rejected | Only `get post put patch delete` exist — all 29,789 corpus steps use those five. A HEAD-documented metadata endpoint is modeled `get` on the same URL when the provider serves it, else recorded in `dropped[]` with a reason |
| Auth boilerplate headers as inputs | `Authorization`, `Date`/`x-ms-date`, `Content-Length` are runtime/auth-managed — mark them `transport: true` in research (B4 skips them); a fixed version pin (`x-ms-version`) is a literal step arg, never an input |

## REST XML request bodies (non-SOAP)

Azure-style REST APIs take raw XML bodies without a SOAP envelope. The schema has no
JSON→XML body serializer, and the `soap_request` step is wrong for them (it builds an
envelope). Model them as a single **string input carrying the raw XML document**, sent as
the body itself with an explicit `Content-Type` — `spread: true` makes the value the body
rather than nesting it under the arg's key (corpus documents it for objects/arrays; for a
raw string body, pilot it through `stackone validate` first, and fall back to the
`upload_binary` step, whose `content:` parameter is the verified raw-body mechanism):

```yaml
inputs:
  - name: body
    description: Raw XML request document, e.g. <BlockList><Latest>...</Latest></BlockList>
    type: string
    in: body
    required: true
steps:
  - stepId: put_block_list
    description: Commit the block list
    stepFunction:
      functionName: request
      parameters:
        url: '/${inputs.container}/${inputs.blob}'
        method: put
        args:
          - name: body
            value: $.inputs.body
            in: body
            spread: true               # raw body, not {"body": "..."}
          - name: Content-Type
            value: application/xml
            in: headers
```

State the expected XML shape (root element, children) in the input's description — the
caller has to author the document, so the description is the contract. Decide this shape
**per endpoint at research time** (record it in `usage_notes`) rather than discovering the
missing serializer mid-build: a measured azureblobstorage run stalled on exactly that
decision across 4 actions.

## 16. HTTP conditional headers are strings, not `datetime_string`

`If-Modified-Since`, `If-Unmodified-Since`, and friends (`x-ms-access-tier-if-*`,
`x-ms-immutability-policy-until-date`) take RFC 7231 IMF-fixdate values
(`Tue, 29 Apr 2025 09:00:00 GMT`), not ISO 8601 — typing them `datetime_string` produces a
value the provider rejects. Type them `string` and say the expected format in the
description. Corpus corroboration: `awss3` types `if_modified_since` as `string`. Typing
an IMF-fixdate header `datetime_string` is a recurring generated defect — the type is
right for the concept and wrong for the wire format.