Learn how the Grepr remapper standardizes log event fields and maps raw status values to severities during pipeline processing. # Remap JSON attribute fields to top-level log event fields The remapper transform moves or copies log event attributes to top-level log event fields, such as `message` or `eventTimestamp`, or to tags. You can also rename attributes by moving a value from one attribute path to another. See [The remapper transform rename rules](#rename-rules). When you use the Grepr UI to create a pipeline, a remapper that transforms a default set of commonly used fields is automatically added to the pipeline, but you can modify which fields are transformed based on your requirements. ## Example remapper transformation For example, this event: ```json { "id": "ABCDEF", "eventTimestamp": 1765481731051, "message": "", "severity": "", "tags": {}, "attributes": { "syslog": { "severity": "10", "appname": "test name" }, "status": "5", "msg": "message 1", "timestamp": { "ms_since_epoch": 1765481730000 }, "eventTime": " " } } ``` Would be transformed using the default configuration to: ```json { "id": "ABCDEF", "eventTimestamp": 1765481731051, "message": "message 1", "severity": "5", "tags": { "service": "test name" }, "attributes": { "syslog": { "severity": "10", "appname": "test name" }, "status": "5", "timestamp": { "ms_since_epoch": 1765481730000 }, "eventTime": " ", "greprMeta": { "messageAttributePath": "msg" } } } ``` * `severity` uses `status: "5"` instead of `syslog.severity: "10"` because `status` has a higher priority in the default configuration. * `tags.service` uses `syslog.appname: "test name"`. * The attribute `msg` is removed because it's remapped to the top-level `message` field and the `message` field has preserve source disabled by default. * The `greprMeta.messageAttributePath` attribute is added to indicate which attribute was used for the remapped `message` field. * If the message attribute is a nested field, for example, `log.msg`, only the msg field is removed. The parent `log` object remains, even if it is empty. * Other remapped attributes like `status` and `syslog.appname` remain in their original locations because their corresponding fields have preserve source enabled by default. * `timestamp: {}` and `eventTime: " "` are not used because they do not contain a valid timestamp value. ## The remapper transform default configuration The following is the default remapper configuration. The **Transformed attribute or tag** column shows the field name in the transformed event after the input event is processed. The **Preserve source** column indicates whether the source attribute is kept in the original location after its value is copied to the target field, and **Default paths searched** are where the remapper looks in the input event for fields to transform. | Transformed attribute or tag | Preserve source | Default paths searched | | --- | --- | --- | | `timestamp` | yes | `date`, `@timestamp`, `T`, `syslog.timestamp`, `eventTime`, `_timestamp`, `Timestamp`, `published_date`, `timestamp` | | `host` | yes | `hostname`, `sourceIPAddress`, `host`, `kubernetes.host`, `syslog.hostname` | | `service` | yes | `dd.service`, `kubernetes.labels.app`, `service`, `eventSource`, `kubernetes.labels.k8s-app`, `syslog.appname` | | `status` | yes | `log.level`, `level`, `status`, `severity`, `syslog.severity`, `L` | | `message` | no | `msg`, `log`, `message`, `M` | | `trace ID` | yes | `dd.trace_id`, `contextMap.dd.trace_id`, `otelTraceId` | You can toggle the **Preserve source** setting for each field in the pipelines UI. When preserve source is enabled, the source attribute remains in its original location after the value is copied to the target field. When disabled, the source attribute is removed after remapping. For example, the `message` field has preserve source disabled by default, so the original `msg` attribute is deleted after being copied to the top-level `message` field. Tag mappings also support the preserve source option. You can configure whether the source attribute is kept or removed after its value is copied to a tag. For the full list of default configuration settings, see the [LogAttributesRemapper specification](/apis/api-spec#schema/LogAttributesRemapper). ## Map status values to severities \[#status-mapping] The remapper converts the raw value it finds in a status attribute, such as `WARN`, `err`, or `13`, to one of Grepr's standard severities: `TRACE`, `DEBUG`, `INFO`, `WARN`, `ERROR`, or `FATAL`. The event's `severity` field stores the number at the start of that severity's OpenTelemetry range, such as `13` for `WARN`. See [The Grepr processing and data models](/grepr-platform/grepr-model). ### The default status mapping \[#default-status-mapping] By default, the remapper recognizes the following names and the first three and first four characters of each name, such as `war` and `warn`: | Severity | Recognized names | | --- | --- | | `TRACE` | `trace` | | `DEBUG` | `debug`, `verbose` | | `INFO` | `informational`, `information`, `notice`, `success`, `pending` | | `WARN` | `warning` | | `ERROR` | `error`, `critical`, `failure`, `failed`, `failing` | | `FATAL` | `fatal`, `emergency`, `alert` | Each name is recognized in lowercase, uppercase, and capitalized form, such as `error`, `ERROR`, and `Error`. Other casings, such as `eRRoR`, aren't recognized. A number from 1 to 24 is kept as its OpenTelemetry severity number, so a finer level such as `14`, which is `WARN2`, is preserved. ### Define your own status mapping \[#status-mapping-custom] To classify status values that the default mapping doesn't recognize, set the `statusMapping` field of the log attributes remapper. For each severity, list `exact` values and `prefixes`, and add `numberRanges` for numeric status values. Your mapping replaces the default mapping and isn't combined with it, so include every value you want classified. Grepr removes leading and trailing whitespace from a status value, then classifies it as follows: * A value equal to one of a severity's `exact` values gets that severity. * Otherwise, a value that starts with one of a severity's `prefixes` gets that severity. When several prefixes match, the longest one applies, so `emerg` takes precedence over `e` for `emergency`. * A whole number within one of the `numberRanges`, including both bounds, gets that range's severity. A range within 1 to 24 can set `severity` to `KEEP_NUMBER` to keep the matched number as the OpenTelemetry severity number. Matching is case-sensitive, so list each casing you expect, such as `e` and `E`. A string that contains a number, such as `"3"`, is matched only by `exact` values and `prefixes`. If nothing matches a value, the remapper tries the next attribute in the status attribute list. Because the order in which you list values doesn't matter, each value can appear only once among all `exact` values and once among all `prefixes`, and number ranges can't overlap. A value can be both an `exact` value and a prefix, in which case the exact match applies. To edit the status mapping in the pipelines UI, click **JSON Parser** in the navigation menu, click **Edit** next to **Attribute Remapper**, then select the **Status Remapping** tab. Type a value and press Enter to add it, or end it with `*` to add a prefix. To match a value that ends with a literal `*`, end it with `\*` instead. **Add all casings** adds the lowercase, uppercase, and capitalized forms of what you type; click a value marked **Aa** to see and remove each casing. To return to Grepr's default mapping, click **Reset to Defaults**. ### Example status mapping \[#status-mapping-example] An application logs its own levels, such as `SEV-CRIT` and `sev-lo`, and also logs syslog-style numeric levels from 0 to 7. The following mapping classifies both: ```json { "type": "log-attributes-remapper", "statusMapping": { "fatal": { "exact": ["SEV-HI", "SEV-CRIT"] }, "error": { "exact": ["sev-hi", "sev-err"] }, "debug": { "prefixes": ["sev-lo", "sev-dbg"] }, "info": { "prefixes": ["sev"] }, "numberRanges": [ { "min": 0, "max": 2, "severity": "FATAL" }, { "min": 3, "max": 3, "severity": "ERROR" }, { "min": 4, "max": 4, "severity": "WARN" }, { "min": 5, "max": 6, "severity": "INFO" }, { "min": 7, "max": 7, "severity": "DEBUG" } ] } } ``` This mapping classifies status values as follows: | Status value | Severity | Reason | | --- | --- | --- | | `"SEV-CRIT"` | `FATAL` | An exact value under `fatal` | | `"sev-dbg-cache"` | `DEBUG` | `sev-dbg` is the longest matching prefix | | `"sev-mid"` | `INFO` | `sev` is the only matching prefix | | `4` | `WARN` | The number range for 4 | | `"Sev-Hi"` | Not classified | Matching is case-sensitive and no prefix starts with `S` | | `"warning"` | Not classified | The default mapping isn't part of this mapping | When a value isn't classified, the remapper tries the next status attribute, and if none matches, the event keeps its existing severity. ### Search by status after remapping \[#status-search] In log searches, `status:error` matches events whose severity is `ERROR`, whichever raw value the mapping classified as `ERROR`. To search for a raw status value, such as one that your mapping classifies, search the attribute it arrived in, such as `@status:sev-mid` or `@level:sev-mid`. This works only while the remapper preserves the source status attribute, which it does by default. ## The remapper transform rename rules \[#rename-rules] To move or copy an attribute value from one path to another path, use a *rename rule*. Rename rules move an attribute value from one path to another in the same log event. Each rule has a **Preserve source** setting that controls whether the source attribute is kept or removed after its value is copied to the target path. If the target path already exists, its value is overwritten. To add rename rules in the pipelines UI, click **JSON Parser** in the navigation menu, click **Edit** next to **Attribute Remapper**, then select the **Rename Rules** tab. Rules can be reordered by dragging them, and they are applied in order from top to bottom. ### Example rename rule Given the following attributes: ```json { "attributes": { "request": { "method": "POST", "path": "/api/users" } } } ``` A rename rule with source `request.method` and target `http_method` produces: ```json { "attributes": { "request": { "path": "/api/users" }, "http_method": "POST" } } ``` The source field `request.method` is removed, while sibling fields like `request.path` are preserved. ### Preserving the source attribute \[#preserve-source-attribute] When **Preserve source** is disabled on a rename rule, the source attribute is removed after its value is copied to the target. To keep the source attribute in its original location, enable the **Preserve source** checkbox on the rule. This copies the value to the target path without removing the original. Using the same example above, with **Preserve source** enabled, the result would be: ```json { "attributes": { "request": { "method": "POST", "path": "/api/users" }, "http_method": "POST" } } ``` The value is copied to `http_method`, but `request.method` remains in its original location. ### Write a path as an array of attribute names \[#array-paths] A source or target path can be written in two forms. In a dotted string, each `.` steps into a nested attribute, so `http.user.agent` means `http` → `user` → `agent`. In a JSON array, each entry is one attribute name, used exactly as written, so `["http", "user", "agent"]` is the same path. Because array entries are never split, the array form can also name an attribute whose name contains a dot. In `{"http": {"user.agent": "curl/8.4.0"}}`, a rule with source `["http", "user.agent"]` and target `user_agent` copies `curl/8.4.0` to `user_agent`. ### Conditional rename rules Each rename rule can include an optional filter query. When a filter is set, the rule only applies to log events that match the query. This allows you to define multiple rules for the same source attribute with different filters, so each rule targets a different subset of events. For example, you could configure two rules for the same source `origin`: * `origin` → `cloud.provider` when the filter matches `env:production` * `origin` → `debug.origin` when the filter matches `env:staging` Because the filters are mutually exclusive, only one rule applies per event. ### Rule ordering Rename rules are applied sequentially in the order they appear. If two rules share the same source attribute, the first matching rule removes the source value. Subsequent rules for the same source will not find a value unless their filters prevent the earlier rule from applying. See the [AttributeRemappingRule specification](/apis/api-spec#schema/AttributeRemappingRule). ## Frequently Asked Questions ### What does the remapper transform do? The remapper transform moves or copies attributes from nested JSON structures to top-level log event fields (like message, severity, host, timestamp) or to tags. This standardizes log event structure and makes important fields easily accessible for filtering, searching, and display. ### Is the remapper automatically applied in the Grepr UI? Yes, when you create a pipeline using the Grepr UI, a remapper is automatically added with a set of predefined default attributes for well-known field names. You can modify this configuration based on your specific requirements. ### What are the default field mappings for the remapper? The default configuration includes mappings for: timestamp (from date, @timestamp, syslog.timestamp, etc.), host (from hostname, sourceIPAddress, kubernetes.host), service (from dd.service, kubernetes.labels.app, syslog.appname), status (from severity, level, log.level), message (from msg, log, message), and trace ID (from dd.trace_id, otelTraceId). ### How does field priority work in the remapper? The remapper uses priority to select which source attribute to map when multiple candidates exist. For example, if both 'status' and 'syslog.severity' are present, the remapper uses the one with higher priority. The default configuration specifies priority ordering for well-known fields. ### What happens to the original attribute after remapping? This depends on the Preserve source setting for each field. When preserve source is enabled, the original attribute remains in its original location after the value is copied to the target. When disabled, the source attribute is removed. For example, the message field has preserve source disabled by default, so the original 'msg' attribute is deleted after being copied to the top-level message field. Most other fields like timestamp, host, and status have preserve source enabled by default, so their source attributes are kept. ### How do I customize the remapper configuration? You can modify the default attribute names and their priority by editing the remapper configuration. This allows you to map custom field names from your logs to standard top-level fields. Access these settings when configuring the remapper step in your pipeline. ### What is greprMeta and why is it added? greprMeta is a metadata field added by the remapper to track remapping information. It includes messageAttributePath, which indicates which attribute was used for the remapped message field. This provides an audit trail of where the message content originated from. ### Can I remap attributes to tags instead of top-level fields? Yes, the remapper can copy or move attributes to the tags section of a log event. For example, you can map syslog.appname to a tag like tags.service. This is useful for adding contextual information for filtering and categorization without cluttering the top-level fields. ### What happens if a remapping source field is empty or has an invalid value? The remapper handles invalid values gracefully. For timestamp fields, if the value doesn't contain a valid timestamp, it's not used for remapping. Empty values are also skipped. The remapper continues to the next priority option if the current source field is invalid. ### Can I use the remapper with nested field paths? Yes, the remapper supports nested field paths like syslog.severity and kubernetes.host. When remapping a nested field, only the immediate field is removed from the parent object if the removal setting is true. Parent objects are preserved even if they become empty after field removal. ### What are rename rules and how do they differ from the default remapper? Rename rules move or copy an attribute value from one path to another within the attributes of a log event. Unlike the default remapper, which maps attributes to top-level fields like message or severity, rename rules restructure attributes without changing top-level fields. By default, the source attribute is removed after the value is copied to the target, but you can enable Preserve source on each rule to keep the original attribute. ### Can I apply a rename rule conditionally? Yes, each rename rule supports an optional filter query. When a filter is set, the rule only applies to events that match the query. This allows you to create multiple rules for the same source attribute with different filters targeting different subsets of events. ### Does the order of rename rules matter? Yes, rename rules are applied sequentially from top to bottom. If two rules share the same source, the first matching rule removes the source value and later rules for the same source will not find a value to move. Use filters to ensure only one rule matches per event when sharing a source attribute. ### Can I reorder rename rules? Yes, you can drag and drop rename rules in the Grepr UI to change their execution order. Rules are applied from top to bottom, so the order determines which rule takes effect first when multiple rules could match. ### What does Preserve source do? Preserve source controls whether the original attribute is kept or removed after its value is copied to the target. When enabled, the source attribute stays in its original location alongside the new target. When disabled, the source attribute is removed after remapping. This setting is available on reserved attribute mappings, tag mappings, and individual rename rules.