--- title: "Field Parser" description: "scan TenXTemplates for JSON and KV fields" source: "https://github.com/log-10x/modules/tree/main/pipelines/run/units/transform/fields/unit.yaml" icon: "material/equal" --- Extracts JSON objects and KV structures from [TenXTemplates](https://doc.log10x.com/run/template "Import joint JSON schemas files to expand events into well-defined TenXObjects.") as named fields. This process enables [TenXObjects](https://doc.log10x.com/api/js/#TenXObject "Provide structured, reflective access to log/trace events read from input(s).") to access nested JSON/KV values at runtime as named members. For an hybrid JSON/plain event whose [text](https://doc.log10x.com/api/js/#TenXBaseObject+text "The content of the event from which this instance was structured.") value is: ```json 17/06/24 17:38:00 INFO ExecutorRunnable: Prepared Local resources Map(__spark__.jar -> resource { scheme: "hdfs" host: "10.10.34.11" port: 9000 file: "/user/curi/.sparkStaging/application_1485248649253_0144/spark-assembly-1.6.0-hadoop2.2.0.jar" } size: 109525492 timestamp: 1497001131801 type: FILE visibility: PRIVATE) ``` To increase a [counter](https://doc.log10x.com/api/js/#TenXCounter "Get and set the values of global atomic counters.") identified by a JSON field (scheme) value with that of a KV entry (size): ```js TenXCounter.inc(this.scheme, this.size); ``` To [drop](https://doc.log10x.com/api/js/#TenXObject+drop "Hard-drops the current instance from the 10x pipeline at construction time,") the instance if either a KV entry (e.g., _visibility_) or JSON field (e.g., _port_) match a specified value: ```js this.drop((this.visibility == "PRIVATE") || (this.port == 9000)); ``` ### :material-code-array: Arrays If a JSON/KV field appears more than once within an TenXObject's text, it can be accessed as an array field to access individual entries (e.g., `this.myField[i]`). ## :material-wrench-outline: Config Files To configure the Field parser unit, [:material-cog: Edit](https://doc.log10x.com/config/app/#module-config "Learn how to edit app and module configurations") these files. Below is the default configuration from: [fields/config.yaml](https://github.dev/log-10x/config/blob/main/pipelines/run/transform/fields/config.yaml "fields/config.yaml"){target="\_blank"}.
Edit Online
Edit config.yaml Locally
```yaml # 🔟❎ 'run' TenXTemplate field extract configuration # Configure how to extract JSON objects and KV structures from TenXTemplates. # To learn more see https://doc.log10x.com/run/transform/fields/ # Set the 10x pipeline to 'run' tenx: run # =============================== Extract Options ============================= field: # 'extract' controls whether to scan TenXTemplate for JSON # objects or key-value lists (e.g., 'X=Y'). extract: true # 'nameBreaks' controls which characters found to the left of a token # whose is a candidate for being a 'key' in a KV field formation should serve # as a terminator for the search. For example, for an object whose 'text' field # contains the following entry ',tx_result=OK' the desired key name should be 'tx_result' # and as such, the name terminator character should be ',' vs. '_' (in which case # the key would have been named 'result'). nameBreaks: ', /\{}.()[]' # 'valueBreaks' controls which characters found to the right of a token # whose is a candidate for being a 'value' in a KV field formation should serve # as a terminator for the search. For example, for an object whose 'text' field # contains the following entry: 'status=RESULT_SUCCESS' the desired KV value # should be 'RESULT_SUCCESS', and as such, the value terminator character # should be ',' vs. '_' (in which case the value would be 'RESULT'). valueBreaks: ', /\{}()[]' ``` ## :material-menu: Options Specify the options below to [configure](/config "configure") the Field parser: |Name|Description| |---|---| |[fieldExtract](#fieldextract "scan TenXTemplate for nested JSON objects or key-value lists")|Scan TenXTemplate for nested JSON objects or key-value lists| |[fieldNameBreaks](#fieldnamebreaks "KV field name terminator chars")|KV field name terminator chars| |[fieldValueBreaks](#fieldvaluebreaks "KV field value terminator chars")|KV field value terminator chars| ### :material-menu-right-outline:**`fieldExtract`** Scan TenXTemplate for nested JSON objects or key-value lists. |Type|Default| |---|---| |Boolean|true| Controls whether to scan TenXTemplates for JSON objects or key-value lists (e.g., `{"price": 10}` or `price=10`). TenXObjects instances of this template will provide access to these fields as named members (e.g., `this.price`). If a field appears more than once in [text](https://doc.log10x.com/api/js/#TenXBaseObject+text "The content of the event from which this instance was structured.") it is accessible as an array (e.g., `this.price[1]`). ### :material-menu-right-outline:**`fieldNameBreaks`** KV field name terminator chars. |Type|Default| |---|---| |String|, /\\{}.()\[\]| Controls which 'text' characters left of a candidate token for being a 'key' in a KV field formation should serve as a terminator for the search. For example, for an TenXObject whose [text](https://doc.log10x.com/api/js/#TenXBaseObject+text "The content of the event from which this instance was structured.") contains the following entry ',tx\_result=OK', the desired key should be 'tx\_result', and so the name terminator character should be ',' vs. '\_' (in which case the key would have been named 'result'). ### :material-menu-right-outline:**`fieldValueBreaks`** KV field value terminator chars. |Type|Default| |---|---| |String|/\\{}()\[\]| Controls which characters found to the right of a token whose is a candidate for being a 'value' in a KV field formation should serve as a terminator for the search. For example, for an object whose [text](https://doc.log10x.com/api/js/#TenXBaseObject+text "The content of the event from which this instance was structured.") field contains the following entry: 'status=RESULT\_SUCCESS' the desired KV value should be 'RESULT\_SUCCESS', and as such the value terminator character should be ',' vs. '\_' (in which case the value would be 'RESULT').
:material-github: This unit is defined in [fields/unit.yaml](https://github.com/log-10x/modules/tree/main/pipelines/run/units/transform/fields/unit.yaml "fields/unit.yaml"){target="\_blank"}.