---
title: "Custom Output"
description: "write TenXObjects to custom event and metric destinations."
source: "https://github.com/log-10x/modules/tree/main/pipelines/run/units/output/stream/unit.yaml"
icon: "material/waves-arrow-right"

---
Defines output destinations for [TenXObjects](https://doc.log10x.com/api/js/#TenXObject "Provide structured, reflective access to log/trace events read from input(s)."), specifying location and [compact](https://doc.log10x.com/run/transform/#compact) format. Pipelines support multiple streams.

### :octicons-package-dependents-24: Extensions

Most users work with [output modules](https://doc.log10x.com/engine/module/) rather than raw streams. Modules provide higher-level constructs for serialization via Java [streams](https://www.baeldung.com/java-outputstream){target="\_blank"} and log4j2 [appenders](https://logging.apache.org/log4j/2.x/manual/appenders.html){target="\_blank"}.

See [output extensions](https://doc.log10x.com/api/output/).

### :octicons-package-24: Output Modules

Output [Modules](https://doc.log10x.com/engine/module/) package stream configuration, JavaScript and documentation files for writing TenXObjects to edge and cloud data destinations which include:

- [:simple-fluentd: Forwarders](https://doc.log10x.com/run/input/forwarder/ "Report, receive and optimize events collected by log forwarders") (Fluentd/Bit) - receive and optimize before shipping
- [:simple-prometheus: Time-series](https://doc.log10x.com/run/output/metric/ "Write aggregated TenXSummary instances to metric outputs (e.g., Prometheus, Datadog)") (Prometheus, Datadog) - publish [TenXSummary](https://doc.log10x.com/api/js/#TenXSummary "Access aggregate values of tenxObjects which share a target set of field values.") metrics
- [:material-bucket-outline: Object Storage](https://doc.log10x.com/run/input/objectStorage/index "Index files uploaded to a object storage container (e.g. AWS S3 bucket).") (S3, Azure Blobs) - index for in-place querying

## :material-menu: Options

Specify the options below to [configure](/config "configure") multiple Custom output:

|Name|Description|Category|
|---|---|---|
|[outputEnabled](#outputenabled "a JavaScript expression that must be evaluated as 'truthy' to enable this output")|A JavaScript expression that must be evaluated as 'truthy' to enable this output|General|
|[outputName](#outputname "logical name for the output stream")|Logical name for the output stream|General|
|[outputFilter](#outputfilter "a JavaScript expression an TenXObject must evaluate as 'truthy' against to be encoded")|A JavaScript expression an TenXObject must evaluate as 'truthy' against to be encoded|General|
|[outputType](#outputtype "type of destination output. Possible values:[file, stdout, event, metric, stream]")|Type of destination output. Possible values:\[file, stdout, event, metric, stream\]|General|
|[outputPath](#outputpath "logical output destination")|Logical output destination|General|
|[outputArgs](#outputargs "custom arguments passed to OutputStream/Writer instance")|Custom arguments passed to OutputStream/Writer instance|General|
|[outputFields](#outputfields "a comma-delimited list of TenXObject fields to encode")|A comma-delimited list of TenXObject fields to encode|Encode|
|[outputEncodeType](#outputencodetype "output format for encoding TenXObject values. Possible values:[json, delimited]")|Output format for encoding TenXObject values. Possible values:\[json, delimited\]|Encode|
|[outputEncodedLinePrefix](#outputencodedlineprefix "char line prefix to prepend when encoding delimited TenXObject field values")|Char line prefix to prepend when encoding delimited TenXObject field values|Encode|
|[outputEncodeDelimiter](#outputencodedelimiter "char delimiter to use when separating encoded delimited TenXObject field values")|Char delimiter to use when separating encoded delimited TenXObject field values|Encode|
|[outputEncodeGroupDelimiter](#outputencodegroupdelimiter "delimiter to separate TenXObjects encoded into as a group")|Delimiter to separate TenXObjects encoded into as a group|Encode|
|[outputAsync](#outputasync "determines whether encoding of this out is done in a dedicated thread")|Determines whether encoding of this out is done in a dedicated thread|Advanced|
|[outputForeach](#outputforeach "name of module options group for whose instances to create matching outputs")|Name of module options group for whose instances to create matching outputs|Advanced|
|[outputGroup](#outputgroup "option names whose values for grouping/synchronizing output streams")|Option names whose values for grouping/synchronizing output streams|Advanced|

### General

#### :material-menu-right-outline:**`outputEnabled`**

A JavaScript expression that must be evaluated as 'truthy' to enable this output.

|Type|Default|Category|
|---|---|---|
|String|""|General|

Enables/disables this output stream. If set, the JavaScript expression must be truthy
to open the output.


#### :material-menu-right-outline:**`outputName`**

Logical name for the output stream.

|Type|Required|Category|
|---|---|---|
|String|✔|General|

Defines a unique logical name for the output (e.g., `datadog`).
Each TenXObject will return this value via its [outputName](https://doc.log10x.com/api/js/#TenXOutput+outputName "Provides a logical name for the output instance as defined by its declaring module.") field.


#### :material-menu-right-outline:**`outputFilter`**

A JavaScript expression an TenXObject must evaluate as 'truthy' against to be encoded.

|Type|Default|Category|
|---|---|---|
|String|""|General|

Defines a JavaScript expression an TenXObject must evaluate as truthy against
to be written the target output. This filter operates in an AND relationship with the
[outputFilters](https://doc.log10x.com/run/output/receive/#outputfilters "JavaScript expressions an TenXObject must evaluate as truthy against to write to output(s)") receiver argument (i.e., both must be evaluated as truthy to pass).


#### :material-menu-right-outline:**`outputType`**

Type of destination output. Possible values:\[file, stdout, event, metric, stream\].

|Type|Required|Category|
|---|---|---|
|String|✔|General|

Defines the mechanism for writing TenXObject, template and summary instance field values to this output.

Possible values:

- **file**: emit field values to a file on disk.

- **stdout**: emit field values to stdout/err device.

- **event**: utilizes the java log4j2 library to emit TenXObjects into a wide array of
  output destinations via log4j appenders (e.g., file, socket, http).
  To learn more see [appender](https://doc.log10x.com/run/output/event/process "Write TenXObject and template values to a sub-process stdin") outputs.

- **metric**: utilizes the [Micrometer library](https://micrometer.io/){target="\_blank"}  to
  emit \[TenXSummary\]([TenXObjects](https://doc.log10x.com/api/js/#TenXSummary "Access aggregate values of tenxObjects which share a target set of field values.") instances as metric counter to time series output destinations (e.g., Prometheus).
  To learn more see [metric outputs](https://doc.log10x.com/run/output/metric "Write aggregated TenXSummary instances to metric outputs (e.g., Prometheus, Datadog)").

- **stream**: instantiates a sub-class of [java.io.OutputStream/Writer](https://www.baeldung.com/java-outputstream){target="\_blank"}
  to write TenXObject field values to a user-provided stream.
  
  This method allows for programmatically defining output destinations.
  For an example, see the default [outputStream](https://github.com/log-10x/config/blob/main/pipelines/run/config/output/event/outputStream/config.yaml){target="\_blank"} configuration.


#### :material-menu-right-outline:**`outputPath`**

Logical output destination.

|Type|Required|Category|
|---|---|---|
|String|✔|General|

Defines the concrete destination for this output stream.

This value is dependent on that of [outputType](https://doc.log10x.com/run/output/stream/#outputtype "type of destination output. Possible values:[file, stdout, event, metric, stream]"). For:

- **file**:  location on disk to append/overwrite

- **event**: a .yaml or .xml file containing a [log4j2 configuration](https://logging.apache.org/log4j/2.x/manual/configuration.html){target="\_blank"} for a logger and appender(s)
  to log TenXObjects via the [Message interface](https://logging.apache.org/log4j/2.x/javadoc/log4j-api/org/apache/logging/log4j/message/Message.html){target="\_blank"}.

- **metric**: fully qualified Java class name implementing the [MetricRegistryFactory](https://github.com/log-10x/pipeline-extensions/blob/main/edge-extensions/src/main/java/com/log10x/ext/edge/micrometer/MetricRegistryFactory.java){target="\_blank"}
  interface to create a micrometer registry.

- **stream**: fully qualified class name derived from [OutputStream](https://docs.oracle.com/javase/8/docs/api/java/io/OutputStream.html){target="\_blank"} or
  [Writer](https://docs.oracle.com/javase/8/docs/api/java/io/Writer.html){target="\_blank"} to instantiate.
  The output will [flush](https://docs.oracle.com/javase/8/docs/api/java/io/Flushable.html#flush--){target="\_blank"} the stream/writer after serializing the values of [outputFields](https://doc.log10x.com/run/output/stream/#outputfields "a comma-delimited list of TenXObject fields to encode").
  for each TenXObject. It is up to the class to implement any required batching and buffering logic.
  An [EvaluatorBean](https://github.com/log-10x/pipeline-extensions/blob/main/edge-extensions/src/main/java/com/log10x/api/bean/EvaluatorBean.java){target="\_blank"} reference
  provides an interface to the 10x JavaScript engine.
  The class to instantiate must declare a constructor with one of the following signatures:
  
  ```java
  public class MyOutputStream extends OutputStream {
  
    // This signature is used (if declared)
    MyOutputStream(Map<String, Object> config, com.log10x.api.bean.EvaluatorBean bean) {...}
  
    // otherwise use this signature 
    MyOutputStream(Map<String, Object> config) {...}
  
    // If neither are declared, utilize a parameterless constructor
    MyOutputStream() {...}
  }
  ```
  
  If the search for a matching constructor fails, the pipeline halts.


#### :material-menu-right-outline:**`outputArgs`**

Custom arguments passed to OutputStream/Writer instance.

|Type|Default|Category|
|---|---|---|
|List|\[\]|General|

If [outputType](https://doc.log10x.com/run/output/stream/#outputtype "type of destination output. Possible values:[file, stdout, event, metric, stream]") is `stream`, this argument specifies a list of values pairs (key1,value1...) passed to the constructor of the
class specified by [outputPath](https://doc.log10x.com/run/output/stream/#outputpath "logical output destination").

A Map combining these values and those of the output's module option group
specified by the [outputForeach](https://doc.log10x.com/run/output/stream/#outputforeach "name of module options group for whose instances to create matching outputs") argument(if present) provide the
stream with module-specific context (e.g., auth tokens, host addresses).


### Encode

#### :material-menu-right-outline:**`outputFields`**

A comma-delimited list of TenXObject fields to encode.

|Type|Required|Category|
|---|---|---|
|List|✔|Encode|

Lists fields from each TenXObject to write to this output. Fields can be:

- [Intrinsic](https://doc.log10x.com/api/js/#TenXObject "Provide structured, reflective access to log/trace events read from input(s)."): Built into all TenXObject and summary instances.
- [Enriched](https://doc.log10x.com/run/initialize/ "Enrich TenXObjects with additional context"): Added via initializer modules (e.g., symbol, GeoIP, file lookups).
- [Extracted](https://doc.log10x.com/run/transform/fields/ "Scan TenXTemplates for JSON and KV fields"): Extracted from JSON/KV pairs in the log event.
- [Calculated](https://doc.log10x.com/run/transform/script/object/#enrich): Derived using custom JavaScript constructors.
- Expression: An inline JavaScript expression defining an output-only field used when writing instances to this output. The field name and value can be explicitly set (e.g., `myFoo=foo()`) or implicitly derived using the `$=` prefix (e.g., `$=foo()` sets the field name to `foo`).

Example:

```yaml
outputFile:
  path: out.log
  fields: 
  - myJsonExtField
  - myEnrichedField
  - myCalcField
  - myExpField=TenXEnv.get("var") ? "myCalcField":"foo()" 
  - encode()
```

Below are examples of TenXObject fields which can be encoded to output:

|Field|Description|Type|
|---|---|---|
|[encode()](https://doc.log10x.com/api/js/#TenXObject+encode "Encodes the current object's log event into a compact, template‐based string.")|[Losslessly compact](https://doc.log10x.com/run/transform/#compact) the instance. The field name is automatically set to `encode`|Expression|
|[fullText](https://doc.log10x.com/api/js/#TenXBaseObject+fullText "A text value extracted from the input stream from")|Full text of the underlying raw/expanded event|Intrinsic|
|[symbolSequence](https://doc.log10x.com/run/initialize/message/#sequence)|Metric-complaint symbol values sequence|Enriched|
|[symbolMessage](https://doc.log10x.com/run/initialize/message/#message)|Logical event message symbol values sequence|Enriched|
|[symbolOrigin](https://doc.log10x.com/run/initialize/message/#class)|Symbol value with [class context](https://doc.log10x.com/run/transform/symbol/#class)|Enriched|
|[namespace\_name](https://doc.log10x.com/run/initialize/k8s/#k8snamespacenamefield "K8s namespace target field name")|Kubernetes namespace context|Enriched|
|[container\_name](https://doc.log10x.com/run/initialize/k8s/#k8scontainernamefield "K8s container name target field name")|Kubernetes container context|Enriched|
|[pod\_name](https://doc.log10x.com/run/initialize/k8s/#k8spodnamefield "K8s pod name target field name")|Kubernetes pod context|Enriched|
|[city](https://doc.log10x.com/run/initialize/geoIP/#geoipcityfield "Field name for city geo-reference")|GeoIP reference lookup (city, country.. )|Enriched|
|[lookupValue](https://doc.log10x.com/run/initialize/lookup/#lookupvaluefield "TenXObject field name to use as the lookup value")|Text file (e.g., .CSV, .TSV) lookup|Enriched|
|[code](https://doc.log10x.com/run/transform/script/object/#enrich)|Calculated HTTP code field|Calculated|
|[template](https://doc.log10x.com/api/js/#TenXBaseObject+template "A sequence of all symbol and delimiter tokens from the object's text field.")|TenXTemplate representation (i.e., [_hidden class_](https://doc.log10x.com/engine/design/#optimization-model))|Intrinsic|
|[timestamp](https://doc.log10x.com/api/js/#TenXObject+timestamp "An array of UNIX epoch values of timestamps parsed from the object's text.")|64bit Unix epoch [timestamps](https://doc.log10x.com/run/transform/#timestamps)|Intrinsic|
|[vars](https://doc.log10x.com/api/js/#TenXBaseObject+vars "An array of  variable sequences extracted from the object's text.")|High-cardinality [variables](https://doc.log10x.com/run/transform/#variables)|Intrinsic|
|[ipAddress](https://doc.log10x.com/api/js/#TenXBaseObject+ipAddress)|Embedded IPv4 addresses|Intrinsic|


#### :material-menu-right-outline:**`outputEncodeType`**

Output format for encoding TenXObject values. Possible values:\[json, delimited\].

|Type|Default|Category|
|---|---|---|
|String|""|Encode|

Specifies how to format TenXObject field values to a stream/event output
Possible values:

- **json**: formats N specified [outputFields](https://doc.log10x.com/run/output/stream/#outputhttpfields) names and values for the target TenXObject as: `{"field":"str","field2":1, "field3": true, .. "fieldN": "value"}`.
- **delimited**: formats N specified [outputFields](https://doc.log10x.com/run/output/stream/#outputhttpfields) values for the target TenXObject as: `"str",1,true,..N-value`.
  The [outputEncodeDelimiter](https://doc.log10x.com/run/output/stream/#outputhttpencodedelimiter) option determines the value separator

NOTE: This argument does not apply to outputs whose [outputType](https://doc.log10x.com/run/output/stream/#outputtype "type of destination output. Possible values:[file, stdout, event, metric, stream]") is set to `metric`.


#### :material-menu-right-outline:**`outputEncodedLinePrefix`**

Char line prefix to prepend when encoding delimited TenXObject field values.

|Type|Default|Category|
|---|---|---|
|String|""|Encode|

Sets the char line prefix which will be prepended to TenXObject field values encoded to a stream/event output.
This argument only applies when [outputEncodeType](https://doc.log10x.com/run/output/stream/#outputencodetype "output format for encoding TenXObject values. Possible values:[json, delimited]") is set to `delimited`.


#### :material-menu-right-outline:**`outputEncodeDelimiter`**

Char delimiter to use when separating encoded delimited TenXObject field values.

|Type|Default|Category|
|---|---|---|
|String|""|Encode|

Sets the char delimiter for separate TenXObject field values encoded to a stream/event output.
This argument only applies when [outputEncodeType](https://doc.log10x.com/run/output/stream/#outputencodetype "output format for encoding TenXObject values. Possible values:[json, delimited]") is set to `delimited`.


#### :material-menu-right-outline:**`outputEncodeGroupDelimiter`**

Delimiter to separate TenXObjects encoded into as a group.

|Type|Default|Category|
|---|---|---|
|String|""|Encode|

Sets the string delimiter used to separate individual 10x events inside a group
when encoded to a stream/event output.


### Advanced

#### :material-menu-right-outline:**`outputAsync`**

Determines whether encoding of this out is done in a dedicated thread.

|Type|Default|Category|
|---|---|---|
|Boolean|true|Advanced|

Controls whether the output uses a dedicated thread for encoding TenXObjects or does so from the calling thread.


#### :material-menu-right-outline:**`outputForeach`**

Name of module options group for whose instances to create matching outputs.

|Type|Default|Category|
|---|---|---|
|String|""|Advanced|

Specifies the name of an options group for whose instances
to replicate this output object. This value enables 10x modules
to create multiple input instances, each receiving a unique set of arguments.

For an example, see [httpOutput](https://doc.log10x.com/run/output/event/http "Write TenXObject and template values to a log4j2 HTTP socket appender").


#### :material-menu-right-outline:**`outputGroup`**

Option names whose values for grouping/synchronizing output streams.

|Type|Default|Category|
|---|---|---|
|List|\[\]|Advanced|

Enables synchronizing different outputs pointing to a shared logical destination.

For example, in cases where multiple outputs are pointing at the same output file or stdout, it is desirable
to synchronize writing between the outputs to ensure that no 'interlacing'
of TenXObject values takes place during write. For this, any variables from the options group instance
defined by `outputForeach` can be used to define a unique 'key' for this output
to synchronize it with other outputs sharing the same value.

The example below, taken from: [file/stream.yaml](https://github.com/log-10x/modules/blob/main/pipelines/run/modules/output/event/file/stream.yaml#L38){target="\_blank"}
groups all log4j appenders writing to the same output file
via their associated 'fileOutput' options group instance's 'outputFilePath' value.

```yaml
- type: event
  foreach: fileOutput

  path: run/modules/output/event/file
  group:
    - $outputFilePath
```

Specify `*` to group by all values of the option group instance.


<br/>:material-github: This unit is defined in [stream/unit.yaml](https://github.com/log-10x/modules/tree/main/pipelines/run/units/output/stream/unit.yaml "stream/unit.yaml"){target="\_blank"}.