--- title: "Pipeline Monitor" description: "emit pipeline debug values to log and the 10x time series backend" source: "https://github.com/log-10x/modules/tree/main/pipelines/shared/units/monitor/unit.yaml" icon: "material/heart-cog-outline" --- Logs diagnostics and publishes health metrics from pipeline [units](https://doc.log10x.com/engine/pipeline/#units). Each unit emits counters (scanned symbols, input events) and states (scan errors), periodically logged and published to the [metric output](https://doc.log10x.com/run/output/metric/ "Write aggregated TenXSummary instances to metric outputs (e.g., Prometheus, Datadog)") the pipeline configures. ## :material-wrench-outline: Config Files To configure the Pipeline monitor unit, [:material-cog: Edit](https://doc.log10x.com/config/app/#module-config "Learn how to edit app and module configurations") these files: ??? tenx-run "Run" ### Run {.hidden-heading} Launch a [run pipeline](https://doc.log10x.com/run "run pipeline") to report, receive and optimize log/trace events in edge and cloud environments. Below is the default configuration from: [monitor/config.yaml](https://github.dev/log-10x/config/blob/main/pipelines/run/monitor/config.yaml "monitor/config.yaml"){target="\_blank"}.

Edit Online

```yaml # 🔟❎ 'run' monitor configuration # # Emit diagnostic information and metrics to the 10x console and log. # To learn more see https://doc.log10x.com/run/monitor/ # Set the 10x pipeline to 'run' tenx: run # ============================== Monitor Options ================================ monitor: # 'pollInterval' sets the frequency by which to emit debug information # for the target units specified below. pollInterval: 1s # 'metricPattern' specifies a regex pattern to match against counters generated # by the specified 'units' below to publish as metrics to the 10x console metricPattern: null # 'units' specifies the run pipeline units to emit period diagnostic # information to the 10x log and console. Possible values: units: # load symbol library file(s) # - symbolLoader # load TenXTemplates file(s) # - templateLoader # load hash values of known TenXTemplates from file(s) # - loadTemplateHashes # read input log/trace event from text content passes a launch argument # - readInline # read input log/trace event from a file # - readFile # read input log/trace event from a java.io.InputStream/Reader # - readStream # split transform workload across multiple cores # - parallelize # transform input events into structured well-defined TenXObjects # - transform # group TenXObjects into composite logical units # - group # aggregate TenXObjects into summary instances # - aggregateSingle # write TenXObjects to java.io.OutputStream/Writer(s) # - streamOutput # write TenXObjects to stdout/err # - stdOutput # write TenXObjects to log4j2 appenders # - logOutput # write TenXObjects to Micrometer registries # - metricOutput ``` ??? tenx-compile "Compile" ### Compile {.hidden-heading} Launch a [compile pipeline](https://doc.log10x.com/compile "compile pipeline") to scan source code and binary files to generate symbol unit and library files. Below is the default configuration from: [monitor/config.yaml](https://github.dev/log-10x/config/blob/main/pipelines/compile/monitor/config.yaml "monitor/config.yaml"){target="\_blank"}.

Edit Online

```yaml # 🔟❎ 'compile' monitor configuration # Emit diagnostic information and metrics to the 10x console and log. # To learn more see https://doc.log10x.com/compile/monitor/ # Set the 10x pipeline to 'compile' tenx: compile # ============================== Monitor Options ================================ monitor: # 'pollInterval' sets the frequency by which to emit debug information # for the target units specified below. pollInterval: null # 'metricPattern' specifies a regex pattern to match against counters generated # by the specified 'units' below to publish as metrics to the 10x console metricPattern: null # 'units' specifies the compile pipeline units to emit period diagnostic # information to the 10x log and console. Possible values: units: # Push output symbol files to a GitHub repository - githubPushFolder # discover input source code/binary files to scan on disk - discoverSources # scan input files to generate output symbol unit files - scanToFolder # launch a subprocess to scan target input files - subprocessScan # link symbol unit files into a symbol library file for use at run time - link ``` ## :material-menu: Options Specify the options below to [configure](/config "configure") the Pipeline monitor: |Name|Description| |---|---| |[monitorPollInterval](#monitorpollinterval "interval to information to log/ publish counters")|Interval to information to log/ publish counters| |[monitorUnits](#monitorunits "pipeline unit names to log and emit debug metrics")|Pipeline unit names to log and emit debug metrics| |[monitorMetricPattern](#monitormetricpattern "regexp pattern to match against counters to publish as metrics")|Regexp pattern to match against counters to publish as metrics| ### :material-menu-right-outline:**`monitorPollInterval`** Interval to information to log/ publish counters. |Type|Default| |---|---| |String|1s| Sets the frequency to log debug information for pipeline units specified by 'monitorUnits' to log/time series backend. ### :material-menu-right-outline:**`monitorUnits`** Pipeline unit names to log and emit debug metrics. |Type|Default| |---|---| |List|\[\]| Specifies which 10x pipeline units will log and emit debug metrics while executing. Each pipeline prints the names of its configured units upon startup on the console and log file. To emit information for all pipeline units, specify '\*'. ### :material-menu-right-outline:**`monitorMetricPattern`** Regexp pattern to match against counters to publish as metrics. |Type|Default| |---|---| |String|""| Defines a pattern to match against counters debug produced by units specified by [monitorUnits](https://doc.log10x.com/shared/monitor/#monitorunits). Each counter provides a name in the following format: `_` (e.g., input\_events\_0). The pipeline will emit counters matching the pattern, [tagging each counter](https://www.baeldung.com/micrometer#1-tags){target="\_blank"} with information about the reporting unit instance (e.g., the destination for a 'run' pipeline output).
:material-github: This unit is defined in [monitor/unit.yaml](https://github.com/log-10x/modules/tree/main/pipelines/shared/units/monitor/unit.yaml "monitor/unit.yaml"){target="\_blank"}.