---
icon: material/pipe
---

Each 10x Engine instance executes a [configurable](https://doc.log10x.com/config/) pipeline to read, process and output data:

<div class="grid cards" markdown>

-   :material-code-tags:{ .lg .middle } __Compile__

    ---

    Scan source code and binary repos to generate symbol library files.

    [:octicons-arrow-right-24: Learn more](https://doc.log10x.com/compile/)

-   :material-play:{ .lg .middle } __Run__

    ---

    Transform event stream into TenXObjects for edge/cloud processing.

    [:octicons-arrow-right-24: Learn more](https://doc.log10x.com/run/)

</div>

## :material-pipe-disconnected: Units

Each pipeline is composed of `units` which are runtime components that read, process and output data. 
Units are chained together to form a pipeline.  

Each pipeline unit defines its configurable values via a [module file](https://doc.log10x.com/config/module/).

For example, the compile [scan unit](https://doc.log10x.com/compile/scan/) provides settings for extracting
symbol values from source code/binary files, while the run [input unit](https://doc.log10x.com/run/input/stream/) provides settings for reading log/trace events. 

The 10x Engine supports the following unit types:

| Type          | Description                                    | Compile Example                                          | Run Example                                        |
|---------------|------------------------------------------------|----------------------------------------------------------|----------------------------------------------------|
| **Provider**  | Loads local/remote resources into the pipeline | [GitHub](https://doc.log10x.com/compile/pull/github/)     | [Symbol](https://doc.log10x.com/run/symbol/)        |
| **Producer**  | Read input data into the pipeline              | [Source](https://doc.log10x.com/compile/sources/)         | [Input](https://doc.log10x.com/run/input/)          |
| **Processor** | Process input data into output objects         | [Scan](https://doc.log10x.com/compile/scan/)              | [Transform](https://doc.log10x.com/run/transform/)  |
| **Publisher** | Write output objects to local/remote output    | [Link](https://doc.log10x.com/compile/link/)            | [Output](https://doc.log10x.com/run/output/)        |
| **Observer**  | Observer and monitor pipeline execution        | [Console](https://doc.log10x.com/compile/console/)        | [Monitor](https://doc.log10x.com/run/monitor/)      |

Upon initialization the 10x Engine scans its [include paths](https://doc.log10x.com/run/bootstrap/#includepaths) folder for sub-folders containing pipeline definition files.

A pipeline will run until all `producer` units have completed, at which point it will signal all units
to begin a shutdown process where each unit flushes its remaining data and the process terminates.