--- title: "Compiler" description: "Scans source code to generate symbol libraries that map log events to their source locations" source: "https://github.com/log-10x/modules/tree/main/apps/compiler/app.yaml" icon: "material/cogs" --- The compiler app produces symbol library files that enable the 10x runtime apps to [transform](https://doc.log10x.com/run/transform/ "Transform log and trace events into well-defined TenXObjects") input events into well-defined TenXObjects. The compiler scans GitHub repos and container images to find symbol values (e.g., log statements, class names, enums), capturing their source file and context within it. The linker packages this information into [symbol library](https://doc.log10x.com/compile/link/#symbol-library) files, similar to package artifacts (e.g., npm, .jar, .NET assembly). Compile custom symbols if: - Using 3rd party frameworks not already in the [default library](https://doc.log10x.com/compile/pull/#default-symbols) - Want symbols from your specific application code - Using proprietary or internal logging frameworks ## :material-piggy-bank-outline: Benefit 10x apps utilize symbol library files at runtime to assign a shared [TenXTemplate](https://doc.log10x.com/engine/design/#aot-compiler) (i.e., _Hidden class_) to incoming log events, creating a shared schema structure for each event type. Operating on well-defined objects enables direct access to [symbol](https://doc.log10x.com/run/transform/structure/#symbols) and [variable](https://doc.log10x.com/run/transform/structure/#variables) values, eliminating the overhead of parsing repetitive JSON structures and evaluating brittle, computationally expensive regex patterns for each event. !!! l1x-gift "Out-of-the-Box Ready" The 10x runtime includes a built-in symbol library with [industry-standard frameworks](https://doc.log10x.com/compile/pull/#default-symbols) (Kubernetes, OpenTelemetry, Spring Boot, Django, and 30+ others). You can immediately run a local cost POC with [OpenTelemetry sample files](https://log10x-public-assets.s3.amazonaws.com/samples/otel-k8s/medium/input/otel-sample.log) or try it with your own logs for out-of-the-box optimization, even without custom symbol compilation. ## :material-cog-transfer-outline: Workflow The Compiler app processes source code and binaries to generate symbol libraries that enable intelligent log processing and optimization.
```mermaid graph LR A["
🔄 CI/CD
Trigger Process
"] --> B["
📥 Pull
Fetch Sources
"] B --> C["
🔍 Scan
Extract Symbols
"] C --> D["
🔗 Link
Build Library
"] D --> E["
📤 Push
Commit Output
"] E --> F["
📦 Distribute
to Runtime Apps
"] classDef cicd fill:#7c3aed88,stroke:#6d28d9,color:#ffffff,stroke-width:2px,rx:8,ry:8 classDef pull fill:#2563eb88,stroke:#1d4ed8,color:#ffffff,stroke-width:2px,rx:8,ry:8 classDef scan fill:#059669,stroke:#047857,color:#ffffff,stroke-width:2px,rx:8,ry:8 classDef link fill:#ea580c88,stroke:#c2410c,color:#ffffff,stroke-width:2px,rx:8,ry:8 classDef push fill:#16a34a88,stroke:#15803d,color:#ffffff,stroke-width:2px,rx:8,ry:8 classDef distribute fill:#f59e0b,stroke:#d97706,color:#ffffff,stroke-width:2px,rx:8,ry:8 class A cicd class B pull class C scan class D link class E push class F distribute ```
🔄 **Invoke**: runs locally for development, periodically as k8s CronJob for continuous updates, or integrated as part of CI/CD pipelines (GitHub Actions) for automated symbol generation 📥 **Pull**: Fetches source code from [GitHub](https://doc.log10x.com/compile/pull/github/), [Docker](https://doc.log10x.com/compile/pull/docker/), [Helm](https://doc.log10x.com/compile/pull/helm/), and existing symbol files for reuse 🔍 **Scan**: Extracts [symbol values](https://doc.log10x.com/compile/scan/) from source code and binaries across multiple programming languages to create symbol unit files 🔗 **Link**: Combines symbol unit files into a single [symbol library](https://doc.log10x.com/compile/link/#symbol-library) file for runtime use by the 10x apps 📤 **Push**: Commits symbol library files to [GitHub](https://doc.log10x.com/compile/push/) for distribution to runtime apps and reuse in future compilation runs 📦 **Distribute**: 10x apps pull symbol libraries via the [`@github` launch macro](https://doc.log10x.com/config/github/) at startup and poll the remote branch for changes (e.g., every 5 minutes). Between compilation runs, events are processed using the previous symbol library version, slightly less efficient for new log patterns until the next run picks up the changes automatically. ## :material-hexagon-multiple-outline: Architecture === ":material-cogs: Compile Phase" The compiler executes periodically (k8s CronJob) or as a CI/CD pipeline (GitHub Actions) step to pull modified repos, process changes, and push updated symbol files to GitHub.
![Architecture diagram: Compiler pulls source code and binary repos, generates symbol library files, and pushes them to GitHub](../../assets/tenx-compile-aot.svg){ align=left }
⚙️ Pull source code and binary repos to generate [symbol library](https://doc.log10x.com/compile/link/#symbol-library) files
=== ":material-play: Runtime Phase" 10x runtime apps, [Reporter](https://doc.log10x.com/apps/reporter/), [Receiver](https://doc.log10x.com/apps/receiver/), [Retriever](https://doc.log10x.com/apps/retriever/), pull symbol libraries when they start up or when libraries are updated.
![Architecture diagram: Runtime apps pull symbol libraries from GitHub at startup and poll for updates](../../assets/tenx-compile.svg){ align=left }
▶️ 10x [apps](https://doc.log10x.com/apps/) pull symbol libraries for structured event processing
See the [Compiler FAQ](https://doc.log10x.com/compile/faq/) for complete details on symbol discovery, compilation workflows, and troubleshooting.
:material-github: This app is defined in [compiler/app.yaml](https://github.com/log-10x/modules/tree/main/apps/compiler/app.yaml "compiler/app.yaml"){target="\_blank"}.