--- title: "Symbol Library Linker" description: "link symbol unit files to an output symbol library file" source: "https://github.com/log-10x/modules/tree/main/pipelines/compile/units/link/unit.yaml" icon: "material/merge" --- Combine symbol units into a single output symbol library file used by the 10x [run](https://doc.log10x.com/run/) pipeline to generate shared [TenXTemplates](https://doc.log10x.com/run/transform/ "Transform log and trace events into well-defined TenXObjects") schemas for input log/trace events. This phase is analogous to a [linker](){target="\_blank"} that takes one or more object files (i.e., symbol units) generated during a compilation process and combines them into a single library file for use at run time. ### :material-zip-box-outline: Symbol Library To dynamically create class and metadata information at run time, the 10x [compile](https://doc.log10x.com/compile/) pipeline scans GitHub and container repos ahead-of-time to generate [symbol libraries](https://doc.log10x.com/run/symbol/ "Import symbol libraries from files to transform events into well-defined TenXObjects"), similar to package artifacts (e.g., npm, .jar, .NET assembly). The 10x [run](https://doc.log10x.com/run/) pipeline utilizes symbol library files to assign a shared [TenXTemplate](https://doc.log10x.com/run/template/ "Import joint JSON schemas files to expand events into well-defined TenXObjects.") (i.e., _hidden class_) to input log/trace events with the same [structure](https://doc.log10x.com/run/transform/structure "Structure input events into well-defined TenXObjects"), creating a cached optimized schema for each event type. Operating on typed TenXObjects enables direct access to [symbol](https://doc.log10x.com/run/transform/structure/#symbols) and [variable](https://doc.log10x.com/run/transform/structure/#variables) values at runtime without repeatedly parsing JSON structures or evaluating complex, brittle regular expressions for each event. ### :material-file-tree: File Structure Each symbol library `.10x.tar` archive file contains: - A `.10x.json` file combining the symbol unit files generated during the [scan](https://doc.log10x.com/compile/scan/ "Capture symbol values from source code/binary files and link to an output symbol library") phase. - A `.10x.pb` Protocol Buffer file which provides a reverse in-mem index of the symbols values contained within the JSON for fast loading and random access. To learn more see the [.proto IDL](https://github.com/log-10x/pipeline-extensions/blob/main/api-extensions/src/main/proto/FsSymbolUnitsIndex.proto){target="\_blank"}. ## :material-wrench-outline: Config Files To configure the Symbol library linker unit, [:material-cog: Edit](https://doc.log10x.com/config/app/#module-config "Learn how to edit app and module configurations") these files. Below is the default configuration from: [link/config.yaml](https://github.dev/log-10x/config/blob/main/pipelines/compile/link/config.yaml "link/config.yaml"){target="\_blank"}.

Edit Online

```yaml # 🔟❎ 'compile' merge configuration # Combine symbol units into a single output symbol library file. # To learn more see https://doc.log10x.com/compile/link/ # Set the 10x pipeline to 'compile' tenx: compile # ============================ Symbol Library Output ========================== # 'outputSymbolLibraryFile' specifies location of output symbol library file which # links the contents of 'outputSymbolFolder' to a single .json artifact and a # compressed .zip version for use at run time by 10x 'run' pipelines. # If a local output dir exists write to it, otherwise use the pipeline's temp folder. # To learn more see https://doc.log10x.com/compile/link outputSymbolLibraryFile: $=TenXEnv.get("TENX_OUTPUT_SYMBOL_LIBRARY_FILE", path("data/shared/symbols", "") + "/" + TenXEnv.get("runtimeName", "symbols") + ".10x.tar") # ============================ Symbol Filter options ========================== # The options below control which symbol files and values are merged into 'outputSymbolLibraryFile' symbol: # 'fileNameFilters', exclude test files from symbol library fileNameFilters: # ── Java ─────────────────────────────────────── - ^.*(Test|Tests|TestCase|TestSuite)\.java$ # *Test.java, *TestCase.java, *TestSuite.java - ^Test.*\.java$ # TestFoo.java (legacy) # ── JavaScript / TypeScript ──────────────────── - ^.*\.(test|spec)\.(js|jsx|ts|tsx|mjs|cjs)$ # *.test.js, *.spec.tsx # ── Python ───────────────────────────────────── - ^test_.+\.py$ # test_utils.py - ^.+_test\.py$ # utils_test.py - ^test(s|case|cases|suite|suites)?\.py$ # test.py, tests.py, testcase.py, testcases.py, testsuite.py # ── Scala ────────────────────────────────────── - ^.*(Spec|Test|Suite)\.scala$ # *Spec.scala, *Test.scala, *Suite.scala # ── C# ───────────────────────────────────────── - ^.*(Test|Tests|TestCase|TestSuite)\.cs$ # *Test.cs, *TestCase.cs, *TestSuite.cs # ── Go ───────────────────────────────────────── - ^.*_test\.go$ # *_test.go # Note: testdata.go is usually a data file, not test code → not filtered here # ── C++ ─────────────────────────────────────── - ^.*(_test|Test|Tests)s?\.(cpp|cc|cxx|hpp|h|hxx)$ # *_test.cpp, *Test.hpp, *Tests.cpp # 'folderFilters', patterns to exclude files by folder # Plain name (e.g., "test") → matches direct parent folder only # Regex (e.g., "[/\\]vendor[/\\]") → matches anywhere in full path folderFilters: - test # direct parent is "test" - tests # direct parent is "tests" - __tests__ # Jest convention - spec # RSpec/Jasmine convention - specs # RSpec/Jasmine convention - unit # unit test folder - integration # integration test folder - e2e # e2e tests (Cypress, Playwright) - end-to-end # alternate e2e naming - acceptance # acceptance test folder - testdata # Go test data folder - fixtures # test fixtures folder - mocks # mock implementations folder - "[/\\\\]vendor[/\\\\][a-z]+\\.[a-z]" # Go vendored deps (vendor/domain.tld pattern) # 'types' controls the types of symbols written into 'outputSymbolLibraryFile' # for reflective use at runtime. # Narrowing the this list reduces the size of the output 10x symbols file. Possible values: types: - class # source code class/interface name (e.g. 'AcmeLogic') - enum # source code enum/literal name (e.g. 'severity: {error,warn,debug}') - log # source code print format passed to a logging method/stream (e.g. 'err("could not connect to %s",..'); - exec # string value extracted from a binary executable input file (e.g. via 'strings' nix command) # - method # code method/function name (e.g. 'foo') # - const # string constant assigned into a variable or method arg (e.g. 'greeting = "hello world"') # - text # string value, usually read from a config file (e.g. JSON/XML/YAML field name) # - package # source code package/namespace/module name (e.g. 'com.acme.core') ``` ## :material-menu: Options Specify the options below to [configure](/config "configure") the Symbol library linker: |Name|Description| |---|---| |[outputSymbolLibraryFile](#outputsymbollibraryfile "location of output symbol library file")|Location of output symbol library file| |[maxSymbolUnitFileSize](#maxsymbolunitfilesize "max file size for 'outputSymbolLibraryFile' before it is split into chunks")|Max file size for 'outputSymbolLibraryFile' before it is split into chunks| |[symbolTypes](#symboltypes "token types read from input source/binary files written to output")|Token types read from input source/binary files written to output| |[symbolFileNameFilters](#symbolfilenamefilters "regex patterns to exclude symbol files from output symbol library")|Regex patterns to exclude symbol files from output symbol library| |[symbolFolderFilters](#symbolfolderfilters "patterns to exclude symbol files by folder from output symbol library")|Patterns to exclude symbol files by folder from output symbol library| |[mergeExistingUnits](#mergeexistingunits "merge symbol unit files already on disk that this run did not itself scan")|Merge symbol unit files already on disk that this run did not itself scan| ### :material-menu-right-outline:**`outputSymbolLibraryFile`** Location of output symbol library file. |Type|Default| |---|---| |File|| Specifies an output symbol library file merging the symbol unit files produced for [inputPaths](https://doc.log10x.com/compile/scan/#inputpaths "input folders to scan"). The symbol library file provides a single artifact to set as the [symbolPaths](https://doc.log10x.com/run/symbol/#symbolpaths "input symbol files and folders") at run time. ### :material-menu-right-outline:**`maxSymbolUnitFileSize`** Max file size for 'outputSymbolLibraryFile' before it is split into chunks. |Type|Default| |---|---| |String|40MB| Sets the maximum file size (e.g., 40MB) for [outputSymbolLibraryFile](https://doc.log10x.com/compile/link/#outputsymbollibraryfile "location of output symbol library file") before splitting into chunks, each weighing no more than this value. This option enables large artifacts to remain [GitHub committable](https://docs.github.com/en/repositories/working-with-files/managing-large-files/about-large-files-on-github){target="\_blank"}. ### :material-menu-right-outline:**`symbolTypes`** Token types read from input source/binary files written to output. |Type|Default| |---|---| |List|\[package, class, enum, log, text, exec\]| Filter symbol values to write to [outputSymbolLibraryFile](https://doc.log10x.com/compile/link/#outputsymbollibraryfile "location of output symbol library file") based on their [context](https://doc.log10x.com/run/transform/symbol/#contexts). This option reduces the size of the output symbol library file by omitting symbol types not required at run time. For possible values, see: [contexts](https://doc.log10x.com/run/transform/symbol/#contexts). ### :material-menu-right-outline:**`symbolFileNameFilters`** Regex patterns to exclude symbol files from output symbol library. |Type|Default| |---|---| |List|\[\]| Filters symbol files to exclude from [outputSymbolLibraryFile](https://doc.log10x.com/compile/link/#outputsymbollibraryfile "location of output symbol library file") using regex patterns. This option is typically used to exclude test files from the symbol library, reducing output size and focusing on production code symbols. Common patterns for excluding test files across languages: - Java: `^.*(Test|Tests|TestCase)\.java$` (JUnit/TestNG test files) - JavaScript/TypeScript: `^.*\.(test|spec)\.(js|jsx|ts|tsx|mjs|cjs)$` (Jest/Mocha/Vitest test files) - Python: `^test_.+\.py$` or `^.+_test\.py$` (pytest/unittest test files) - Scala: `^.*(Spec|Test)\.scala$` (ScalaTest/Specs2 test files) - C#: `^.*(Test|Tests)\.cs$` (xUnit/NUnit/MSTest test files) - Go: `^.*_test\.go$` (built-in testing framework test files) - C++: `^.*(_test|Test)s?\.(cpp|cc|cxx|hpp|h|hxx)$` (GoogleTest/Catch2/Boost.Test test files). ### :material-menu-right-outline:**`symbolFolderFilters`** Patterns to exclude symbol files by folder from output symbol library. |Type|Default| |---|---| |List|\[\]| Filters symbol files to exclude from [outputSymbolLibraryFile](https://doc.log10x.com/compile/link/#outputsymbollibraryfile "location of output symbol library file") based on folder location. This option complements [symbolFileNameFilters](https://doc.log10x.com/compile/link/#symbolfilenamefilters "regex patterns to exclude symbol files from output symbol library") by excluding files based on their containing folder. **Plain folder name** (e.g., `test`) - matches when the file's direct parent folder has this name. Only alphanumeric characters, underscores, and hyphens are treated as plain names. **Regex pattern** (anything else) - matched against the full file path (case-insensitive). Use `[/\\]` for cross-platform path separators (Unix `/` and Windows `\`). **Examples:** - `test` - Files directly inside a `test` folder - `__tests__` - Files directly inside `__tests__` (Jest convention) - `[/\\]vendor[/\\]` - Files anywhere under a `vendor` folder (Go vendored deps) - `[/\\]node_modules[/\\]` - Files anywhere under `node_modules`. ### :material-menu-right-outline:**`mergeExistingUnits`** Merge symbol unit files already on disk that this run did not itself scan. |Type|Default| |---|---| |Boolean|false| Merges symbol unit files already present under [outputSymbolFolder](https://doc.log10x.com/compile/scan/#outputsymbolfolder "location of output symbol folder") that the current run did not itself scan, so a folder of pre-compiled units can be linked into a [symbol library](https://doc.log10x.com/compile/link/#outputsymbollibraryfile "location of output symbol library file") with no source scan. Default false: only units this run scanned are merged.
:material-github: This unit is defined in [link/unit.yaml](https://github.com/log-10x/modules/tree/main/pipelines/compile/units/link/unit.yaml "link/unit.yaml"){target="\_blank"}.