--- title: "Symbol Message Calculator" description: "enrich TenXObjects with logical message symbol sequence and origin values" source: "https://github.com/log-10x/modules/tree/main/pipelines/run/modules/initialize/message/module.yaml" icon: "material/origin" --- Adds a name to every event, made of the known words of the log statement that most likely wrote it, such as `Receive_ListRecommendations_for_product_ids`. Events from one statement share that name whatever IDs or timestamps they carry, unless a value that is itself a known word splits them. The [Reporter](https://doc.log10x.com/apps/reporter/ "Report on costly log/trace events _before_ they ship to output") reports cost by this name, the [Receiver](https://doc.log10x.com/apps/receiver/ "Receive log/trace events from a coupled forwarder; in read-only mode emit metrics only, in read-write mode also receive and optionally compact events before they ship to output") samples or mutes by it, and [metric outputs](https://doc.log10x.com/run/output/metric/ "Write aggregated TenXSummary instances to metric outputs (e.g., Prometheus, Datadog)") publish counts per name. ## :material-tag-outline: Fields { #pattern-identity-pattern-vs-template } The module writes these fields on each event: | Field | Example | Option | |---|---|---| | `message_pattern` | `Receive_ListRecommendations_for_product_ids` | [`symbolMessageField`](#symbolmessagefield "field that holds the name"), on in the shipped configuration | | Pattern hash | An 11-character hash of the name, for joins and metric tags | [`symbolMessageHashField`](#symbolmessagehashfield "field that holds a hash of the name"), off by default; metrics carry it either way | | Origin | `recommendation_server.py:ListRecommendations` | [`symbolOriginField`](#symboloriginfield "field that holds the statement's source location"), off by default | | Skeleton | The message with `~` for each word of the name and `$` for each value | [`symbolSkeletonField`](#symbolskeletonfield "field that holds the message layout"), off by default | A template is the exact shape of one line format, which [compact](https://doc.log10x.com/run/transform/#compact) uses to store events. One log statement can produce several templates. Templates of one statement that differ only in values outside the library, such as timestamps and numbers, share one name; a value made of library words is read as part of the name, as the browser names are in the Web example below. In the engine's metrics, `tenx_hash` is the pattern hash. On a Splunk compact event, the field of that name holds the template hash. ## :material-target: Message Extraction The module names each event in five steps, run in this order: ??? tenx-extractors "1. Pick the text" The module reads the field named by `inputField`, such as `log`, so envelope keys such as `stream` and `kubernetes` stay out of the name. With several fields listed, it reads the first one the event has. For a multi-line event, it reads the first line that holds a message. In a stack trace that is the exception line. The .NET console logger writes `info: cart.cartstore.ValkeyCartStore[0]` and then `GetCartAsync called with userId=8d3b2c1a` as two records, and the second one names the event. ??? tenx-symbolinitializer "2. Find the source" Each phrase the [symbol library](https://doc.log10x.com/compile/link/#symbol-library) knows points to the places in code that can print it. The module picks the place that best explains the line, comparing in this order: 1. Words from the message itself. A phrase matched only inside a JSON value or brackets ranks lower, so a long error string carried in an attribute does not outweigh the short message it belongs to. 2. The length of the longest phrase that place prints. 3. How many distinct words of the line that place explains, and their total length. Any remaining tie resolves the same way on every run and every node. ??? tenx-safety "3. Check the evidence" The source is accepted when at least two words of the message match it; otherwise the event is named by step 5 and gets no origin. A path such as `/var/lib/app` counts as one word, and parts of IDs do not count. ??? tenx-text "4. Build the name" The name is the matched words in the order they appear, joined with `_`, up to [`symbolMaxLen`](#symbolmaxlen "longest name, in characters") characters (120 by default). - The name stops at a timestamp, a bracket or the end of the line, so request IDs, thread names and node IDs printed before the message stay out. - A logger or class name printed between the timestamp and the message stays in, as in `oteldemo_AdService_Targeted_ad_request_received_for`. - When the message quotes an exception after `:`, at most half the length goes to the quoted text, so two statements quoting the same exception keep different names. - Numbers, IDs, timestamps, repeated words and any word outside the symbol library never enter the name. A value that is itself a known word can, as the browser names do in the Web example below. A known word that is part of a value is skipped, so lines from `opensearch-0` and `opensearch-1` share one name: - a word joined by `-`, `_`, `.` or `$` to a number or another value, as in `req-b3e2` or `subdir2`; - a word inside a `host:port` such as `www.evernote.com:443`, or inside a path that has a value in it; - an upper-case code that mixes letters and digits, such as `L9ECAV7KIM`. ??? tenx-default "5. Fall back" - **One word found:** the name is the line's known words from the start of the message. Bracketed prefixes, `msg=`-style keys and [reserved](https://doc.log10x.com/run/transform/symbol/#symbolsequencereserved "words that do not count as evidence of a source") words are skipped. A logger name such as `org.apache.kafka.log.LocalLog` is kept, since it often tells two statements apart. - **No known word:** the name is `template_` plus a hash of the line's shape, its fixed words and punctuation with the values removed. The name holds while the shape holds. A stack frame or other line listed in [`messageNegators`](#messagenegators "line starts that mark a continuation line") that arrives as its own event is named the same way. The brackets, `key=value` separators, message keys and reserved words these steps use are options of the [Symbol](https://doc.log10x.com/run/transform/symbol/#symbolpreambleenclosures "brackets whose text before the message is skipped") unit, set in its config file. :material-github: See the [JavaScript implementation](https://github.com/log-10x/modules/blob/main/pipelines/run/modules/initialize/message/message-template.js){target="\_blank"} of this module on Github. ## :material-swap-horizontal: When Names Change A name stays the same across IDs, timestamps, nodes, restarts, pod renames and the order events arrive in. The same engine version, symbol library and configuration always give the same name. A name changes when: - the statement's text changes in code; - the symbol library is recompiled and a word on the line changes between value and known word; - an engine upgrade changes the naming rules. Every release that changes the naming rules or the default symbol library says so in its release notes; across releases that change neither, names stay the same (no pattern renamed from 1.1.140 to 1.1.142 on two public captures, [benchmarks/naming-stability](https://github.com/log-10x/benchmarks/tree/main/naming-stability)). ## :material-file-document-outline: Examples === ":simple-opentelemetry: OTel Demo" ```json { "stream": "stderr", "log": "2025-04-17 14:32:40,287 INFO [main] [recommendation_server.py:47] - Receive ListRecommendations for product ids:['L9ECAV7KIM', '0PUK6V6EV0']", "kubernetes": { "container_name": "recommendationservice", "namespace_name": "default" } } ``` **Pattern:** `Receive_ListRecommendations_for_product_ids` **Origin:** `recommendation_server.py:ListRecommendations` === ":simple-dotnet: .NET" A header record from the .NET console logger, then the message: ```json { "stream": "stdout", "log": "info: cart.cartstore.ValkeyCartStore[0]", "kubernetes": { "container_name": "cart" } } { "stream": "stdout", "log": " GetCartAsync called with userId=02fec73e-9f03-11f0-9b9e-a666c4b68b87", "kubernetes": { "container_name": "cart" } } ``` **Pattern:** `GetCartAsync_called_with_userId` **Origin:** `ValkeyCartStore.cs:ValkeyCartStore` === ":fontawesome-brands-java: Java" ```json { "stream": "stdout", "log": "2025-10-01 20:12:37 - oteldemo.AdService - Targeted ad request received for [accessories] trace_id=262794c5d52cea66092b4d8de7d1c6ed span_id=11dc1636e05888c3 trace_flags=01 ", "kubernetes": { "container_name": "ad", "namespace_name": "default" } } ``` **Pattern:** `oteldemo_AdService_Targeted_ad_request_received_for` **Origin:** `AdService.java:getAds` === ":material-web: Web" ```json { "stream": "stdout", "log": "192.168.43.96 - - [01/Aug/2025:22:21:50 +0000] \"GET /products/LensCleaningKit.jpg HTTP/1.1\" 200 101928 \"http://frontend-proxy:8080/\" \"Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/133.0.0.0 Safari/537.36\"", "kubernetes": { "container_name": "image-provider", "namespace_name": "default" } } ``` **Pattern:** `GET_HTTP_http_frontend_proxy_Mozilla_X_Linux_AppleWebKit_KHTML_like_Gecko_Safari` **Origin:** none ## :material-wrench-outline: Config Files To configure the Symbol Message Calculator module, [:material-cog: Edit](https://doc.log10x.com/config/app/#module-config "Learn how to edit app and module configurations") these files. Below is the default configuration from: [message/config.yaml](https://github.dev/log-10x/config/blob/main/pipelines/run/initialize/message/config.yaml "message/config.yaml"){target="\_blank"}.
Edit Online
Edit config.yaml Locally
```yaml # 🔟❎ 'run' symbol lookup configuration # Configure a symbol origin lookup to enrich TenXObjects # To learn more see https://doc.log10x.com/run/initialize/message/ # Set the 10x pipeline to 'run' tenx: run # =============================== Dependencies ================================ include: run/modules/initialize/message # ============================== Symbol Options =============================== symbol: # 'contexts' lists the kinds of symbols that can name an event: 'log' is text # passed to a logging call in source code, 'exec' is text read from a program # file. Values, in any case: package, class, method, log, enum, const, text, # exec, any. Order does not matter. # To learn more see https://doc.log10x.com/run/transform/symbol/#contexts contexts: log,exec # 'messageField' is the field that holds each event's name. messageField: message_pattern # 'messageHashField' is the field that holds an 11-character hash of the name. # Metrics and aggregation carry the hash either way. Name a field here to also # add the hash to events sent back to a log forwarder: # # messageHashField: tenx_hash # 'maxLen' is the longest name to write, in characters. 0 means no limit. maxLen: 120 # ============================== Message Options ============================== # 'negators' lists line starts that mark a line as a continuation of the event # above it, such as a stack frame. A continuation line that arrives as its own # event is named by its shape, template_, and not from the symbol library. message: negators: - " " # any indented line (stack frames, indented app detail, multi-line JSON, banners) - "\t" # real tab byte, for msgpack/forward inputs that preserve raw whitespace - "\\t" # literal backslash+t, for file/JSON inputs that keep escape sequences raw - "Caused by: " # Java chained exception (column 0) - "Suppressed: " # Java suppressed exception (column 0) - "--- End of " # .NET inner exception marker (column 0) - "created by " # Go goroutine creator (column 0) - "#0 " # PHP stack frame depth 0 - "#1 " # PHP stack frame depth 1 - "#2 " # PHP stack frame depth 2 - "#3 " # PHP stack frame depth 3 - "#4 " # PHP stack frame depth 4 - "#5 " # PHP stack frame depth 5 ``` ## :material-menu: Options Specify the options below to [configure](/config "configure") the Symbol Message Calculator: |Name|Description| |---|---| |[symbolContexts](#symbolcontexts "kinds of symbols that can name an event")|Kinds of symbols that can name an event| |[symbolMaxLen](#symbolmaxlen "longest name, in characters")|Longest name, in characters| |[symbolMessageField](#symbolmessagefield "field that holds the name")|Field that holds the name| |[symbolMessageHashField](#symbolmessagehashfield "field that holds a hash of the name")|Field that holds a hash of the name| |[symbolOriginField](#symboloriginfield "field that holds the statement's source location")|Field that holds the statement's source location| |[symbolSkeletonField](#symbolskeletonfield "field that holds the message layout")|Field that holds the message layout| |[messageNegators](#messagenegators "line starts that mark a continuation line")|Line starts that mark a continuation line| ### :material-menu-right-outline:**`symbolContexts`** Kinds of symbols that can name an event. |Type|Default| |---|---| |String|log,exec| A comma-separated list of the [symbol contexts](https://doc.log10x.com/run/transform/symbol/#contexts) that can name an event. The default `log,exec` matches text passed to a logging call in source code and text read from a program file, such as the output of `strings`. Order in the list does not matter. ### :material-menu-right-outline:**`symbolMaxLen`** Longest name, in characters. |Type|Default| |---|---| |Number|120| The longest name to write, in characters. Set `0` for no limit. ### :material-menu-right-outline:**`symbolMessageField`** Field that holds the name. |Type|Default| |---|---| |String|symbolMessage| The field that holds each event's name. The shipped configuration sets `message_pattern`. The name comes from the [symbolSequence](https://doc.log10x.com/api/js/#TenXObject+symbolSequence) function. ### :material-menu-right-outline:**`symbolMessageHashField`** Field that holds a hash of the name. |Type|Default| |---|---| |String|""| The field that holds an 11-character hash of the name. Unset by default. The hash is [TenXString.hash](https://doc.log10x.com/api/js/#TenXString.hash) of the name: xxHash64 of its UTF-8 bytes, big-endian, in base64url without padding. Any tool can compute the same value, so it works as a join key, a metric tag or a forwarder filter. Metrics and aggregation carry the hash whether or not this option is set. Events sent back to a [log forwarder](https://doc.log10x.com/run/input/forwarder/ "Report, receive and optimize events collected by log forwarders") return as they arrived; set this option to add the hash to them, for example `symbolMessageHashField tenx_hash`. ### :material-menu-right-outline:**`symbolOriginField`** Field that holds the statement's source location. |Type|Default| |---|---| |String|| The field that holds where the statement is in code, such as `recommendation_server.py:ListRecommendations`. Unset by default. The value comes from the [symbolOrigin](https://doc.log10x.com/api/js/#TenXObject+symbolOrigin) function. ### :material-menu-right-outline:**`symbolSkeletonField`** Field that holds the message layout. |Type|Default| |---|---| |String|""| The field that holds the message with `~` in place of each word of the name and `$` in place of each value. Unset by default. A viewer holding the name and this field can print the line as it was written, punctuation and spacing included, without access to the logs. ### :material-menu-right-outline:**`messageNegators`** Line starts that mark a continuation line. |Type|Default| |---|---| |List|\[\]| Line starts that mark a line as a continuation of the event above it, such as a stack frame. Empty by default. The shipped configuration lists: - a space, a tab or a literal `\t`: any indented line; - `Caused by: ` and `Suppressed: `: Java chained exceptions; - `--- End of `: the .NET inner exception marker; - `created by `: the Go goroutine creator; - `#0 ` to `#5 `: PHP stack frames. A continuation line that arrives as its own event is named by its shape, `template_` plus a hash, and not from the symbol library. The [group](https://doc.log10x.com/run/initialize/group/ "Combine multi-line events into TenXObject group instances") module keeps such a line with the event above it, even when it holds a level word such as `info`.
:material-github: This module is defined in [message/module.yaml](https://github.com/log-10x/modules/tree/main/pipelines/run/modules/initialize/message/module.yaml "message/module.yaml"){target="\_blank"}.