--- title: "Timestamp Parser" description: "extract alphanumeric and epoch timestamp values from input events" source: "https://github.com/log-10x/modules/tree/main/pipelines/run/units/transform/timestamp/unit.yaml" icon: "material/stamper" --- Identify unix/alphanumeric timestamp structures within [TenXTemplates](https://doc.log10x.com/run/template/#structure). This component enables [TenXObjects](https://doc.log10x.com/api/js/#TenXObject "Provide structured, reflective access to log/trace events read from input(s).") to access embedded timestamp values at runtime as 64-bit epoch values. For example, for the following Spark log event: ``` 15/09/01 18:14:50 INFO spark.SecurityManager: Changing view acls to: yarn,yxsu1 ``` The first 17 characters represent a timestamp. Each instance of this TenXTemplate can access its own timestamp value via the [timestamp](https://doc.log10x.com/api/js/#TenXObject+timestamp "An array of UNIX epoch values of timestamps parsed from the object's text.") array field, which returns 64-bit Unix epoch value(s) (e.g., 1000577690). The date/time format of any timestamps found (e.g., 'DD/MM/YY HH:MM:SS') is encoded into the object's TenXTemplate. Subsequent TenXObjects associated with that template will skip the timestamp discovery process, making timestamp extraction highly efficient. ## :material-wrench-outline: Config Files To configure the Timestamp parser unit, [:material-cog: Edit](https://doc.log10x.com/config/app/#module-config "Learn how to edit app and module configurations") these files. Below is the default configuration from: [timestamp/config.yaml](https://github.dev/log-10x/config/blob/main/pipelines/run/transform/timestamp/config.yaml "timestamp/config.yaml"){target="\_blank"}.
Edit Online
Edit config.yaml Locally
```yaml # 🔟❎ 'run' timestamp parser configuration # Identify unix/alphanumeric timestamp structures within TenXTemplates. # https://doc.log10x.com/run/transform/timestamp/ # Set the 10x pipeline to 'run' tenx: run # ============================= Timestamp Options ============================= timestamp: # 'maxPerObject' controls the max number of timestamps to add into an TenXObject's # 'timestamp' array. Set to 0 for unlimited. maxPerObject: 0 # 'searchDirection' controls the direction(s) from with timestamps are searched for # within the object's 'text' field. Possible values: # - fromStart: search for 'maxPerObject' from the start of 'text', # limiting search to 'searchLengthLimitFromStart' characters # - fromEnd: search backward for 'maxPerObject' from the end of 'text', # limiting search to 'searchLengthLimitFromEnd' characters # - fromStartAndEnd: search for 'maxPerObject' from both the start and end of 'text' # limiting search to 'searchLengthLimitFromStart' and 'searchLengthLimitFromEnd' respectively # - none: do not parse timestamps searchDirection: fromStartAndEnd # 'searchLengthLimitFromStart' limits the number of characters to search for # timestamps from the beginning of the object's 'text' field. 0 for unlimited searchLengthLimitFromStart: 0 # 'searchLengthLimitFromStart' limits the number of characters to search for # timestamps from the end of the object's 'text' field. 0 for unlimited searchLengthLimitFromEnd: 0 # 'zone' controls the timezone for formatting timestamp epoch # values to string. The 'java.time.ZoneId.of(String zoneId)' # is used to obtain the timezone from the 'zone' value. # If set to null, the host OS timezone is used. zone: null # 'literals' contains an array of strings to treat as a part # of any timestamp candidate found when structuring an TenXObject. literals: - T - Z - I # Go INFO - E # Go ERROR - W # Go WARN # 'patterns' specifies an array of date-time formats to attempt when parsing timestamps from input events. # Timestamp formats that appear frequently within an input stream can be 'bumped' higher within the list below. patterns: # Most common formats - "'I'MMdd HH:mm:ss.S" # Used in Kubernetes kube-apiserver logs with INFO prefix, 1-digit microsecond precision. - "'I'MMdd HH:mm:ss.SS" # Used in Kubernetes kube-apiserver logs with INFO prefix, 2-digit microsecond precision. - "'I'MMdd HH:mm:ss.SSS" # Used in Kubernetes kube-apiserver logs with INFO prefix, 3-digit microsecond precision. - "'I'MMdd HH:mm:ss.SSSS" # Used in Kubernetes kube-apiserver logs with INFO prefix, 4-digit microsecond precision. - "'I'MMdd HH:mm:ss.SSSSS" # Used in Kubernetes kube-apiserver logs with INFO prefix, 5-digit microsecond precision. - "'I'MMdd HH:mm:ss.SSSSSS" # Used in Kubernetes kube-apiserver logs with INFO prefix, 6-digit microsecond precision. - "'W'MMdd HH:mm:ss.S" # Used in Kubernetes kube-apiserver logs with WARNING prefix, 1-digit microsecond precision. - "'W'MMdd HH:mm:ss.SS" # Used in Kubernetes kube-apiserver logs with WARNING prefix, 2-digit microsecond precision. - "'W'MMdd HH:mm:ss.SSS" # Used in Kubernetes kube-apiserver logs with WARNING prefix, 3-digit microsecond precision. - "'W'MMdd HH:mm:ss.SSSS" # Used in Kubernetes kube-apiserver logs with WARNING prefix, 4-digit microsecond precision. - "'W'MMdd HH:mm:ss.SSSSS" # Used in Kubernetes kube-apiserver logs with WARNING prefix, 5-digit microsecond precision. - "'W'MMdd HH:mm:ss.SSSSSS" # Used in Kubernetes kube-apiserver logs with WARNING prefix, 6-digit microsecond precision. - "'E'MMdd HH:mm:ss.S" # Used in Kubernetes kube-apiserver logs with ERROR prefix, 1-digit microsecond precision. - "'E'MMdd HH:mm:ss.SS" # Used in Kubernetes kube-apiserver logs with ERROR prefix, 2-digit microsecond precision. - "'E'MMdd HH:mm:ss.SSS" # Used in Kubernetes kube-apiserver logs with ERROR prefix, 3-digit microsecond precision. - "'E'MMdd HH:mm:ss.SSSS" # Used in Kubernetes kube-apiserver logs with ERROR prefix, 4-digit microsecond precision. - "'E'MMdd HH:mm:ss.SSSSS" # Used in Kubernetes kube-apiserver logs with ERROR prefix, 5-digit microsecond precision. - "'E'MMdd HH:mm:ss.SSSSSS" # Used in Kubernetes kube-apiserver logs with ERROR prefix, 6-digit microsecond precision. - "'F'MMdd HH:mm:ss.S" # Used in Kubernetes kube-apiserver logs with FATAL prefix, 1-digit microsecond precision. - "'F'MMdd HH:mm:ss.SS" # Used in Kubernetes kube-apiserver logs with FATAL prefix, 2-digit microsecond precision. - "'F'MMdd HH:mm:ss.SSS" # Used in Kubernetes kube-apiserver logs with FATAL prefix, 3-digit microsecond precision. - "'F'MMdd HH:mm:ss.SSSS" # Used in Kubernetes kube-apiserver logs with FATAL prefix, 4-digit microsecond precision. - "'F'MMdd HH:mm:ss.SSSSS" # Used in Kubernetes kube-apiserver logs with FATAL prefix, 5-digit microsecond precision. - "'F'MMdd HH:mm:ss.SSSSSS" # Used in Kubernetes kube-apiserver logs with FATAL prefix, 6-digit microsecond precision. - "yyyy-MM-dd HH:mm:ss.SSSSSSZ" # Used in macOS unified logs (log show), with microsecond precision and timezone offset (e.g., -0400). - "yyyy-MM-dd HH:mm:ss" # Widely used in Java applications, databases (e.g., MySQL, PostgreSQL), and application servers (e.g., Tomcat, JBoss). - "yyyy-MM-dd'T'HH:mm:ss.SSSZ" # Common in web services, APIs, Java (with DateTimeFormatter.ISO_OFFSET_DATE_TIME), Python (with datetime.isoformat()), and systems requiring precise timestamps with timezone information. - "MMM dd HH:mm:ss" # Frequently seen in syslog, Unix-based systems, network devices (e.g., Cisco routers), and web servers (e.g., Apache, Nginx). - "MMM d HH:mm:ss" # Classic syslog (RFC 3164) on days 1-9: the day is space-padded, e.g. "Jan 3 17:38:36". - "MMM d HH:mm:ss" # Same date without the padding (strftime %b %-d), e.g. "Nov 9 12:01:01" in Thunderbird-style system logs. - "dd/MMM/yyyy:HH:mm:ss Z" # Standard syslog format, used in firewalls (e.g., Cisco ASA, Palo Alto), network monitoring tools, and Unix-based systems. - "yyyy-MM-dd HH:mm:ss,SSS" # Common in Java logging frameworks like Log4j and Logback. - "yyyy-MM-dd'T'HH:mm:ss,SSS" # Common in Java logging with ISO 8601 date-time, comma-separated milliseconds. - "MM/dd/yyyy HH:mm:ss" # Used in Windows event logs, .NET applications, and U.S.-based systems. - "EEE MMM dd HH:mm:ss yyyy" # Human-readable format used in various logs, including some web servers and application logs. - "EEE MMM dd HH:mm:ss zzz yyyy" # java.util.Date.toString() and the Unix date command, e.g. "Thu Jan 15 10:00:01 UTC 2026". Common in Java application logs and shell-script output. - "yyyy-MM-dd'T'HH:mm:ss" # Simplified ISO 8601 format, used in many modern applications and frameworks. - "HH:mm:ss" # Time-only format, used when the date is implied or provided separately, common in embedded systems and some programming languages. - "yyyyMMdd HH:mm:ss" # Compact format used in some legacy systems and batch processing logs. - "yyyyMMdd-H:m:s:SSS" # Compact date, hyphen, unpadded time, colon before the milliseconds, e.g. "20171223-22:15:29:606" or "20171224-0:0:0:234". Android HealthApp and similar mobile logs. - "yyyyMMdd-H:m:s:SS" # Same with two fraction digits, e.g. "20171223-22:30:24:28". - "yyyyMMdd-H:m:s:S" # Same with one fraction digit. - "MM.dd HH:mm:ss" # Month.day without a year, e.g. "10.30 16:49:06". Proxifier and other Windows utility logs. - "MMM dd, yyyy h:mm:ss a" # Human-readable format with 12-hour clock, used in application logs and some U.S.-based systems. - "MMM dd, yyyy hh:mm:ss a" # Human-readable format with 12-hour clock (padded hour), used in application logs and some U.S.-based systems. # Common formats with slight variations - "yyyy-MM-dd HH:mm:ss.SSS" # Extended precision format used in Java applications and databases requiring millisecond accuracy. - "dd-MMM-yyyy HH:mm:ss.SSS" # Common in Java logging frameworks (e.g., Log4j, SLF4J) and application logs requiring human-readable dates with millisecond precision. - "yyyy-MM-dd'T'HH:mm:ss.S'Z'" # Microsecond precision ISO 8601 format, 1-digit precision. - "yyyy-MM-dd'T'HH:mm:ss.SS'Z'" # Microsecond precision ISO 8601 format, 2-digit precision. - "yyyy-MM-dd'T'HH:mm:ss.SSS'Z'" # Microsecond precision ISO 8601 format, 3-digit precision. - "yyyy-MM-dd'T'HH:mm:ss.SSSS'Z'" # Microsecond precision ISO 8601 format, 4-digit precision. - "yyyy-MM-dd'T'HH:mm:ss.SSSSS'Z'" # Microsecond precision ISO 8601 format, 5-digit precision. - "yyyy-MM-dd'T'HH:mm:ss.SSSSSS'Z'" # Microsecond precision ISO 8601 format, 6-digit precision. - "yyyy-MM-dd'T'HH:mm:ss.S'Z'" # Nanosecond precision ISO 8601 format, 1-digit precision. - "yyyy-MM-dd'T'HH:mm:ss.SS'Z'" # Nanosecond precision ISO 8601 format, 2-digit precision. - "yyyy-MM-dd'T'HH:mm:ss.SSS'Z'" # Nanosecond precision ISO 8601 format, 3-digit precision. - "yyyy-MM-dd'T'HH:mm:ss.SSSS'Z'" # Nanosecond precision ISO 8601 format, 4-digit precision. - "yyyy-MM-dd'T'HH:mm:ss.SSSSS'Z'" # Nanosecond precision ISO 8601 format, 5-digit precision. - "yyyy-MM-dd'T'HH:mm:ss.SSSSSS'Z'" # Nanosecond precision ISO 8601 format, 6-digit precision. - "yyyy-MM-dd'T'HH:mm:ss.SSSSSSS'Z'" # Nanosecond precision ISO 8601 format, 7-digit precision. - "yyyy-MM-dd'T'HH:mm:ss.SSSSSSSS'Z'" # Nanosecond precision ISO 8601 format, 8-digit precision. - "yyyy-MM-dd'T'HH:mm:ss.SSSSSSSSS'Z'" # Nanosecond precision ISO 8601 format, 9-digit precision. - "yyyy-MM-dd'T'HH:mm:ssZZZZZ" # ISO 8601 with timezone offset, used in systems where timezone information is critical. - "yyyy-MM-dd HH:mm:ss.SSSZZZZZ" # Similar to above but without the 'T' separator, used in database logs and application servers. - "yyyy-MM-dd HH:mm:ss.S" # Microsecond precision format, 1-digit precision. - "yyyy-MM-dd HH:mm:ss.SS" # Microsecond precision format, 2-digit precision. - "yyyy-MM-dd HH:mm:ss.SSS" # Microsecond precision format, 3-digit precision. - "yyyy-MM-dd HH:mm:ss.SSSS" # Microsecond precision format, 4-digit precision. - "yyyy-MM-dd HH:mm:ss.SSSSS" # Microsecond precision format, 5-digit precision. - "yyyy-MM-dd HH:mm:ss.SSSSSS" # Microsecond precision format, 6-digit precision. - "MM/dd/yyyy*HH:mm:ss*SSS" # Used in some U.S.-based systems, particularly in legacy applications or specific logging frameworks. - "M/d/yyyy h:mm:ss a:SSS" # Common in systems using 12-hour time format, such as some Windows applications or older logging systems. - "M/d/yyyy hh:mm:ss a:SSS" # Common in systems using 12-hour time format with milliseconds and padded hour, such as some Windows applications. - "M/dd/yyyy hh:mm:ss a" # Similar to above, used in systems where millisecond precision is not needed. - "yyyy-MM-dd'T'HH:mm:ss.SSSX" # ISO 8601 with basic timezone offset format, used in systems that require standardized timestamps. - "yyyy-MM-dd'T'HH:mm:ss.SSSz" # ISO 8601 with timezone name, used in applications that need to display timezone information. - "yyyy-MM-dd'T'HH:mm:ss'Z'" # ISO 8601 format assuming UTC, used in systems where all times are in UTC. - "yy/MM/dd HH:mm:ss" # Two-digit year format, used in compact logs or older systems. - "MMM dd HH:mm:ss ZZZZ" # Used in some Unix-based systems and web servers, includes timezone offset. - "HH:mm:ss,SSS" # Time-only format with milliseconds, used in performance logs or systems where date is provided separately. - "yyyy-MM-dd*HH:mm:ss" # Used in some application logs where the separator is a space or asterisk. - "yyyy MMM dd HH:mm:ss.SSS" # Human-readable format with milliseconds, used in some application logs. - "dd/MMM/yyyy HH:mm:ss" # Used in European systems and some web applications. - "yyyy-MM-dd'T'HH:mm:ss.SSS''Z''" # ISO 8601 with milliseconds and literal 'Z', used in systems where 'Z' is explicitly included. - "MMM dd yyyy HH:mm:ss" # Human-readable format, used in various logs. - "yyyy-MM-dd HH:mm:ss ZZZZ" # Format with timezone offset, used in systems requiring timezone information. - "yyyy-MM-dd HH:mm:ssZZZZZ" # Similar to above, used in application logs. - "dd MMM yyyy HH:mm:ss" # European format, used in some web applications and databases. - "MMdd_HH:mm:ss" # Compact format without separators, used in file names or space-constrained logs. - "yyyy-MM-dd HH:mm:ss,SSSZZZZZ" # Java logging format with timezone offset. - "yyyyMMdd HH:mm:ss.SSS" # Compact format with milliseconds, used in some legacy systems. - "yyyy/MM/dd HH:mm:ss" # Format used in some Asian systems, particularly in Japan. - "dd/MM/yyyy HH:mm:ss" # Common in European systems, including some web applications and databases. - "MM-dd-yyyy HH:mm:ss" # U.S. format variant, used in some older systems. - "yyyyMMddHHmmss" # Compact format for file names or database timestamps, used in systems where space is a concern. - "EEE, dd MMM yyyy HH:mm:ss zzz" # RFC 1123 format, used in HTTP headers, web servers, and email systems. ``` ## :material-menu: Options Specify the options below to [configure](/config "configure") the Timestamp parser: |Name|Description| |---|---| |[timestampPatterns](#timestamppatterns "date-time formats for parsing timestamps from input events")|Date-time formats for parsing timestamps from input events| |[timestampSearchDirection](#timestampsearchdirection "controls the text direction(s) in which to search for timestamps")|Controls the text direction(s) in which to search for timestamps| |[timestampMaxPerObject](#timestampmaxperobject "maximum number of timestamps per TenXObject")|Maximum number of timestamps per TenXObject| |[timestampSearchLengthLimitFromStart](#timestampsearchlengthlimitfromstart "maximum number of characters to scan for timestamps from event start")|Maximum number of characters to scan for timestamps from event start| |[timestampSearchLengthLimitFromEnd](#timestampsearchlengthlimitfromend "max number of characters to scan for timestamps from event end")|Max number of characters to scan for timestamps from event end| |[timestampZone](#timestampzone "timezone to use when formatting timestamp Unix epoch values")|Timezone to use when formatting timestamp Unix epoch values| |[timestampLiterals](#timestampliterals "timestamp literals")|Timestamp literals| ### :material-menu-right-outline:**`timestampPatterns`** Date-time formats for parsing timestamps from input events. |Type|Default| |---|---| |List|\[\[config.yaml\](https://github.com/log-10x/config/tree/main/pipelines/run/config/transform/timestamp/config.yaml)\]| Lists date-time formats in [Java specification](https://docs.oracle.com/javase/8/docs/api/java/time/format/DateTimeFormatter.html){target="\_blank"} for parsing timestamp epoch values from input events. ### :material-menu-right-outline:**`timestampSearchDirection`** Controls the text direction(s) in which to search for timestamps. |Type|Default| |---|---| |String|fromStart| Controls the direction(s) to search for timestamps within an TenXObject's [text](https://doc.log10x.com/api/js/#TenXBaseObject+text "The content of the event from which this instance was structured.") field. Possible values: - **fromStart**: search for [timestampMaxPerObject](https://doc.log10x.com/run/transform/timestamp/#timestampmaxperobject "maximum number of timestamps per TenXObject") from the start of \[text\]https://doc.log10x.com/api/js/#TenXBaseObject+text) to restrict the search to [searchLengthLimitFromStart](https://doc.log10x.com/run/transform/timestamp/#timestampsearchlengthlimitfromstart "maximum number of characters to scan for timestamps from event start") characters - **fromEnd**: search backward for [timestampMaxPerObject](https://doc.log10x.com/run/transform/timestamp/#timestampmaxperobject "maximum number of timestamps per TenXObject") from the end of [text](https://doc.log10x.com/api/js/#TenXBaseObject+text "The content of the event from which this instance was structured.") to limit the search to [searchLengthLimitFromEnd](https://doc.log10x.com/run/transform/timestamp/#timestampsearchlengthlimitfromend "max number of characters to scan for timestamps from event end") characters - **fromStartAndEnd** search for [timestampMaxPerObject](https://doc.log10x.com/run/transform/timestamp/#timestampmaxperobject "maximum number of timestamps per TenXObject") from both the start and end of [text](https://doc.log10x.com/api/js/#TenXBaseObject+text "The content of the event from which this instance was structured.") to limit the search to [searchLengthLimitFromStart](https://doc.log10x.com/run/transform/timestamp/#timestampsearchlengthlimitfromstart "maximum number of characters to scan for timestamps from event start") and [searchLengthLimitFromEnd](https://doc.log10x.com/run/transform/timestamp/#timestampsearchlengthlimitfromend "max number of characters to scan for timestamps from event end") respectively - **none**: do not parse timestamps. ### :material-menu-right-outline:**`timestampMaxPerObject`** Maximum number of timestamps per TenXObject. |Type|Default| |---|---| |Number|0| Controls the maximum number of timestamps to add into an TenXObject's [timestamp](https://doc.log10x.com/api/js/#TenXObject+timestamp "An array of UNIX epoch values of timestamps parsed from the object's text.") array. Set to 0 for unlimited. ### :material-menu-right-outline:**`timestampSearchLengthLimitFromStart`** Maximum number of characters to scan for timestamps from event start. |Type|Default| |---|---| |Number|0| Limits the number of characters to search for timestamps from the beginning of the object's [text](https://doc.log10x.com/api/js/#TenXBaseObject+text "The content of the event from which this instance was structured.") field. Set 0 for unlimited. ### :material-menu-right-outline:**`timestampSearchLengthLimitFromEnd`** Max number of characters to scan for timestamps from event end. |Type|Default| |---|---| |Number|0| Limits the number of characters to search for timestamps from the end of the object's [text](https://doc.log10x.com/api/js/#TenXBaseObject+text "The content of the event from which this instance was structured.") field. Set 0 for unlimited. ### :material-menu-right-outline:**`timestampZone`** Timezone to use when formatting timestamp Unix epoch values. |Type|Default| |---|---| |String|[host OS timezone](https://docs.oracle.com/javase/8/docs/api/java/time/ZoneId.html#systemDefault-- "host OS timezone"){target="\_blank"}| Controls the timezone used when formatting timestamp epoch string values. The [ZoneId.of](https://docs.oracle.com/javase/8/docs/api/java/time/ZoneId.html#of-java.lang.String-){target="\_blank"} obtains the timezone from the 'zone' value. ### :material-menu-right-outline:**`timestampLiterals`** Timestamp literals. |Type|Default| |---|---| |List|\[T, Z\]| An array of strings literal values to treat as valid parts of timestamp (e.g., 'Z', 'T', 'GMT').
:material-github: This unit is defined in [timestamp/unit.yaml](https://github.com/log-10x/modules/tree/main/pipelines/run/units/transform/timestamp/unit.yaml "timestamp/unit.yaml"){target="\_blank"}.