--- title: "File Input" description: "read events from an input file/glob path" source: "https://github.com/log-10x/modules/tree/main/pipelines/run/modules/input/file/module.yaml" icon: "material/math-log" --- Configures a input file path or glob pattern from which to read events to transform into typed [TenXObjects](https://doc.log10x.com/api/js/#TenXObject "Provide structured, reflective access to log/trace events read from input(s)."). This input does NOT tail its target file/glob. To tail files on disk, use a [forwarder](https://doc.log10x.com/run/input/forwarder "Report, receive and optimize events collected by log forwarders") input. ## :material-wrench-outline: Config Files To configure the File input module, [:material-cog: Edit](https://doc.log10x.com/config/app/#module-config "Learn how to edit app and module configurations") these files. Below is the default configuration from: [file/config.yaml](https://github.dev/log-10x/config/blob/main/pipelines/run/input/file/config.yaml "file/config.yaml"){target="\_blank"}.
Edit Online
Edit config.yaml Locally
```yaml # 🔟❎ 'run' File input configuration # Configure file/glob event inputs. To learn more https://doc.log10x.com/run/input/file # NOTE: input files are read once and are not tailed. To continuously tail files # use 10x in conjunction with any of the supported file forwarders in: # https://doc.log10x.com/run/input/forwarder (e.g., Fluentd/Bit, Logstash, Filebeat) # Set the 10x pipeline to 'run' tenx: run # =============================== Dependencies ================================ include: run/modules/input/file # =============================== File Options ================================ # Multiple file inputs can be defined below inputFile: # ---------------------------- General Options ---------------------------- # 'name' sets a unique logical name across all pipeline inputs - name: fileInput # ---------------------------- File Options --------------------------------- # 'path' specifies an absolute/canonical glob file path. path: $=path("data/sample/input/[!{.}]*.log") # ---------------------------- Parsing Options ---------------------------- # 'extractors' defines a list of JSON/regex extractor names # used to capture and redact event values from 'path' to transform # into TenXObjects. To learn more see: https://doc.log10x.com/run/input/extract extractors: [ # fluentd, # fluentK8s ] # 'sourcePattern' defines a regex pattern that captures a 'source' value # for each event read from a file to transform into an TenXObject. # This value will be used to sequence all events sharing this 'source' value correctly # so they are not mixed with events originating from different locations. sourcePattern: null # 'sourceFields' defines a list of JSON fields to extract from matching capture groups of the inputFileSourcePattern # or the entire event if not set as the 'source' value for each event read from this input. sourceFields: [ ] # --------------------------- Ancillary Options --------------------------- # 'printProgress' controls whether a progress gage is printed to the console printProgress: true # 'charset' sets the charset used to decode bytes read from this input. Defaults to UTF-8. charset: UTF-8 ``` ## :material-menu: Options Specify the options below to [configure](/config "configure") multiple File input: |Name|Description| |---|---| |[inputFileName](#inputfilename "logical name of this input")|Logical name of this input| |[inputFilePath](#inputfilepath "a file path or glob pattern to read from")|A file path or glob pattern to read from| |[inputFileEnabled](#inputfileenabled "enable/disable this file input")|Enable/disable this file input| |[inputFileExtractors](#inputfileextractors "JSON / regex extractor names")|JSON / regex extractor names| |[inputFileSourcePattern](#inputfilesourcepattern "event source capture regex pattern")|Event source capture regex pattern| |[inputFileSourceFields](#inputfilesourcefields "event source JSON fields")|Event source JSON fields| |[inputFilePrintProgress](#inputfileprintprogress "sets whether this input prints throughput stats to the console")|Sets whether this input prints throughput stats to the console| |[inputFileCharset](#inputfilecharset "charset used to decode bytes read from this input")|Charset used to decode bytes read from this input| ### :material-menu-right-outline:**`inputFileName`** Logical name of this input. |Type|Default| |---|---| |String|""| Sets a logical name (e.g., 'myInput') for this input. The [inputName](https://doc.log10x.com/api/js/#TenXBaseObject+inputName "Returns the context in which the current object, template or summary was created.") field returns this value at run time to allow for identifying and operating on instances originating from this input. ### :material-menu-right-outline:**`inputFilePath`** A file path or glob pattern to read from. |Type|Default|Names| |---|---|---| |File|path("\/data/sample/\[!{.}\]\*.log")|\[inputFilePath, -f\]| specifies a path that is either an absolute or canonical file path. For example: ```yaml path: $=path("/data/sample/[!{.}]*.log") ``` ### :material-menu-right-outline:**`inputFileEnabled`** Enable/disable this file input. |Type|Default| |---|---| |Boolean|true| Specifies whether the open the input to read events from the target [inputFilePath](https://doc.log10x.com/run/input/file/#inputfilepath "a file path or glob pattern to read from"). ### :material-menu-right-outline:**`inputFileExtractors`** JSON / regex extractor names. |Type|Default| |---|---| |List|\[\]| Defines a list of JSON/regex extractor names to capture events from 'inputFilePath' to transform into TenXObjects. Each item in the list should map to an extractor defined via a configuration file or startup arguments group. To learn more see: [extractors](https://doc.log10x.com/run/input/extract "Filter, redact, and capture events from an input stream to transform into TenXObjects"). ### :material-menu-right-outline:**`inputFileSourcePattern`** Event source capture regex pattern. |Type|Default| |---|---| |String|""| Defines a regex pattern to capture 'source' values for each event read from this input. Source values provide a method for sequencing TenXObjects read from this input in separate logical queues. For example, capturing a 'host' or 'fileName' value as a 'source' creates a separate logical queue for each unique file/host origin by which TenXObjects read from this input are aggregated and written to output. The [source](https://doc.log10x.com/api/js/#TenXObject+source "Returns the source value assigned to this instance by its input source pattern") member returns each TenXObject's assigned value at run time. . ### :material-menu-right-outline:**`inputFileSourceFields`** Event source JSON fields. |Type|Default| |---|---| |List|\[\]| Defines a list of JSON fields to extract from matching capture groups of [inputFileSourcePattern](https://doc.log10x.com/run/input/file/#inputfilesourcepattern "event source capture regex pattern") or the entire event if not set as the 'source' value for each event read from this input. Source values provide a method for sequencing TenXObjects read from this input in separate logical queues. For example, capturing a 'host' or 'fileName' value as a 'source' creates a separate logical queue for each unique file/host origin by which TenXObjects read from this input are aggregated and written to output. The [source](https://doc.log10x.com/api/js/#TenXObject+source "Returns the source value assigned to this instance by its input source pattern") member returns each TenXObject's assigned value at run time. ### :material-menu-right-outline:**`inputFilePrintProgress`** Sets whether this input prints throughput stats to the console. |Type|Default| |---|---| |Boolean|false| Sets whether this input prints throughput stats to the console. ### :material-menu-right-outline:**`inputFileCharset`** Charset used to decode bytes read from this input. |Type|Default| |---|---| |String|""| Sets the charset used to decode bytes read from this input into characters. Defaults to UTF-8.
:material-github: This module is defined in [file/module.yaml](https://github.com/log-10x/modules/tree/main/pipelines/run/modules/input/file/module.yaml "file/module.yaml"){target="\_blank"}.