Full Access Splunk SPLK-1003 Tutorials

REGEX =

* Enter a regular expression to operate on your data.

FORMAT =

* NOTE: This option is valid for both index-time and search-time field extraction. Index-time field extraction configuration require the FORMAT settings. The FORMAT settings is optional for search-time field extraction configurations.

* This setting specifies the format of the event, including any field names or values you want to add.

DEST_KEY =

* NOTE: This setting is only valid for index-time field extractions.

* Specifies where SPLUNK software stores the expanded FORMAT results in accordance with the REGEX match.

A Heavy Forwarder is a Splunk instance that can parse and filter data before forwarding it to another Splunk instance, such as an indexer1. A Heavy Forwarder can also perform index-time field extractions using the TRANSFORMS setting2.

The TRANSFORMS setting is used to configure data transformations in the transforms.conf file3. The transforms.conf file contains settings and values that you can use to configure host and source type overrides, anonymize sensitive data, route events to different indexes, create index-time and search-time field extractions, and set up lookup tables3.

The TRANSFORMS setting can be deployed to the Heavy Forwarder where the syslog files are being monitored, so that the logs can be rerouted based on the event message before they are forwarded to the indexer2. This can improve the performance and efficiency of data processing and indexing2.

The Universal Forwarder (UF) handles data forwarding to indexers. When monitoring a continuous and high-volume syslog stream over TCP, the UF may not detect an end-of-file (EOF) condition, which is typicallyrequired to trigger load balancing between multiple indexers. This can result in the UF continuously sending data to a single indexer, potentially leading to uneven load distribution.

To address this, Splunk provides the forceTimebasedAutoLB setting in the outputs.conf configuration file. Enabling this setting allows the UF to switch between indexers at regular time intervals, regardless of EOF detection. This ensures a more balanced distribution of data across multiple indexers, even in scenarios with continuous data streams like syslog.