ProcessTSV¶
This formatter allows modification of TSV encoded data. Each field can be processed by different directives and the result of all directives will be stored back to the original location.
Parameters¶
Delimiter (default: t)
Defines the separator used to split values. By default this parameter is set to “t”.
QuotedValue
When set to true, values that start and end with a quotation mark are not scanned for delimiter characters. I.e. those values will not be split even if they contain delimiter characters. By default this parameter is set to false.
Directives
Defines an array of actions to apply to the TSV encoded data. Directives are processed in order of appearance. Directives start with the index of the field, followed by an action followed by additional parameters if necessary. Parameters, key and action are separated by the “:” character. By default this parameter is set to an empty list.
replace
<string> <new string> Replace a given string inside the field’s value with another one.prefix
<string> Prepend the given string to the field’s valuepostfix
<string> Append the given string to the field’s valuetrim
<characters> Remove the given characters from the start and end of the field’s value.quote
Surround the field’s value with quotation marks after all directives have been processed.time
<from fromat> <to format> Read a timestamp in the specified time.Parse-compatible format and transform it into another format compatible with time.Format.
** (unnamed) **
remove Removes the field from the resultagent
{<field>, <field>, …} Parse the field’s value as a user agent string and insert the given fields into the TSV after the given index. If no fields are given all fields are returned.
mozilla
mozilla versionplatform
the platform usedos
the operating system usedlocalization
the language usedengine
codename of the browser engineengine_version
version of the browser enginebrowser
name of the browserversion
version of the browser
QuotedValues
(no documentation available)
Parameters (from core.SimpleFormatter)¶
ApplyTo
This value chooses the part of the message the formatting should be applied to. Use “” to target the message payload; other values specify the name of a metadata field to target. By default this parameter is set to “”.
SkipIfEmpty
When set to true, this formatter will not be applied to data that is empty or - in case of metadata - not existing. By default this parameter is set to false
Examples¶
ExampleConsumer:
Type: consumer.Console
Streams: console
Modulators:
- format.processTSV:
Delimiter: ","
Directives:
- "0:time:20060102150405:2006-01-02 15\\:04\\:05"
- "2:remove"
- "11:agent:os:engine:engine_version"