GrokToJSON¶
GrokToJSON is a formatter that applies regex filters to messages. It works by combining text patterns into something that matches your logs. See https://www.elastic.co/guide/en/logstash/current/plugins-filters-grok.html#_grok_basics for more information about Grok.
The output format is JSON.
Parameters¶
RemoveEmptyValues
When set to true, empty captures will not be returned. By default this parameter is set to “true”.
NamedCapturesOnly
When set to true, only named captures will be returned. By default this parameter is set to “true”.
SkipDefaultPatterns
When set to true, standard grok patterns will not be included in the list of patterns. By default this parameter is set to “true”.
Patterns
A list of grok patterns that will be applied to messages. The first matching pattern will be used to parse the message.
Parameters (from core.SimpleFormatter)¶
ApplyTo
This value chooses the part of the message the formatting should be applied to. Use “” to target the message payload; other values specify the name of a metadata field to target. By default this parameter is set to “”.
SkipIfEmpty
When set to true, this formatter will not be applied to data that is empty or - in case of metadata - not existing. By default this parameter is set to false
Examples¶
This example transforms unstructured input into a structured json output. Input:
us-west.servicename.webserver0.this.is.the.measurement 12.0 1497003802
Output:
{
"datacenter": "us-west",
"service": "servicename",
"host": "webserver0",
"measurement": "this.is.the.measurement",
"value": "12.0",
"time": "1497003802"
}
Config:
exampleConsumer:
Type: consumer.Console
Streams: "*"
Modulators:
- format.GrokToJSON:
Patterns:
- ^(?P<datacenter>[^\.]+?)\.(?P<service>[^\.]+?)\.(?P<host>[^\.]+?)\.statsd\.gauge-(?P<application>[^\.]+?)\.(?P<measurement>[^\s]+?)\s%{NUMBER:value_gauge:float}\s*%{INT:time}
- ^(?P<datacenter>[^\.]+?)\.(?P<service>[^\.]+?)\.(?P<host>[^\.]+?)\.statsd\.latency-(?P<application>[^\.]+?)\.(?P<measurement>[^\s]+?)\s%{NUMBER:value_latency:float}\s*%{INT:time}
- ^(?P<datacenter>[^\.]+?)\.(?P<service>[^\.]+?)\.(?P<host>[^\.]+?)\.statsd\.derive-(?P<application>[^\.]+?)\.(?P<measurement>[^\s]+?)\s%{NUMBER:value_derive:float}\s*%{INT:time}
- ^(?P<datacenter>[^\.]+?)\.(?P<service>[^\.]+?)\.(?P<host>[^\.]+?)\.(?P<measurement>[^\s]+?)\s%{NUMBER:value:float}\s*%{INT:time}