GrokToJSON¶
GrokToJSON is a formatter that applies regex filters to messages. It works by combining text patterns into something that matches your logs. See https://www.elastic.co/guide/en/logstash/current/plugins-filters-grok.html#_grok_basics for more information about Grok.
The output format is JSON.
Parameters¶
Patterns
A list of grok patterns that will be applied to messages. The first matching pattern will be used to parse the message.
Parameters (from core.SimpleFormatter)¶
ApplyTo
This value chooses the part of the message the formatting should be applied to. Use “” to target the message payload; other values specify the name of a metadata field to target. By default this parameter is set to “”.
SkipIfEmpty
When set to true, this formatter will not be applied to data that is empty or - in case of metadata - not existing. By default this parameter is set to false
Examples¶
This example transforms unstructured input into a structured json output. Input:
us-west.servicename.webserver0.this.is.the.measurement 12.0 1497003802
Output:
{
"datacenter": "us-west",
"service": "servicename",
"host": "webserver0",
"measurement": "this.is.the.measurement",
"value": "12.0",
"time": "1497003802"
}
Config:
exampleConsumer:
Type: consumer.Console
Streams: "*"
Modulators:
- format.GrokToJSON:
Patterns:
- ^(?P<datacenter>[^\.]+?)\.(?P<service>[^\.]+?)\.(?P<host>[^\.]+?)\.statsd\.gauge-(?P<application>[^\.]+?)\.(?P<measurement>[^\s]+?)\s%{NUMBER:value_gauge:float}\s*%{INT:time}
- ^(?P<datacenter>[^\.]+?)\.(?P<service>[^\.]+?)\.(?P<host>[^\.]+?)\.statsd\.latency-(?P<application>[^\.]+?)\.(?P<measurement>[^\s]+?)\s%{NUMBER:value_latency:float}\s*%{INT:time}
- ^(?P<datacenter>[^\.]+?)\.(?P<service>[^\.]+?)\.(?P<host>[^\.]+?)\.statsd\.derive-(?P<application>[^\.]+?)\.(?P<measurement>[^\s]+?)\s%{NUMBER:value_derive:float}\s*%{INT:time}
- ^(?P<datacenter>[^\.]+?)\.(?P<service>[^\.]+?)\.(?P<host>[^\.]+?)\.(?P<measurement>[^\s]+?)\s%{NUMBER:value:float}\s*%{INT:time}