TextToJSON¶
This formatter uses a state machine to parse arbitrary text data and transform it to JSON.
Parameters¶
StartState
Defines the name of the initial state when parsing a message. When set to an empty string the first state from the directives array will be used. By default this parameter is set to “”.
TimestampRead
Defines a time.Parse compatible format string used to read time fields when using the “dat” directive. By default this parameter is set to “20060102150405”.
TimestampWrite (default: 2006-01-02 15:04:05 MST)
Defines a time.Format compatible format string used to write time fields when using the “dat” directive. By default this parameter is set to “2006-01-02 15:04:05 MST”.
UnixTimestampRead
Defines the unix timestamp format expected from fields that are parsed using the “dat” directive. Valid valies are “s” for seconds, “ms” for milliseconds, or “ns” for nanoseconds. This parameter is ignored unless TimestampRead is set to “”. By default this parameter is set to “”.
Directives
Defines an array of directives used to parse text data. Each entry must be of the format: “State:Token:NextState:Flags:Function”. State denotes the name of the state owning this entry. Multiple entries per state are allowed. Token holds a string that triggers a state transition. NextState holds the target of the state transition. Flags is an optional field and is used to trigger special parser behavior. Flags can be comma separated if you need to use more than one. Function defines an action that is triggered upon state transition. Spaces will be stripped from all fields but Token. If a fields requires a colon it has to be escaped with a backslash. Other escape characters supported are n, r and t. By default this parameter is set to an empty list.
Directive rules
There are some special cases which will cause the parser to do additional actions.
- When writing a value without a key, the state name will become the key.
- If two keys are written in a row the first key will hold a null value.
- Writing a key while writing array elements will close the array.
Directive flags
Flags can modify the parser behavior and can be used to store values on a stack across multiple directives.
continue
Prepend the token to the next match.append
Append the token to the current match and continue reading.include
Append the token to the current match.push
Push the current state to the stack.pop
Pop the stack and use the returned state if possible.
Directive actions
Actions are used to write text read since the last transition to the JSON object.
key
Write the parsed section as a key.val
Write the parsed section as a value without quotes.esc
Write the parsed section as a escaped string value.dat
Write the parsed section as a timestamp value.arr
Start a new array.obj
Start a new object.end
Close an array or object.arr+val
arr followed by val.arr+esc
arr followed by esc.arr+dat
arr followed by dat.val+end
val followed by end.esc+end
esc followed by end.dat+end
dat followed by end.
Parameters (from core.SimpleFormatter)¶
ApplyTo
This value chooses the part of the message the formatting should be applied to. Use “” to target the message payload; other values specify the name of a metadata field to target. By default this parameter is set to “”.
SkipIfEmpty
When set to true, this formatter will not be applied to data that is empty or - in case of metadata - not existing. By default this parameter is set to false
Examples¶
The following example parses JSON data.
ExampleConsumer:
Type: consumer.Console
Streams: console
Modulators:
- format.TextToJSON:
Directives:
- "findKey :\": key : : "
- "findKey :}: : pop : end "
- "key :\": findVal : : key "
- "findVal :\\:: value : : "
- "value :\": string : : "
- "value :[: array : push : arr "
- "value :{: findKey : push : obj "
- "value :,: findKey : : val "
- "value :}: : pop : val+end"
- "string :\": findKey : : esc "
- "array :[: array : push : arr "
- "array :{: findKey : push : obj "
- "array :]: : pop : val+end"
- "array :,: array : : val "
- "array :\": arrString : : "
- "arrString :\": array : : esc "