Profiler

The “Profiler” consumer plugin autogenerates messages in user-defined quantity, size and density. It can be used to profile producers and configurations and to provide a message source for testing.

Before startup, [TemplateCount] template payloads are generated based on the format specifier [Message], using characters from [Characters]. The length of each template is determined by format size specifier(s) in [Message].

During execution, [Batches] batches of [Runs] messages are generated, with a [DelayMs] ms delay between each message. Each message’s payload is randomly selected from the set of template payloads above.

Parameters

Enable (default: true)

Switches this plugin on or off.

Runs (default: 10000)

Defines the number of messages per batch.

Batches (default: 10)

Defines the number of batches to generate.

TemplateCount

Defines the number of message templates to generate. Templates are generated in advance and a random message template is chosen from this set every time a message is sent.

Characters (default: abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ01234567890)

Defines the set of characters use when generated templates.

Message (default: %256s)

Defines a go format string to use for generating the message templaets. The length of the values generated will be deduced from the format size parameter - “%200d” will generate a digit between 0 and 200, “%10s” will generate a string with 10 characters, etc.

DelayMs (default: 0, unit: ms)

Defines the number of milliseconds to sleep between messages.

KeepRunning

If set to true, shuts down Gollum after Batches * Runs messages have been generated. This can be used to e.g. read metrics after a profile run.

Parameters (from core.SimpleConsumer)

Streams

Defines a list of streams a consumer will send to. This parameter is mandatory. When using “*” messages will be sent only to the internal “*” stream. It will NOT send messages to all streams. By default this parameter is set to an empty list.

ShutdownTimeoutMs (default: 1000, unit: ms)

Defines the maximum time in milliseconds a consumer is allowed to take to shut down. After this timeout the consumer is always considered to have shut down. By default this parameter is set to 1000.

Modulators

Defines a list of modulators to be applied to a message before it is sent to the list of streams. If a modulator specifies a stream, the message is only sent to that specific stream. A message is saved as original after all modulators have been applied. By default this parameter is set to an empty list.

ModulatorRoutines

Defines the number of go routines reserved for modulating messages. Setting this parameter to 0 will use as many go routines as the specific consumer plugin is using for fetching data. Any other value will force the given number fo go routines to be used. By default this parameter is set to 0

ModulatorQueueSize

Defines the size of the channel used to buffer messages before they are fetched by the next free modulator go routine. If the ModulatorRoutines parameter is set to 0 this parameter is ignored. By default this parameter is set to 1024.

Examples

Generate a short message every 0.5s, useful for testing and debugging

JunkGenerator:
  Type: "consumer.Profiler"
  Message: "%20s"
  Streams: "junkstream"
  Characters: "abcdefghijklmZ"
  KeepRunning: true
  Runs: 10000
  Batches: 3000000
  DelayMs: 500