I want to make an alert in bosun which will check for "level:fatal" message occurrences in logstash and send out an alert each time a unique message x host combination occurs. In order for this to be useful, it should group the alerts by tags of 'message' and 'host' and report the values of those tags in the template. My lscount query looks like this:
$fatal_log_rate = lscount("logstash","message,host","level:fatal", "5m", "5m", "")
My problem is that when running this alert in bosun, rather than grouping by the entirety of the message field like I would like, it is decomposing the message field of a single message into many groups. It looks like it is splitting on spaces, colons, newlines, brackets, and equals signs. For example, if my message is:
message: There was an error at: org.abc.Class:42.
ABC Component failed (reason=300)
The resulting bosun output from the lscount above would give me 12 groups containing messages "there", "was", "an", "error", "at", "org.abc.Class", "42", "abc", "component", "failed", "reason", "300".
The grouping by host is also not as expected, although that is splitting on the first period. For example, our hostname might be core2.abc.xyz.net, but results from the lscount above is split on host=core2 and host=abc.xyz.net.
Is there any way to group an alert by the all or part of the message field, and then get the full message field to report in the template?
