I'm trying to parse messages from multiple applications from a single container inside a kubernetes pod using fluentd... Fluentd, Kibana and Elasticsearch are working well and I have all my logs showing up and am otherwise happy. However, I need to process a series of container log differently. In this case the logs I need to further parse are all in a single namespace. My container log looks something like the following:
somekeyword1 somenumber somestuff optionals
somekeyword2 somenumber somestuff optionals
somekeyword2 somenumber somestuff optionals
somekeyword1 somenumber somestuff optionals
...
I would like to process that log with a regex based on the original somekeywordN (but am open to suggestions) So within fluentd, how do I perform that level of processing on a log message for containers in a particular namespace?
I've tried this:
https://github.com/fabric8io/fluent-plugin-kubernetes_metadata_filter/issues/73
but even given that, I'm not sure how to run my regexes on a particular log