2
votes

My use case is to ship log files from various applications to Elasticsearch so that I can view them from Kibana.

I wanted to know can Filebeat be configured for grok expression so that application team can manage their log parsing at their end and central logging system / deployment is unaffected? If it can be then the need for Logstash is questionable. I can see Filebeat supports multi-line expressions, but can I define other grok expression and add/remove fields by parsing the message?

1

1 Answers

1
votes

Filebeat does not perform grok processing. So you will need to send the data to Logstash for processing.

There is a new feature coming in Elasticsearch 5.0 called Ingest Node that will allow some simple grok processing to be performed without needing Logstash (i.e. Filebeat -> Elasticsearch).