I'm really new to ELK and I have set up an ELK stack where FileBeat sends the logs to LogStash for some processing and then outputs to Elasticsearch.
I was wondering if it is possible to maintain the index name set in filebeat.yml all the way to Elasticsearch. The reason why I want that is because I want multiple indices for different types of app servers that I have generating logs. If I leave out index in logstash.conf, it defaults; but if I specify something, obviously that takes effect. I simply want it to use what was set in FileBeat.
Or is there some way to configure multiple output sections where log types can be evaluated so I can name them appropriately?
filebeat.yml
# Optional index name. The default index name is set to filebeat in all lowercase.
index: "something-%{+yyyy.MM.dd}"
logstash.conf
output {
elasticsearch {
hosts => ["somehost:12345"]
index => "my_filebeat_index_name_would_be_preferred-%{+yyyy-MM-dd}"
}
}
I would like to continue to use LogStash because I have custom GROK patterns etc and not to go directly to Elastic. Any help would be greatly appreciated.
Thanks.