How do I get the filename from the source value supplied by filebeat in logstash?

2
votes

I want to get the filename from the source value provided by filebeat. 

output {
  if [type] == "wxnumber" {
    elasticsearch {
        hosts => "localhost:9200"
        sniffing => false
        manage_template => false
        index => "%{[source]}"
        document_type => "%{[@metadata][type]}"
    }
  }
}

The %{[source]} is usually like /aaa/bbb/ccc.log. How do I set the index to the ccc.log?

1
elasticsearchlogstashelastic-stackfilebeat
What's the result you're getting, from the above config? – Kulasangar
@ Kulasangar , Thank for you reply. The result of %{[source]} is like /aaa/bbb/ccc.log. I just want get the ccc.log for this variable. – microchao
I've updated the answer! Check if it works. – Kulasangar
Thanks a lot. But I don't know the filename. Maybe it will be ddd.log, eee.log, fff.log and so on. – microchao
What if you use a regex in order to trim the path and get only certain value as mentioned here – Kulasangar

1 Answers

0
votes

Maybe you could use the mutate in order to replace it with how your log file should be named:

if [%{[source]}] =~ /aaa/bbb/ccc.log {
  mutate {
     replace => ["%{[source]}]", "ccc.log"]
  }
}

This SO might be helpful!