Not able ot fetch the indices related data in kibana

Question

I am able to create the indices using logstash.conf. My input type is gelf. I am sending the logstash logs to kibana. Here is my logstash.conf

input 
{ gelf { } 
} 
output 

{
 stdout { codec => rubydebug }
 elasticsearch {
  hosts =>  ["elk.lera.com:80"]
  index => "templeton-math-%{+YYYY.MM.dd}"
 }

elasticsearch {
  hosts =>  ["elk.lera.com:80"]
  index => "templeton-science-%{+YYYY.MM.dd}"
 }
 elasticsearch {
  hosts =>  ["elk.lera.com:80"]
  index => "templeton-bio-%{+YYYY.MM.dd}"
 }
 elasticsearch {
  hosts =>  ["elk.lera.com:80"]
  index => "templeton-lang-%{+YYYY.MM.dd}"
 }
}

Issue: logs are sent to all the indices now. I would like to send the logs to respective indices.

I have added like

if[tag] == "templeton-math"{
elasticsearch {
  hosts =>  ["elk.lera.com:80"]
  index => "templeton-math-%{+YYYY.MM.dd}"
 }
}

It is giving an error INFO logstash.agent - No persistent UUID file found. Generating new UUID {:uuid=>"67f7a48e-fc7c-499b-85a0-3fd6979f88f6", :path=>"/var/lib/logstash/uuid"} 14:58:14.308 [LogStash::Runner] ERROR logstash.agent - Cannot create pipeline {:reason=>"Expected one of #, => at line 22, column 9 (byte 179) after output \n\n{\n\n elasticsearch {\n hosts "} 2017-10-11 14:58:14,355 Api Webserver ERROR No log4j2 configuration file found. Using default configuration: logging only errors to the console.

Please paste your final logstash config file after adding all the respective ES hosts condition. — Hatim Stovewala

Hatim Stovewala Hatim Stovewala · Accepted Answer · 2017-10-11T16:34:59

Try this.

output {
    stdout { codec => rubydebug }

    if [tag] == "templeton-math" {
        elasticsearch {
            hosts =>  ["elk.lera.com:80"]
            index => "templeton-math-%{+YYYY.MM.dd}"
        }
    }

    if [tag] == "templeton-science" {
        elasticsearch {
            hosts =>  ["elk.lera.com:80"]
            index => "templeton-science-%{+YYYY.MM.dd}"
        }
    }

    if [tag] == "templeton-bio" {
        elasticsearch {
            hosts =>  ["elk.lera.com:80"]
            index => "templeton-bio-%{+YYYY.MM.dd}"
        }

    }

    if [tag] == "templeton-lang" {
        elasticsearch {
            hosts =>  ["elk.lera.com:80"]
            index => "templeton-lang-%{+YYYY.MM.dd}"
        }
    }
}

Not able ot fetch the indices related data in kibana

1 Answers