Logstash not opening input port for filebeat

Question

I am having trouble in setting up the ELK stack,

Filebeat is configured,

filebeat.prospectors:
- input_type: log
    paths:
       - /var/log/syslog
       #- c:\programdata\elasticsearch\logs\*
       document_type: syslog

output.logstash:
  # The Logstash hosts
   hosts: ["localhost:5044"]
   bulk_max_size: 1024

These are the input files for logstash,

For input,

input {
   beats {
     type => beats
     port => 5044
   }
 }

For Output,

 output {
  elasticsearch {
    hosts => ["localhost:9200"]
    manage_template => false
    index => "%{[@metadata][beat]}-%{+YYYY.MM.dd}"
    document_type => "%{[@metadata][type]}"
  }
 }

I can see in the filebeat logs, that it is giving me this error,

 2017-04-19T15:30:55+05:30 ERR Connecting error publishing events (retrying): dial tcp 127.0.0.1:5044: getsockopt: connection refused

Clearly, the port is not opened by logstash, Also I can see via netstat that port is not open for listening.

Why the port is not opened by logstash, am I missing something ? Also, when the port will open, will it automatically keeps sending the data to logstash, and logstash then to elasticsearch ?

I would run Logstash with debug logging enabled to see why it's not binding to port 5044 and provide logs in the question. By default it should bind to all interfaces (0.0.0.0). Until netstat shows that Logstash is listening I would leave Filebeat out of the equation. — A J

sysadmin1138 sysadmin1138 · Accepted Answer · 2017-04-19T13:46:56

I'm somewhat confused by why you have filebeat polling the logs, when you have a full logstash instance also on the same box. Logstash can do what Filebeat can and avoid this whole problem.

input {
  file {
    path => [ "/var/log/syslog" ]
    type => "syslog"
  }
}

However, you wanted to know why Logstash wasn't opening up the port. I suggest changing your beats input to be this, to test it out:

input {
   beats {
     type => beats
     host => "localhost"
     port => 5044
   }
}

Which will tell the beats input to bind to 'localhost' specifically, which is where Filebeat is expecting to find a listening port.

Logstash not opening input port for filebeat

2 Answers