Unable to set pipeline from filebeat to logstash

Question

I have the filebeat running in a docker container and logstash running in a different docker container.

In configuration of filebeat.yml, the logstash IP is set as with the IP of logstash as http://:5044.

Facing the below error

WARN DNS lookup failure "http://172.17.0.2:5044": lookup http://localhost:5044/: invalid domain name 2017/04/14 14:16:51.537977 single.go:126: INFO Connecting error publishing events (retrying): lookup http://localhost:5044/: invalid domain name 2017/04/14 14:16:51.538000 single.go:152: INFO send fail

Configuration of filebeat.yml with regards to log stash configuration

output: 
 logstash: 
  enabled: true
 hosts: 
  - "172.17.0.2:5044"

Should the docker ip of log stash be used or a separate IP be used?

This is a configuration problem. Please provide the configuration you are using. — A J
In filebeat.yml, logstash as output is configured with only the below field uncommented, ### Logstash as output logstash: # The Logstash hosts hosts: ["172.17.0.2:5044/"] Also, the prospector is configured to a path as below paths: - /var/log/*.log Apart from these two none of the fields in filebeat.yml are uncommented and used. — Sudhi

A J A J · Accepted Answer · 2017-04-17T13:35:36

The connection between Beats and Logstash is not based on the HTTP protocol so do not configure the hosts option with a URL. Each Logstash host should be of the format host:port. As you can see from the error message it is trying to resolve the full value you specified as a domain name which is wrong (i.e. it's doing the equivalent of nslookup "http://localhost:5044/").

The indentation in the config shown in the question is off as well. The hosts setting should be a child of logstash so it needs to be indented. Try using this instead to avoid any indentation issues:

output.logstash.hosts: ['logstash:5044']

If both the Logstash and Filebeat containers are in the same Docker environment then you can link the two containers and use the Logstash container's name as the hostname in your config file. If they are on separate hosts then you need to expose port 5044 from LS and use the host machine's IP in your Filebeat configuration.

Unable to set pipeline from filebeat to logstash

1 Answers