Logstash Crashes When Filebeat Running

Question

Logstash is running well without beats configuration over tcp and I can see the all logs when I send over tcp.

input {tcp{
   port => 8500   }
}
output {   elasticsearch { hosts => ["elasticsearch:9200"] }

}

But I want to send logs to logstash from filebeat. I changed logstash config with this:

input {
  beats {
    port => 5044
  }
}

output {   elasticsearch { hosts => ["elasticsearch:9200"] }

}

This is docker run for logstash

docker run -d -p 8500:8500  -h logstash --name logstash --link elasticsearch:elasticsearch -v C:\elk2\config-dir:/config-dir docker.elastic.co/logstash/logstash:7.5.2 -f /config-dir/logstash.conf

I am running filebeat in docker with following:

docker run -d docker.elastic.co/beats/filebeat:6.8.6 setup --template -E output.logstash.enabled=true -E 'output.logstash.hosts=["127.0.0.1:5044"]'

But whenever I run filebeat, logstash and filenbeat containers are being stopped:

There is no docker log meaningfull:

[2020-01-24T14:13:37,104][INFO ][logstash.agent           ] Successfully started Logstash API endpoint {:port=>9600}
[2020-01-24T14:13:37,978][INFO ][logstash.javapipeline    ] Pipeline terminated {"pipeline.id"=>".monitoring-logstash"}
[2020-01-24T14:13:38,657][INFO ][logstash.runner          ] Logstash shut down.

This seems more like containers issue than the title of your question suggests. Your config file looks fine. — JBone
The different type of input-plugin (beats vs tcp) won't be the cause since the beats-input-plugin also listens on tcp. — apt-get_install_skill
Furthermore, I assume that Filebeat and Logstash each run in a seperate container? If yes, then -E 'output.logstash.hosts=["127.0.0.1:5044"] is not correct because 127.0.0.1 (localhost) will refer to your filebeat docker container and not your system/machine. — apt-get_install_skill
@apt-get_install_skill yes they run at separate containers. What should be the correct value? — mgnfcnt
You would need to link the filbeat and logstash container as well, since they have to communicate with each other. The alternative is to create a docker-network where you put all of your containers into. (See docs.docker.com/engine/reference/run/#network-settings). From the docs: Containers can communicate via their IP addresses by default. To communicate by name, they must be linked. — apt-get_install_skill

Rajitha Abeysekara Rajitha Abeysekara · Accepted Answer · 2020-01-28T11:34:03

You need to expose your beat listening port

docker run -d -p 5044:5044  -h logstash --name logstash --link elasticsearch:elasticsearch -v C:\elk2\config-dir:/config-dir docker.elastic.co/logstash/logstash:7.5.2 -f /config-dir/logstash.conf

Logstash Crashes When Filebeat Running

1 Answers