Filebeat sends logs to all logstash intstances

0
votes

I am doing a prove of concept installing ELK to manage all the logs from different applications. What I have is two instances of Elasticsearch (9200 9201), Kibana (5601 5602) and Logstash (5044 5045) and one nginx acting as a reverse proxy and two different machines running different applications and sending their logs to Logstash through Filebeat.

I want to set permissions to the users, so some users can access to the logs from machine A and some others can access to the logs in machine B, but when I start Filebeat in both machines, each of them should send their own logs to different port in logstash as it is configured like this but what I get when I access to kibana (both ports) is all the logs from both machines.

Is it possible to split the logs in the different instances of the installation?

Thanks in advance.

2
nginxlogstashelastic-stackfilebeat
Are you trying to run an Elasticsearch cluster or are you trying to run two independent Elasticsearch nodes where data from machine A goes to one node and data from machine B goes to the other node? – A J
They are two independent nodes as you describe in your question. The purpose of it is to divide the access to differente environments to the users. – kendrick john

2 Answers

1
votes

When you run two Elasticsearch instances on the same host they will automatically form a cluster unless you make changes to the configuration to prevent this behavior.

You can list the nodes in the cluster by running the following command. If there is more than one node listed then your two nodes have formed a cluster.

curl -XGET 'localhost:9200/_cat/nodes?v&pretty'

You need to set unique cluster.name values in each node's configuration. The default value is elasticsearch. When the two nodes discover each other and their cluster.name is the same they form a cluster.

-1
votes

It is right to change both clusternames and give them different names, furthermore to keep both nodes independent it is necessary to edit the filebeat template because by default it is pointing to elasticsearch:9200.