just trying to build a test app for learning purposes on how to collect Docker logs to EFK (Elasticsearch7.10.1 + Fluentd + Kibana7.10.1) stack.
Elastic starts up fine and is reachable from http://localhost:5601/
But fluentd-* is not available as an index pattern, I assume do to the error I am getting on the logs from kibana:
kibana_1 | {"type":"log","@timestamp":"2021-01-03T23:46:32Z","tags":["error","elasticsearch","monitoring"],"pid":6,"message":"Request error, retrying\nGET http://elasticsearch:9200/_xpack => connect ECONNREFUSED 172.20.0.3:9200"}
kibana_1 | {"type":"log","@timestamp":"2021-01-03T23:46:32Z","tags":["warning","elasticsearch","monitoring"],"pid":6,"message":"Unable to revive connection: http://elasticsearch:9200/"}
kibana_1 | {"type":"log","@timestamp":"2021-01-03T23:46:32Z","tags":["warning","elasticsearch","monitoring"],"pid":6,"message":"No living connections"}
kibana_1 | {"type":"log","@timestamp":"2021-01-03T23:46:32Z","tags":["warning","plugins","licensing"],"pid":6,"message":"License information could not be obtained from Elasticsearch due to Error: No Living connections error"}
kibana_1 | {"type":"log","@timestamp":"2021-01-03T23:46:32Z","tags":["warning","plugins","monitoring","monitoring"],"pid":6,"message":"X-Pack Monitoring Cluster Alerts will not be available: No Living connections"}
kibana_1 | {"type":"log","@timestamp":"2021-01-03T23:46:32Z","tags":["error","elasticsearch","data"],"pid":6,"message":"[ConnectionError]: connect ECONNREFUSED 172.20.0.3:9200"}
kibana_1 | {"type":"log","@timestamp":"2021-01-03T23:46:32Z","tags":["error","savedobjects-service"],"pid":6,"message":"Unable to retrieve version information from Elasticsearch nodes."}
172.20.0.3:9200 and http://elasticsearch:9200/ are not reachable through browser
http://localhost:9200/ is reachable
What am I missing? I have been working on this for a week and don't know where to look anymore, thanks!
Docker-compose.yml
version: '2'
services:
web:
image: httpd
ports:
- "8080:80"
links:
- fluentd
logging:
driver: "fluentd"
options:
fluentd-address: localhost:24224
tag: httpd.access
fluentd:
build: ./fluentd
volumes:
- ./fluentd/conf
links:
- "elasticsearch"
ports:
- "24224:24224"
- "24224:24224/udp"
elasticsearch:
image: elasticsearch:7.10.1
environment:
- "network.host=0.0.0.0"
- "transport.host=127.0.0.1"
expose:
- 9200
ports:
- "9200:9200"
kibana:
image: kibana:7.10.1
environment:
server.host: 0.0.0.0
elasticsearch.hosts: http://localhost:9200
ports:
- "5601:5601"
Dockerfile
# fluentd/Dockerfile
FROM fluent/fluentd:v1.11.5-debian-1.0
RUN ["gem", "install", "fluent-plugin-elasticsearch", "--no-document", "--version", "4.0.4"]
fluentd.conf file
# fluentd/conf/fluent.conf
<source>
@type forward
port 24224
bind 0.0.0.0
</source>
<match *.**>
@type copy
<store>
@type elasticsearch
host elasticsearch
port 9200
logstash_format true
logstash_prefix fluentd
logstash_dateformat %Y%m%d
include_tag_key true
type_name access_log
tag_key @log_name
flush_interval 1s
</store>
<store>
@type stdout
</store>
</match>