How to import Varnishlog file into Elasticsearch using Filebeat?

Question

I would to import a "varnishcsa.log" into Elasticsearch to, at the end, discover/visualize data with Kibana.

I am begining to use the Elastic Stack : Elasticsearch > Filebeat > Kibana.

There is no default "Varnish" build-in Filebeat module.
I did some tests (filebeat, logstash), following some instructions... but without success.

• https://www.elastic.co/guide/en/logstash/current/plugins-inputs-varnishlog.html
• https://www.elastic.co/guide/en/logstash/current/filebeat-modules.html

I am searching a way to import a Varnishlog file, like the following (working) one for a "mysql-slow-queries.log".

# run Elasticsearch and Kibana
./elasticsearch/bin/elasticsearch
./kibana/bin/kibana

# Import MySQL Slow Queries logs using Filebeat
./filebeat/filebeat -e --modules=mysql --setup  -M \
"mysql.slowlog.var.paths=[/path/to/mysql-slow-queries.log]"

Can you help me?
I am a beginner, so do not hesitate to give examples... :).
Thank you.

Answer 1

You don't need a default module to send your logs with Filebeat. See an example config here and use it to configure filebeat to send your varnish logs to Elasticsearch.

- type: log
  paths:
    /var/log/varnish/varnishncsa.log
  fields:
  log_type: varnish
  fields_under_root: true