I am trying to feed Zabbix log data into an ELK stack to do some timing correlations. However, I'm not getting the filter specified by my Logstash centralized server configuration that I would if it was acting as an agent. I'm using Filebeat to send the log entries to Logstash for processing. However, the fields.tags line is not being acknowledged by Logstash, but is visible in Elasticsearch/Kibana. Here are my configs.
filebeat:
prospectors:
-
paths:
- /var/log/zabbix/zabbix_proxy.log
input_type: log
fields:
tags: ["zabbix", "zabbix-proxy"]
registery_file: /var/lib/filebeat/registry
output:
logstash:
hosts: ["elkls.com:5044"]
logging:
files:
rotateeverybytes: 12345656
and then my Logstash configuration file
input {
beats {
port => 5044
}
}
filter {
if "zabbix" in [fields.tags] {
grok {
match => {
"message" => {
"filter stuff here"
}
}
}
}
}
output {
elasticsearch {
hosts => [ "elkhost.com:9200"]
}
}
Logstash isn't acting upon the fields.tags line, and filtering/cutting up the log lines being passed to it from Filebeat. Am I accessing those variables correctly? I do something similar with other files, but only when Logstash is acting as an agent, and reading directly from a file.
