fluentd logs for httpd vhosts not being forwarded to logging server

Question

I'm having a little difficulty setting up fluentd to forward httpd access logs for vhosts. I have five vhosts and I want to log access and errors for each individually and secondly, to have fluentd tail these files and forward the logs to a logging server. The first is no problem, but the second is.

With this conf file, all httpd logs from all vhosts are written to a single file and are correctly forwarded to the logging server.

<source>
  type tail
  format apache2
  tag apache.access
  path /var/log/apache/access_log
  pos_file /var/log/apache/access_log.pos
</source>

<match apache.access>
  type forward
  send_timeout 60s
  recover_wait 10s
  heartbeat_interval 1s
  phi_threshold 16
  hard_timeout 60s
  <server>
    name internal-1
    host 192.168.0.245
    port 24224
  </server>
</match>

However, when I change the path to the logfile in httpd-vhosts.conf like this:

CustomLog "/var/log/apache/internal-wiki/access_log" combined

and change td-agent.conf to this:

<source>
  type tail
  format apache2
  tag internalwiki.access
  path /var/log/apache/internal-wiki/access_log
  pos_file /var/log/apache/internal-wiki/access_log.pos
</source>

<match internalwiki.access>
  type forward
  send_timeout 60s
  recover_wait 10s
  heartbeat_interval 1s
  phi_threshold 16
  hard_timeout 60s
  <server>
    name internal-1
    host 192.168.0.245
    port 24224
  </server>
</match>

Logs are correctly written to the CustomLog, but are not being forwarded to the logging server.

The output of td-agent.log is

2015-11-09 12:23:44 +0900 [warn]: no patterns matched tag="internalwiki.access"

If I change the match type to stdout and tail td-agent.log on the local machine, this is fine.

td-agent is running as root on both servers and file perms are 666 so td-agent should be able to read access_log

Port 24224 on the logging server is open, I've checked with nmap, and I can telnet to port 24224 and see entries in td-agent.log on the logging server, so there is no problem with the network.

So, what am I doing wrong?

so if you leave the tail and just change the match to stdout does it work? if it complains about the tags, then try to set the match tag to ** for just a test and see what tag do the messages get forwarded with. — dutzu
Yes, stdout works as expected. The apache access logs are aggregated in td-agent.log on the local machine. Changed the match pattern to ** results in the same problem, the logs are not forwarded to the logging server. So this would suggest some kind of communication problem between the two servers. However, they can ping each other, the ports are open and I can telnet from the webserver to the logging server on port 24224, so I can't see where the communication is being prevented. — John Darville

John Darville John Darville · Accepted Answer · 2015-11-11T01:28:50

This was a basic mistake... The webserver was configured correctly, but the receiving server was not.

I had not included source and match patterns in the td-agent.conf on the logging server.

Adding this solved my problems.

<source>
  type forward
</source>
<match *.access>
  type stdout
</match>

So I will mark this as resolved.

fluentd logs for httpd vhosts not being forwarded to logging server

1 Answers