I'm having a little difficulty setting up fluentd to forward httpd access logs for vhosts. I have five vhosts and I want to log access and errors for each individually and secondly, to have fluentd tail these files and forward the logs to a logging server. The first is no problem, but the second is.
With this conf file, all httpd logs from all vhosts are written to a single file and are correctly forwarded to the logging server.
<source>
type tail
format apache2
tag apache.access
path /var/log/apache/access_log
pos_file /var/log/apache/access_log.pos
</source>
<match apache.access>
type forward
send_timeout 60s
recover_wait 10s
heartbeat_interval 1s
phi_threshold 16
hard_timeout 60s
<server>
name internal-1
host 192.168.0.245
port 24224
</server>
</match>
However, when I change the path to the logfile in httpd-vhosts.conf like this:
CustomLog "/var/log/apache/internal-wiki/access_log" combined
and change td-agent.conf to this:
<source>
type tail
format apache2
tag internalwiki.access
path /var/log/apache/internal-wiki/access_log
pos_file /var/log/apache/internal-wiki/access_log.pos
</source>
<match internalwiki.access>
type forward
send_timeout 60s
recover_wait 10s
heartbeat_interval 1s
phi_threshold 16
hard_timeout 60s
<server>
name internal-1
host 192.168.0.245
port 24224
</server>
</match>
Logs are correctly written to the CustomLog, but are not being forwarded to the logging server.
The output of td-agent.log is
2015-11-09 12:23:44 +0900 [warn]: no patterns matched tag="internalwiki.access"
If I change the match type to stdout and tail td-agent.log on the local machine, this is fine.
td-agent is running as root on both servers and file perms are 666 so td-agent should be able to read access_log
Port 24224 on the logging server is open, I've checked with nmap, and I can telnet to port 24224 and see entries in td-agent.log on the logging server, so there is no problem with the network.
So, what am I doing wrong?