Reading log files from zip file using logstash-forwarding

Question

I am using logstash-forwarder to send the logs to logstash server using lumberjack input plugin.

One of my requirement is to read logs from old archived .ZIP files using logstash-forwarder. I tried to use .zip in my configuration file as follows

"files": [
    {
      "paths": ["C:\\jboss-eap-6.1.0.Alpha\\standalone\\log\\**\\*.zip"
     }
 ]

But it is not working.

Please let me know, if it is possible to read logs from .ZIP file.

thanks in advance.

Filebeat is the replacement of logstash-forwarder. Both don't support the feature but there is an open issue for this: github.com/elastic/beats/issues/637 I recommend you to follow the issue for further details and progress. — ruflin

Alain Collins Alain Collins · Accepted Answer · 2016-01-08T14:45:57

logstash-forwarder is really designed to send the latest log information to logstash. It's basically doing a 'tail -f' of your log file, and sending the new stuff. As such, it (and most shippers) don't handle gzip'ed files.

One option is to unzip them and send them.

There's also a gzip_lines codec that you can install, though I wasn't able to get it to work under logstash 1.5.

The S3 input filter handles zipped files (since it knows you're not streaming from S3).

Reading log files from zip file using logstash-forwarding

1 Answers