Logstash Configuration Change -- Reparse Old Logs

Question

I'm trying to re-parse old logs with a newer configuration; is there any way to do this?

I'm using two servers: one with logstash-forwarder (lumberjack) and one with elasticsearch and logstash. (*All of these are the most recent releases.)

I have seen: http://logstash.net/docs/1.4.2/inputs/file#sincedb_path and do not have a sincedb on the forwarder server. (*I'm aware that sincedb is optional.)

So if sincedb is optional, where is the tail located -- obviously the logs are being tracked, but I cannot find where.

Thanks in advance!

Answer 1

The documentation link that you sent is for logstash, not logstash-forwarder.

logstash-forwarder put its registry in a .logstash-forwarder file. Sometimes this file is in the startup directory (which can change if you're starting it by hand!), but check your startup script.

logstash-forwarder will process any active files that match the pattern given. Older versions defined "active" as "within 24 hours"; if you compile from source, you can set this in the configuration ("dead time", I believe). Otherwise, you might have to update the modification time on the file (UNIX: touch).

Note that this will not update any records in Elasticsearch - new documents will be inserted.

Good luck!