Logstash 1.4.2 multiline codec

Question

Part of the log I'm trying to use:

2014-06-27 14:47:48 Error: Fatal Error (4): syntax error, unexpected 'CakeLog' (T_STRING) in [/public_html/Config/log.php, line 5]
2014-06-27 14:47:48 Error: [FatalErrorException] syntax error, unexpected 'CakeLog' (T_STRING)
Stack Trace:
#0 lib/Cake/Error/ErrorHandler.php(204): ErrorHandler::handleFatalError(4, 'syntax error, u...', '/home/...', 5)
#1 [internal function]: ErrorHandler::handleError(4, 'syntax error, u...', '/home/do...', 5, Array)
#2 /home/shared_user/cakephp-git/lib/Cake/Core/App.php(929): call_user_func('ErrorHandler::h...', 4, 'syntax error, u...', '/home/...', 5, Array)
#3 /lib/Cake/Core/App.php(902): App::_checkFatalError()
#4 [internal function]: App::shutdown()
#5 {main}

My logstash 1.4.2 config (using alsmost exact same codec as described here http://logstash.net/docs/1.4.2/codecs/multiline):

input {
  file {
    type => "cake-error"
    path => "/home/user/domains/example.com/public_html/tmp/logs/error.log"
    codec => multiline {
      pattern => "^%{TIMESTAMP_ISO8601}"
      negate => true
      what => "previous"    
    }
  }
}

Only the first error (the one without the PHP stack trace) is outputted. How can I get the other working?

Are there additional lines in the file you are trying to parse? Because multiline won't spit out the event until it figures out the next line can't be added on. — Alcanzar
If there are any new log input to your file, the second event with multiline will be output. It is because logstash determines the second event still not ending until it receives a new log. — Ben Lim

user2511309 user2511309 · Accepted Answer · 2014-08-14T18:51:04

Here is why it's not working: https://github.com/elasticsearch/logstash/issues/1482. The end of a multiline log message can only be determined when a new one comes in.

Logstash 1.4.2 multiline codec

1 Answers