some issue with grok timestamp pattern
2020-3-4 10:22:37 >> this will match with this pattern %{TIMESTAMP_ISO8601:my_time}
2020-3-4 0:2:37 >> this will fail with this pattern %{TIMESTAMP_ISO8601:my_time}
also tried to match the pattern by using this separate pattern like YEAR MONTH AND DAY but again it will break when it reaches time %{HOUR}:%{MINUTE}:%{SECOND}. Any idea ?
The issue is with how the minute pattern is defined in logstash: (?:[0-5][0-9]). This pattern expects a two digit minute number, which breaks in your second case (2020-3-4 0:2:37).
I've changed the pattern to accept a one-digit number of minutes: (?:[0-5][0-9]|[0-9])
You can then use this custom pattern:
(?<my_time>%{YEAR}-%{MONTHNUM}-%{MONTHDAY}[T ]%{HOUR}:?(?:[0-5][0-9]|[0-9])(?::?%{SECOND})?%{ISO8601_TIMEZONE}?)
which is the TIMESTAMP_ISO8601 with MINUTE replaced by my pattern.
(?:[0-5][0-9])) expects a two digit numbers in all cases. - baudsp