Kusto - Extract string field into new columns using parse operator

Question

I have got a column in my kusto table which contains some log messages of the following form

msg1

2020-10-29T12:57:08+00:00 dc1-k30-asw05.nw.domain.com cron: :- addNeighbor: Created neighbor ac:1f:6b:8b:09:99 on Vlan100

msg2

2020-10-29T15:55:20+00:00 dc1-k12-asw30.domain.com cron: :- validatePortSpeed: Unable to validate speed for port 100000000005c. Not supported by platform

Now I want to extract some sub strings from it into separate columns, the sub strings I am interested in are the timestamp value (2020-11-02T10:31:21+00:00) this is basically the start of the line, the programwhich emitted the log which is cron in this case and lastly the actual log message. So I am using the parse operator to do that instead of using multiple extract and evaluating the pattern multiple times.

parse kind = regex message with @".*?" syslogTime:string @"\s+.*domain.com .*?" program:string @": " msg

This query results in following outputs

"syslogTime": 2020-10-29T12:57:08+00:00,
"program":   cron: :- addNeighbor,
"msg": Created neighbor ac:1f:6b:8b:09:99 on Vlan100,

"syslogTime": 2020-10-29T15:55:20+00:00,
"program":  cron: :- validatePortSpeed,
"msg": Unable to validate speed for port 100000000005c. Not supported by platform,

As seen above the first field is parsed correctly however for the second column program the value matched in incorrect, the regex engine is doing a greedy match till the second : even though I have used a non-greedy quantifier @"\s+.*domain.com .*?".

I have also tried using the ungreedy flag U as to the parse query but with that the last part of the message is not being captured.

parse kind = regex flags=U message with @".*" syslogTime:string @"\s+.*domain.com.*" program:string @": " msg

The output I get

"syslogTime": 2020-11-02T08:47:35+00:00,
"program":  cron,
"msg":

"syslogTime": 2020-10-29T15:53:36+00:00,
"program":  cron,
"msg":

For the second column i.e. program I want to match till the first ":". I have tried multiple variations of the above queries but none of them yielded the expected output. So I am not sure what am I missing here.

Rohit Rohit · Accepted Answer · 2020-11-02T11:56:33

I was able to solve this by modifying the parse expression to following.

parse kind = regex flags=U message with  @".*" syslogTime:string @"\s+.*domain.com.*" program:string  @": " msg @"$"

Kusto - Extract string field into new columns using parse operator

1 Answers