Why is Splunk not showing miliseconds for JSON?

Question

hello,

Splunk is not showing up miliseconds for JSON logs. I have find some Questions and Answers here in splunk community, but without success.

Description:

I have HFs, indexer cluster and search head cluster.

HF props.conf

    [k8s:dev]
#temporary removed to fix 123123
#INDEXED_EXTRACTIONS = JSON
TIME_PREFIX = {\\"@timestamp\\":\\"
TIME_FORMAT = %Y-%m-%dT%H:%M:%S.%6N
TRUNCATE = 200000
TRANSFORMS-discard_events = setnull_whitespace_indented,setnull_debug_logging
SEDCMD-RemoveLogProp = s/("log":)(.*)(?="stream":)//

HF transforms.conf

[setnull_java_stacktrace_starttab]
SOURCE_KEY = field:log
REGEX = ^\tat\s.*
DEST_KEY = queue
FORMAT = nullQueue

[setnull_whitespace_indented]
SOURCE_KEY = field:log
REGEX = ^\s+.*
DEST_KEY = queue
FORMAT = nullQueue

[setnull_debug_logging]
SOURCE_KEY = field:log
REGEX = .*?\sDEBUG\s
DEST_KEY = queue
FORMAT = nullQueue

Search props.conf

#workaround, see 123123


[k8s:dev]
KV_MODE = json

Everything looks fine in web ADD DATA in HF and SEARCH too.

But not when I search it.

I can insert only part of the JSON.

{"log":"{\"@timestamp\":\"2021-08-03T09:00:57.539+02:00\",\"@version\":\"1\",\"message\":

Question is:

1.what am I doing wrong?

Is it possible to configure TIME_PREFIX and TIME_FORMAT for KV_MODE on search? Because as I know they are used in HF during parsing.
Is it possible to configure KV_MODE?

Thank you very much for your suggestions.

warren warren · Accepted Answer · 2021-08-03T19:07:06

The value that actually goes into _time is in Unix epoch seconds.

It doesn't matter what precision you look for with TIME_FORMAT= ... it still only goes into _time in whole seconds.

If you want to keep the higher-resolution value for use elsehow, you'll need to add a specific field extraction for them.

Since this is JSON, you can probably do the following in a search:

...
| eval timestamp=strftime('@timestamp',"%Y-%m-%dT%H:%M:%S.%6N"))
...

Why is Splunk not showing miliseconds for JSON?

1 Answers