Splunk splitting xml log event

Question

We have logs that log an event to a single file. Each log entry looks something like this:

<LogEntry>
  <UserName>IIS APPPOOL\ASP.NET v4.0</UserName>
  <TimeStamp>02/28/2014 13:54:17</TimeStamp>
  <ThreadName>20</ThreadName>
  <CorrelationId>7a0d464d-556c-4d47-820f-0cf01322e54c</CorrelationId>
  <LoggerName>-Api-booking</LoggerName>
  <Level>INFO</Level>
  <Identity></Identity>
  <Domain>API-1-130380690118132000</Domain>
  <CreatedOn>02/28/2014 13:54:22</CreatedOn>
  <ExceptionObject />
  <RenderedMessage>"7a0d464d-556c-4d47-820f-0cf01322e54c" - "GET https://myapi.com/booking" - API-"Response": 
"Unauthorized"</RenderedMessage>
</LogEntry>

When we import these logs into Splunk, the log entry is split up incorrectly into 3 parts e.g.

1-

<LogEntry>
  <UserName>IIS APPPOOL\ASP.NET v4.0</UserName>

2-

<CreatedOn>02/28/2014 02:57:55</CreatedOn>
  <ExceptionObject />
  <RenderedMessage>"66d8cdda-ff62-480a-b7d2-ec175b151e5f" - "POST https://myapi.com/booking" - API-"Response": 
"Bad Request"</RenderedMessage>
</LogEntry>

3-

<TimeStamp>02/28/2014 02:57:29</TimeStamp>
  <ThreadName>21</ThreadName>
  <CorrelationId>66d8cdda-ff62-480a-b7d2-ec175b151e5f</CorrelationId>
  <LoggerName>-Api-booking</LoggerName>
  <Level>INFO</Level>
  <Identity></Identity>
  <Domain>/LM/W3SVC/1/ROOT/Api-1-130380256918440000</Domain>

How can I configure Splunk to see these as a single log event?

hobbes3 hobbes3 · Accepted Answer · 2014-03-13T23:30:21

props.conf (pay attention to LINE_BREAKER)

[your_xml_sourcetype]
TIME_PREFIX = <TimeStamp>
MAX_TIMESTAMP_LOOKAHEAD = 19
TZ = GMT
# A performance tweak is to disable SHOULD_LINEMERGE and then set the 
# LINE_BREAKER to "line ending characters coming before a new time stamp"
# (note the direct link of the TIME_FORMAT to the regex of LINE_BREAKER).
TIME_FORMAT = %m/%d/%Y %T
LINE_BREAKER = ([\r\n]+)<LogEntry>
SHOULD_LINEMERGE = False
# 10000 is default, should be set on a case by case basis
TRUNCATE = 5000

# If the data does not have nice key=value pairs, (or some other readily
# machine parseable format, like JSON or XML), set KV_MODE = none so that
# Splunk doesn't spin its wheels on attempting to look for key = value
# pairs which don't exist.
KV_MODE = xml

# Leaving PUNCT enabled can impact indexing performance. Customers can
# comment this line if they need to use PUNCT
ANNOTATE_PUNCT = false

More information here: http://docs.splunk.com/Documentation/Splunk/latest/Admin/Propsconf

Splunk splitting xml log event

1 Answers