Extract data with Logstash and Xpath

Question

I want to extract data (timestamp and message) via Xpath plugin in Logstash from XML files to display only them in fields in kibana.

XML sample:

<log4j:event logger="logger4test" timestamp="1496297008092"><log4j:message>sample message</log4j:message></log4j:event>

Logstash conf:

input {
    file {
            path => "/opt/logs/*"
            start_position => beginning
            sincedb_path => "/dev/null"
            type => "xml"
    }
}

filter {
    xml {
            remove_namespaces => true
            source => "file"
            store_xml => false
            xpath => [
                    "//event/@timestamp", "time",
                    "//message/text()", "lmessage"
            ]
    }

    if [type] == "xml" {
            mutate {
                    replace => [
                            "time", "%{time}",
                            "lmessage", "%{lmessage}"
                    ]
            }
    }
}

However the result is in lmessage is the value %{lmessage} and in time: %{time} and not as expected the real message and time.

Can please somebody help me? There are also no errors in the Logstash logs.

Johannes J Johannes J · Accepted Answer · 2017-07-24T08:41:16

Resolved the issue myself. The problem was the "source" field. I had to replaced "file" with "message" because logstash need this information from where it reads iits data and "message" is the fitting attribute of logstash.

xml {
        remove_namespaces => true
        source => "message"
        store_xml => false
        xpath => [
                "//event/@timestamp", "time",
                "//message/text()", "lmessage"
        ]
}

Extract data with Logstash and Xpath

1 Answers