Whats the best way to send XML data to Kafka topic?

Question

I am trying to send a XML data to Kafka topic using Kafka 0.9.0 version Java API, because from 0.9.0 version they are suggesting to use Java API instead Scala API to get better performance.

http://kafka.apache.org/090/documentation.html#producerapi

My XML File has around 5 MB data, what is the XML API i can use to read XML file and convert as String and then send that String to Kafka Topic.

Or can i convert that XML as JSON data and send it to Kafka? also should i consider splitting my XML data to multiple small amount of data and send to Kafka Topic?

So, your question is "how do I read a file into a String"? The format of that file doesn't really matter — OneCricketeer
This sounds like a question that will give answers based on opinion instead of facts. Can you rephrase it? — qwelyt
@cricket_007 :My question is specific to Kafka, how do i effectively handle XML data, not just converting xml to string OR xml to json.. — Shankar

Kris Kris · Accepted Answer · 2016-10-08T11:18:48

The API to read/write XML is up to the programmer/environment. Further, Apache Kafka does not enforce any particular format for event data beyond a simple key/value model. It will work equally well with XML, JSON,Avro or any other data format. It's recommended that any format, be it XML, JSON, provided it should be used in a consistent manner. if you are starting a new project with Kafka, there are things to consider like performance, ease of use, programming language support etc before selecting and use the correct serialise format etc. Avro is one of the most successful format for stream data.

Whats the best way to send XML data to Kafka topic?

1 Answers