Reading from Kafka topic using Spark Structured Streaming: Can multi-line JSON published to Kafka topic be parsed by Spark?

1
votes

Can it be possible to parse/read the multi-line JSON published to Kafka topic by spark using Structured streaming?

1
apache-sparkapache-kafkaspark-structured-streaming

1 Answers

1
votes

if you are using spark version greater than 2.2, following would work. 

 spark.read
  .option("multiLine", true).option("mode", "PERMISSIVE")
  .json("/path/to/user.json")