0
votes

I'm trying to call an HTTP GET API using HTTP batch source plugin in data fusion. The response of the API is a complex dynamic nested json because of which I cannot manually specify the output schema. Is there anyway to overcome this problem.

Thanks in advance!

1

1 Answers

1
votes

Looking into the CDAP documentation for Parsing nested JSON events, you might find that this plugin purely uses JsonPath library for JSON key-values extractions with significant expressions usage, that could be reasonable for dynamic key JSON parsing:

The "root member object" for parsing any JSON is referred to as $, regardless of whether it's an array or an object. It also uses either dot notation or bracket notation for defining the levels of parsing. For example: $.employee.name or $['employee']['name']

You can always refer to the JSON Path CDAP directive guidance to get familiar with dedicated functions and operators to manage filtering of the dynamic data.