I have an ADF pipeline loading raw log data as JSON files into a Data Lake Gen 2 container.
We now want to extract information from those JSON files and I am trying to find the best way to get information from said files. I found that Azure Data Lake Analytics and U-SQL scripts are pretty powerful and also cheap, but they require a steep learning curve.
Is there a recommended way to parse JSON files and extract information from them? Would Data Lake tables be an adequate storage for this extracted information and act then as a source for downstream reporting process?
And finally, will Azure Data Factory ever be able to parse nested arrays JSONs?