I am exploring Azure Data Lake and I am new to this field. I explored many things and read many articles. Basically I have to develop Power BI dashboard from data of different sources.
In classic SQL Server stack I can write an ETL (Extract, Transform, Load) process to bring the data from my system databases into the Data Warehouse database. Then use that Data Warehouse with Power BI by using SSAS etc.
But I want to use Azure Data Lake and I explored Azure Data Lake Store and Azure Data Lake Analytic(U-SQL). I draw following architecture diagram.
- Is there any thing which I am missing in current flow of the application?
- I can get data directly from Azure Data Lake using Power BI so there is no need of Data Warehouse. Am I right?
- I can create a database in Azure Data Lake is that will be my Data Warehouse?
- What will be the best format for the Output data from Original file in Azure Data Lake e.g .csv?