Azure SQL Data Warehouse

2
votes

It would be nice if I know the various ways to load data into Azure SQL Data Warehouse.

I could connect to Azure SQL DW using JDBC from my Ubuntu machine.

  1. Is there any direct way to load from Azure Blob storage into Azure SQL DW without using Azure Data Factory ?

  2. Ways to load data from Ubuntu to Azure SQL DW.

  3. Java API to talk to Azure.

  4. Directly loading data from HDInsights to SQL DW.

Thanks

2
javasqlazureazure-sqldw
"Java API to talk to Azure"? Specifically to connect to Azure SQL DW and run queries? Or to do admin tasks in Azure? Can you clarify what tasks? - GregGalloway

2 Answers

6
votes

The best and fastest way to load data to Azure SQL DW is to use Polybase to load delimited flat files sitting in Azure Blob Storage. See Polybase described here.

For loading from HDInsight I suppose you could use SQOOP (though I haven't tried that to load Azure SQL DW) but the best approach is to have HDInsight write to a flat file (for example by inserting to an external table in Hadoop) and then using Polybase in Azure SQL DW to read the files.

Sorry I can't help with your other questions about Java and Ubuntu.

1
votes

this hint is a bit crazy but it will work. take a local sql server vm and configure Azure SQL DB Transactional Replication. -> https://channel9.msdn.com/Shows/Data-Exposed/Azure-SQL-DB-Transactional-Replication

Going this way will fill up your azure sql db. you have to connect ubuntu(JDBC) to insert all content to sql server. This will replicate your data to azure sql db. And azure sql db can be easily migrated to azure sql dw with two steps ;-)

https://azure.microsoft.com/en-us/documentation/articles/sql-data-warehouse-get-started-provision/

your advantage, you connect with low latency to your local sql server and just wait until Transactional Replication is done. ;-)

regards patrick