2
votes

I am new to Azure data factory, currently I am working on to move the data from HDInsight Cluster with WASB(windows azure storage blob) to Azure SQL database using Azure data factory.

Before I used the Azure data factory, I used SQOOP for moving the data from HDInsight Cluster with WASB to Azure SQL database, but my requirement is to use azure data factory instead of SQOOP.

Getting started with Sqoop in HDInsight

For that I opened copy wizard from Azure Data factory and select the source as “HDFS” like this below figure.

enter image description here

But I am not clear about which credentials are entered under the HDFS source connection window, even I read the below documentation about Linked service properties of HDFS

Move data from on-premises HDFS using Azure Data Factory

Can you please explain which credentials are entered under HDFS connection and also tell me how to move the data from HDFS to Azure SQL database?

1

1 Answers

-1
votes

If you have an Azure Data Factory instance on Azure portal, please click Copy data(preview) button as Fig 1 below to open a new window(Fig 2) for pipeline setup wizard.

Fig 1. Copy data (preview)

enter image description here

Fig 2. Pipeline wizard

enter image description here Then, you just need to follow the wizard to set the information of source for HDFS and target for SQL Azure to be ready for running the pipeline.

More details for configuring two dataset of HDFS source & SQL Azure targat, you can refer to two offical tutorials for HDFS & SQL Azure.

Meanwhile, the other way to move data from HDInsight to SQL Azure is using [Sqoop][5], and you can refer to the offical blog Getting started with Sqoop in HDInsight to know the usage.