How to copy data from ADLS Gen2 to on prem hdfs?

0
votes

I have a storage account and Gen2 ADLS. I created a ADF pipeline to copy data from HDFS to ADLS and now I want to create pipeline to copy back the data to HDFS however ADF connections does not support HDFS as a sink. The data in ADLS GEN2 is in parquet format. Are there any ways I could copy these files back to HDFS?

1
azurehadoophdfsazure-data-factoryon-prem
Data Factory can not achieve that directly. You need think about achieve that in code level, then call it in Data Factory. – Leon Yue

1 Answers

0
votes

As you said, Data Factory does not support HDFS as a sink, we can not achieve that directly. You need think about achieve that in code level, then call it in Data Factory. Using Azure Function or Databricks: enter image description here

HTH.