copy azure blob files to azure synapse using ADF

0
votes

My source files are in azure blob storage in CSV format.

I want these files unzipped & move to Azure synapse using ADF. In the copy activity source, I have given the input for the blob storage and in sink I have given the input for the synapse. But I am not sure which copy method I need to choose in Polybase or Copy Command (Preview) or Bulk insert . I tried with Polybase and Bulk insert copy methods. But I got the below error when running the pipeline.

ErrorCode=FailedDbOperation,'Type=Microsoft.DataTransfer.Common.Shared.HybridDeliveryException,Message=Error happened when loading data into SQL Data Warehouse.,
Source=Microsoft.DataTransfer.ClientLibrary,''Type=System.Data.SqlClient.SqlException,
Message=User does not have permission to perform this action.,Source=.Net SqlClient Data Provider,
SqlErrorNumber=15247,Class=16,ErrorCode=-2146232060,State=1,Errors=[{Class=16,Number=15247,State=1,Message=User does not have permission to perform this action.,},],'

But in Azure sql Datawarehouse, I can able to insert the data using sql scripts.So, I believe above User Permission Error is not the issue to fail me pipeline.

Can you guide me please , as I am very new to Azure?

Thanks

1
azureazure-storage-blobsazure-data-factoryazure-synapse
IIRC you provide credentials to connect while setting up the datasource/sink. Use that credential to ensure it has write access to the right schema/table. – Kashyap
PS: It's (harder but) a LOT cheaper to create your own orchestration to use COPY command to load csv files into Synapse instead of Data factory. – Kashyap
Do you use the same user/account to connect to the Azure synapse? – Leon Yue
Thanks Kashyap for the reply. Can you give more info for the orchestration way please ? – Gowri Shankar
Hi Guys.. Thank you much for your ans. But the issue is in the firewall settings of the DB.It is fixed now and I am able to copy the file to dw. – Gowri Shankar

1 Answers

0
votes

The issue is caused by the database firewall settings.

We are glad to hear that it's fixed now.