Copying a file from an FTP location into Azure DataLake

0
votes

I have followed all steps shown in the MSDN documentation to Copy File from FTP.

So far, the data sets are created, linked servers were created, the pipeline is created. The diagram for the pipeline shows the logical flow. However, when I schedule the ADF, to do the work for me. It fails. The input dataset passes, but when executing the output dataset, I am presented with the following error.

Copy activity encountered a user error at Source side: ErrorCode=UserErrorFileNotFound,'Type=Microsoft.DataTransfer.Common.Shared.HybridDeliveryException,Message=Cannot find the file specified. Folder path: 'Test/', File filter: 'Testfile.text'.,Source=Microsoft.DataTransfer.ClientLibrary,''Type=System.Net.WebException,Message=The remote server returned an error: (500) Syntax error, command unrecognized.,Source=System,'.

I can physically navigate to the folder and see for myself the file, but when using the ADF, I ma having issues. The firewall is set to allow the connection. Still I am getting this issue. As there is very minimal logging, I am unable to nail down the issue. Could someone help me out here?

PS: Cross Posted at MSDN

1
azureazure-data-factoryazure-data-lake
Hey Paul, good to meet you at SQLBits. How big is the source file you are trying to copy from the FTP site?? Is it more than 30MB? I've had an idea following on from your tweets. Can you please try copying a small file? One that is around 10MB in size. Let me know. Thanks – Paul Andrew
@PaulAndrew, good to have met you too. The file is tiny, just about 603 KB. – PaulFrancis
Ok, cancel that idea then! Next, have you tried creating an empty destination file? – Paul Andrew
Sorry @PaulAndrew was away on holiday. I could create/upload files to the folder in the datalake manually. – PaulFrancis

1 Answers

0
votes

I encountered the same error and I was able to solve it by adding "enableSsl": true, "enableServerCertificateValidation": true