Data Factory v2 copy from FTP strange fails

0
votes

I am trying to binary copy a few .ZIP files sequentially from FTP to ADLS. Sometimes its failed, sometimes not, it's really strange for me. I got this type of error only working with this external FTP server. Error type:

{ "errorCode": "2200", "message": "Failure happened on 'Sink' side. ErrorCode=UserErrorFailedToReadFtpData,'Type=Microsoft.DataTransfer.Common.Shared.HybridDeliveryException,Message=Failed to read data from ftp: The remote server returned an error: (530) Not logged in.,Source=Microsoft.DataTransfer.ClientLibrary,''Type=System.Net.WebException,Message=The remote server returned an error: (530) Not logged in.,Source=System,'", "failureType": "UserError", "target": "Copy from FTP" }

A connection is good, as I said sometimes it copy files without any errors, this is a simple activity so I don't know what can cause this type of error. Sometimes it throws an error after copying 50mb on adls. Can it be related to the FTP server?

2
azure-data-factory-2
@FangLiu is this familiar to you maybe? – DraganB
You mean the same linked service sometimes work, sometimes not? What is your authentication type? Does it pass test connection in UI? – Fang Liu

2 Answers

1
votes

A possible root cause could be : Your FTP server does not support SSL but you enabled SSL in the FTP linked service. If so, You can disable the SSL in FTP linked service. Check out the FTP properties here: https://docs.microsoft.com/en-us/azure/data-factory/data-factory-ftp-connector

1
votes

From telemetry, it shows Copy can sometimes pass or fail with same payload, so it looks like a transient failure. But it is hard to determine the RCA from error message ("530 Not logged in"). What I'm suspecting is Copy hit throttling or similar transient issue from FTP server which will block the read request in the middle.

For further troubleshoot, could you check from FTP server side to see whether there's any detailed failure log. Besides, it will be a great help if I can get a test account to test the FTP server behavior and try to repro the issue. Please let me know if it is possible for you.

Regards, Gary