How to transfer blobs from an Azure container to a FTP server/location?

0
votes

The project I have been assigned has a requirement of transferring data from an Azure Blob Container to a FTP location (Veeva Vault to be precise). I tried the following approaches:

  1. Tried to create a pipeline in Azure Data factory. But could not set the sink location as the FTP server.
  2. Created a Linux (Ubuntu 18.04) VM on Azure. On the VM I mounted the Azure container and then uploaded the data to the FTP server. This method was successful.

The constraint here is that every thing should be on Azure itself and should be automated. The second approach I have followed seems to be a wastage of resources and has many factors which can lead to failure of the migration. (I am new to Azure platform)

Also the FTP server is secured.

Is there any better approach to transfer the data from Azure container to the FTP server?

1
azureftpdata-transfer
There are several ways to achieve what you are describing, and finding the right way needs a bit more information: Should this happen automatically whenever a blob is uploaded, or on a schedule? Are these blobs large or small? Many or few? I would probably look at Azure Functions (with triggers) for this kind of data transfer - but it all depends on exactly how this is supposed to work. - Christian Rygg
Thanks for answering. The transfer should not be automatic, i.e. the uploading of the blobs should be manual (no schedule or anything). The blobs are basically documents (CSV's, PDF's etc.) but the quantity can range from 80,000 blobs to 2,00,000 blobs. I had tried Azure funcitons but i couldn't implement it. - Shashank Mishra

1 Answers

1
votes

You can use Azure Logic App and the Blob Storage and SFTP connectors:

https://docs.microsoft.com/en-us/azure/connectors/connectors-create-api-azureblobstorage

https://docs.microsoft.com/en-us/azure/connectors/connectors-create-api-sftp

PS: You can either use Blob Trigger as the starting point for your workflow and also schedule to batch the transfer according to some interval you define