Facading Azure storage blob with sFTP service

7
votes

We have a requirement to create large (1G-16G) row data reports compress and encrypt them. Our customers will consume those reports over sFTP. We are replacing an existing implementation so our customer should get this change transparently.

Azure Blob service does not expose sFTP service so we will need some way to facade it with sFTP service. Something similar to FTP to Azure Blob Storage Bridge based on worker role. The worker role will expose sFTP endpoint to the outside world. We will setup a container per customer and limit the access from worker roles only so containers will be protected from direct access.

My questions are:

  1. What do you think of such approach?
  2. Will the sFTP using worker role can be dynamically scaled/shrink while keeping the same DNS name from customer point of view?
  3. Does Azure Blob service support compression or encryption?
  4. Are you familiar with sFTP worker role similar to FTP to Azure Blob Storage Bridge (preferable open source)?

Related Questions:
In Windows Azure: What are web role, worker role and VM role?
Azure Architecture Design

1
azuresftpazure-storage-blobsazure-worker-roles
How about creating VM and setting up IIS FTP on the folder on VM's VHD. VHD will persist on Blob Storage anyway, you will not have to make too many changes to your normal design. Debugging and maintaining worker roles are painful, that's why Azure websites came in picture and we are more comfortable with that. – Akash Kava
I need sFTP and IIS provides FTP only; anyway if I will need to intercepts the sftp calls myself and manipulate them I will prefer a java based solution like Apache SSHD. I hoped that using worker roles will help in automatic scale for the sftp access. I am missing a component that will route DNS query to the right worker role. Why did you found worker roles so painful? – Haim Raman
If you need Java based solution then VM is the best, you can install Java, Tomcat or any software you need. VM can have multiple hard disks and you can scale up when you need more performance. Worker roles do not have ability to install third party software. Worker role has complete different management hassle, one wrong config and you spend hours to figure out what is wrong. – Akash Kava
You got me confused here I was reading: blogs.msdn.com/b/mariok/archive/2011/01/05/… and it looks like you can deploy a war/application server as a worker role – Haim Raman
Ok I take my words back, you can install other things in worker role, but it is still complex, not as easy as it is in VM, ultimately worker roles run on some VM, configuring it is nightmare. We are all familiar with Windows OS and its easy to go through remote desktop and investigate things. Worker role does it automatically, but VM will give you everything that worker role has to offer. – Akash Kava

1 Answers

1
votes

You could possibly achieve this by simply exposing the blob storage endpoints directly over HTTPS and look to use Shared Access Signatures (http://www.windowsazure.com/en-us/documentation/articles/storage-dotnet-shared-access-signature-part-1/) to restrict access to those blobs.

Based on your feedback then - perhaps look to leverage Linux to run an sFTP server and use either the Java, Node or PHP Azure SDKs to achieve the same objective of pulling your content from blob storage on a scheduled basis without the need of a worker role (cron should see you right). To be honest you'll be doubling your storage use (not that it's that expensive) as you'll pull the blob out of storage and onto the VHD of your VM but this approach would work.

I notice you said you were encrypting the files written to blob storage - you could just use a PKI approach and share the key with customers allowing them to pull directly via HTTPS. Anyway, sFTP it appears to be :).