Launch Azure Container Service on Upload to Blob Storage

1
votes

I have a use case where I'd like to launch a job on an Azure Container Service cluster to process a file being uploaded to Blob storage. I know that I can trigger an Azure Functions instance from the upload, but I haven't been able to find examples in the documentation of starting a job within Functions.

This diagram illustrates the AWS equivalent of what I want:

enter image description here

Thanks!

1
azureaws-lambdaazure-functionsazure-blob-storageazure-container-service
Confused as to whether you're trying to solve this on Azure or AWS. – Andrew Shepherd
why do you need a job in functions? – 4c74356b41
You can just run the 'job' inside your Azure Function, or if its a longrunning task (longer as 5 / 10 minutes) you can just make sure a Logic App or WebJob is triggerd by placing an item on some kind of queue or via a HTTP request. If you want to run multiple functions inside 1 Azure Function, check out the Durable Functions extension for Azure Functions. They are awesome! – Jan_V
@AndrewShepherd Sorry if there's confusion, I'm running this on Azure and included an AWS diagram because I know this is supported in AWS and am looking for an equivalent setup – Thuy Guevarra
@Jan_V for this use case it makes more sense to process these files in Container Service, I'm looking very specifically for a way to start the job in response to the file upload. Doesn't necessarily have to be from Functions, although that was the first place I looked because on AWS you can do this from Lambda in response to S3. – Thuy Guevarra

1 Answers

2
votes

The Azure Event Grid feature is what you need. It is still in preview, but you can subscribe to the Blob Created event. You can set the subscriber endpoint to an Azure Function that puts a message in a queue to trigger your job, or you can expose a service on your cluster that will accept the request and do whatever you need done. Microsoft provides a guide at https://docs.microsoft.com/en-us/azure/storage/blobs/storage-blob-event-quickstart?toc=%2fazure%2fevent-grid%2ftoc.json#create-a-message-endpoint