Azure LogicApp for files migration

1
votes

I am trying to figure out if Azure LogicApp can be used for files/documents migration from Azure Blob Storage to a custom service, where we have REST API. Here is the shortlist of requirements I have right now:

  1. Files/documents must be uploaded into Azure Storage weekly or daily, which means that we need to migrate only new items. The amount of files/documents per week is about hundreds of thousands
  2. The custom service REST API is secured and any interaction with endpoints should have JWT passed in the headers

I did the following exercise according to tutorials:

Scheduled migration from Blobg Storage to Custom REST endpoint

Everything seems fine, but the following 2 requirements make me worry:

  • Getting only new files and not migrate those that already moved
  • Getting JWT to pass security checks in REST

For the first point, I think that I can introduce a DB instance (for example Azure Table Storage) to track files that have been already moved, and for the second one I have an idea to use Azure Function instead of HTTP Action. But everything looks quite complicated and I believe that there might be better and easier options.

Could you please advise what else I can use for my case?

1
azureazure-functionsazure-logic-apps
For the first point, why not use When a blob is added or modified trigger as your logic app's trigger. Then it will just do operation on the new blob item. - Hury Shen
And for the second point, I'm not clear about your question. Are you worried about how to get the jwt token or about how to use jwt token to do HTTP action request in you logic app ? - Hury Shen
@HuryShen yeah, I know how to get the token, I am more concerned if I can do this in build-in HTTP-action - Anuar Nurmakanov
You can use "HTTP" action to request the endpoint to get response data in json format(include the token) and then parse the json and get the token. Then put the token in next HTTP action's header. Is this your concern about ? - Hury Shen
Just provide my steps below for your reference. - Hury Shen

1 Answers

1
votes

For the first point, you can use "When a blob is added or modified" trigger as the logic app's trigger. Then it will just do operation on the new blob item.

For the second point, just provide some steps for your reference:

1. Below is a screenshot that I request for the token in logic app in the past. enter image description here

2. Then use "Parse JSON" action to parse the response body from the "HTTP" action above. enter image description here

3. After that, your can request your rest api (with the access token from "Parse JSON" above) enter image description here