How to configure dynamic httpfile source in Azure Data Factory?

Question

I'm trying to pull in data into an Azure SQL DB from an external API. I will be making about different 50 API each day, each dynamically driven with parameter from the Azure SQL DB.

I started by using Azure Data Factory. But I can't get beyond setting up a dynamic http source for the API call.

First I created a lookup activity to get a list of keys/param that I will be passing to the API calls.

Then mapped the lookup output to a ForEach items. Inside the ForEach, created a Copy DataFlow activity with the http source and Azure SQL DB as a sink.

How can I inject the ForEach item value into the API call?

domain.com/api/[ForEach Item]/GetData
domain.com/api?item=[ForEach Item] 

Thank you.

Answer 1

in your activity inside foreach you can specified parameters with value @item().['name of column'] name of column represent item from your foreach activity, yourparameter = @item().columnName and after that you can pass this parameter into your api call. check this https://docs.microsoft.com/en-us/azure/data-factory/control-flow-for-each-activity

Answer 2

I had success passing the entire Relative URL path from a lookup into the ForEach task, then using @{item().columnName} in the DataSets Relative URL. In your case the value of @{item().columnName} will be a list (for each list) of: domain.com/api/[value]/GetData all combined ahead of time.

In theory you should be able to dynamically create the URL dynamically using @concat, but building the list ahead of time does work.