Azure data factory: From Salesforce to REST API

0
votes

I have created data factory source as Salesforce for which i am querying lead and then want to pass each lead email id as an argument (POST request) to a REST endpoint.

I am not sure in the pipeline what sink should i put and if it is HTTP File dataset then how to pass the email id from Source to sink in the argument?

1
azure-data-factory

1 Answers

1
votes

The only way I know how to surface data in ADF itself is through a lookup activity. You can then iterate through the results of the lookup using a forEach activity. Reference the result of the lookup using in the ForEach items parameter:

@activity('nameoflookupactivity').output.value

If you need to add multiple IDs at once I think to only way would be to concatenate your IDs in a sink like a SQL database. You would first have to copy your IDs to a table in SQL. Then in Azure SQL DB/SQL server 2017 you could use the following query:

SELECT STRING_AGG([salesforce_Id], ',') AS Ids FROM salesforceLeads;

The ADF tutorial for incrementally loading multiple tables discusses the ForEach activity extensively, you can find it here: ADF Incrementally Loading multiple Tables Tutorial

For more information about the STRING_AGG function check out:STRING_AGG Documentation

Thanks,

Jan