Azure Data Factory - Multiply sources - Transform and output as JSON only

1
votes

Hi I am still new to the ADF scene, my question is if I have multiple sources (REST API, on-prem SQL server) and all I want to do is to fetch the data and transform it into a specific JSON format and POST it to another REST API endpoint and I do not need to store the data on the azure cloud. 

Source (REST API, on-prem SQL) -> Transform to specific JSON format -> POST JSON to another REST API endpoint

What would be the correct approach on ADF? Do I just use Azure custom activity to run a python script for transforming? Appreciate some pointers.

1
azureazure-data-factoryazure-data-factory-2
The transform is pure or you have to do some logical merge? - Jay Gong
Hi @JayGong, just some unit conversions and text parsing. - chad

1 Answers

1
votes

According to the copy activity document,you only could get data from REST API,not support for sending request to REST API.Additional,copy activity doesn't support multiple source mapping to single sink(which is mentioned in your question,you have REST api and SQL DB source dataset).

So,my rough idea is using Azure Function Activity to get data from multiple source in the azure function method and return the json format as you want.Then execute Web Activity to call your REST API,set the json in the request body.

In doing so, I don't feel very different from custom activity. Custom activity is also more flexible.You could balance above solutions.