Azure Data Factory: complex (JSON) state management

1
votes

I need to handle some complex state with Azure Data Factory (ADF). AFAIK, the only mechanic that Azure provide you with - is SetVariable activity which accepts only single value.

What to do if I need something more complex? A JSON structure for instance

How can I perform the following steps:

  1. Read some state from SQL database
  2. Enrich the SQL-query results with ADF pipeline variables
  3. Pass the enriched result as JSON payload to ADF's Webhook activity
  4. Accept JSON as response to Webhook activity and store it into SQL database
1
azureazure-data-factory

1 Answers

1
votes

Set variable activity only supports following formats, which is exactly what you mentioned in your question.

enter image description here

I highly recommand you to refer my previous answer:Azure Data Factory - SQL to nested JSON. With that solution, you could get json string by executing sql in the copy activity SQL DB source dataset.

Then please pass the json string into your Web Activity as parameter and parse the json string in the Web Activity inside. I think this could solve your problem.