Multiple JSON payload to CSV file

0
votes

i have a task to generate CSV file from multiple JSON payloads (2). Below are my sample data providing for understanding purpose

 - Payload-1

[
  {
    "id": "Run",
    "errorMessage": "Cannot Run"
  },
  {
    "id": "Walk",
    "errorMessage": "Cannot Walk"
  }
]

 - Payload-2 (**Source Input**) in flowVars

[
  {
    "Action1": "Run",
    "Action2": ""
  },
  {
    "Action1": "",
    "Action2": "Walk"
  },
  {
    "Action1": "Sleep",
    "Action2": ""
  }
]

Now, i have to generate CSV file with one extra column to Source Input with ErrorMessage on one condition basis, where the id in payload 1 matches with sourceInput field then errorMessage should assign to that requested field and generate a CSV file as a output

i had tried with the below dataweave

%dw 1.0
%output application/csv header=true
---
flowVars.InputData map (val,index)->{
        Action1: val.Action1,
        Action2: val.Action2,
         (
            payload filter ($.id == val.Action1 or $.id == val.Action2) map (val2,index) -> {

                ErrorMessage: val2.errorMessage replace /([\n,\/])/ with ""
            }
        )
}

But, here im facing an issue with, i'm able to generate the file with data as expected, but the header ErrorMessage is missing/not appearing in the file with my real data(in production). Kindly assist me.

and Expecting the below CSV output

Action1,Action2,ErrorMessage
Run,,Cannot Run
,Walk,Cannot Walk
Sleep,
2
csvdataweave

2 Answers

0
votes

Hello the best way to solve this kind of problem is using groupBy. The idea is that you groupBy one of the two parts to use the join by and then you iterate the other part and do a lookup. This way you avoid O(n^2) and transform it to O(n)

%dw 1.0
%var payloadById = payload groupBy $.id
%output application/csv
---
flowVars.InputData map ((value, index) -> 
  using(locatedError = payloadById[value.Action2][0] default payloadById[value.Action1][0]) (
    (value ++ {ErrorMessage: locatedError.errorMessage replace /([\n,\/])/ with ""}) when locatedError != null otherwise value
  )
)
filter $ != null
0
votes

Assuming "Payload-1" is payload, and "Payload-2" is flowVars.actions, I would first create a key-value lookup with the payload. Then I would use that to populate flowVars.actions:

%dw 1.0
%output application/csv header=true

// Creates lookup, e.g.:
// {"Run": "Cannot run", "Walk": "Cannot walk"}
%var errorMsgLookup = payload reduce ((obj, lookup={}) ->
  lookup ++ {(obj.id): obj.errorMessage})
---
flowVars.actions map ((action) -> action ++ errorMsgLookup[action.Action1])

Note: I'm also assuming flowVars.action's id field is unique across the array.