Nifi JOLT: flat JSON object to a list of JSON object

Question

I am trying with Nifi JOLT processor to shift the flat JSON object to a list of JSON object as in below output. The params "p_7_1_0", "px_2_7_1_0", "pv_7_1_1" may be different in names or numbers (e.g. I could have { "timestamp": 1559347670, "pw_2_1_0": 1, "p_2_2_1_0": 1 } )

Could someone help me with the jolt specs?

Input Json:

{  "timestamp": 1559347670,  "p_7_1_0": 6,  "px_2_7_1_0": 1,  "pv_7_1_1": 1 }

Expected output JSON:

{
  "values": [ 
     {
        "key": "p_7_1_0",
        "value": 6,
        "timestamp": 1559347670
      },
     {
        "key": "px_2_7_1_0",
        "value": 1,
        "timestamp": 1559347670
      },
     {
        "key": "pv_7_1_1",
        "value": 1,
        "timestamp": 1559347670
      }
    ]
}

Thanks in advance

hanzo2001 hanzo2001 · Accepted Answer · 2021-03-11T19:33:10

After reading up on this question JOLT transformation to copy single value along an array

and the answer https://stackoverflow.com/a/50438480/2733184

I can see that what you want is eerily similar. However, I would have never hit the nail on the head with the question that needed to be asked.

I encourage you to go to the aforementioned Q and the A, read all of it (including the comments inside the spec) and give them some upvotes.

[
  {
    "operation": "shift",
    "spec": {
      "timestamp": "timestamp",
      // put everything but the timestamp in a member (the pivot)
      "*": "all.&"
    }
  },
  {
    "operation": "shift",
    "spec": {
      "all": {
        "*": {
          // grab the member key and put it in its final place
          "$": "values[#2].key",
          // grab the member value and put it in its final place
          "@": "values[#2].value",
          // Walk back two steps (* -> all -> root) and grab the timestamp
          "@(2,timestamp)": "values[#2].timestamp"
          // I wish I understood the logic behind "#2" but I don't
          // and I'll have to read on it further
        }
      }
    }
  }
]

I hope someone can explain what # is for. My immediate guess is that it is like & (the member name) but it looks like it is the member position (?).

Nifi JOLT: flat JSON object to a list of JSON object

1 Answers