Apache NiFi evaluatejsonpath processor: JSONPath expression to concatenate 2 attributes

2
votes

I am using EvaluateJsonPath processor in NiFi to specify a composite primary key for writing my JSON data into elasticsearch. I have to create an attribute called 'key' by concatenating two attributes, lets say 'attr1' and 'attr2'. In EvaluateJsonPath configuration, I added a property 'key' with value '${attr1}${attr2}' (I'm using this key to avoid redundancy in elasticsearch). But I'm getting an error like : 'key' is invalid because specified expression was not valid: ${attr1}${attr2}.

What is the proper syntax for concatenating 2 attributes in a json record using EvaluateJsonPath processor?

2
apache-nifijson-path-expression

2 Answers

5
votes

I assume you mixed the term attributes with different contexts. I did that when I picked up NiFi for the first time and started working with JSON. To get things clarified, there are two types of attributes in this particular case:

  1. FlowFile attributes - These are the attributes that are sort of like metadata of the FlowFile, like filename, filesize, mime.type, etc. When you add an attribute using a processor like UpdateAttribute they get added to the metadata of the FlowFile i.e. FlowFile attributes not to the FlowFile content itself.
  2. JSON attributes - These are the fields that are in the FlowFile content. Ex: Name is an attribute here -> { "Name" : "Smith, John"}

Having said that, using expressions like ${projectName}, ${filename} are evaluated against the FlowFile attributes. This syntax is called NiFi Expression Language. they won't be evaluated against the content of the FlowFile. EvaluateJsonPath takes a JSON path expression like $.Name

So for example, lets assume you have the following JSON document in the FlowFile content:

{
    "attr1": "some value for attr1",
    "attr2": "some value for attr2"
}

In this case, you would have to configure the EvaluateJsonPath processor with new properties which look like the following:

attribute1 : $.attr1
attribute2 : $.attr2

Once done, flowfiles that pass through the EvaluateJsonPath will have the above two attributes added to the flowfile attributes which you can read using NiFi Expression like ${attribute1}. Then connect EvaluateJsonPath processor to UpdateRecord processor where you can add the combined composite key with the help of NiFi Expression Language to the FlowFile JSON content.

Useful Links

1
votes

EvaluateJsonPath expects a JSONPath statement, it looks like in this case you want to take two attributes and create one from them using NiFi Expression Language. If that's the case, then UpdateAttribute should be what you want. If you're looking to insert the composite key into the content, then UpdateRecord should be what you want.