NiFi Route on JSON Attribute

Question

Trying to use NiFi to route on an attribute. I am attempting to take a json file, where two of the json records contain the following attributes (there are other json documents with different attributes in this file):

{
  "ts" : "2020-010-07T12:00:00.448392Z",
  "uid" : "CHh3F30dkfueLhnxSk",
  "id.orig_h" : "10.10.10.10",
  "id.orig_p" : 19726,
  "id.resp_h" : "172.10.10.20",
  "id.resp_p" : 443,
  "proto" : "tcp",
  "conn_state" : "SH",
  "local_orig" : false,
  "local_resp" : false,
  "missed_bytes" : 0,
  "history" : "F",
  "orig_pkts" : 1,
  "orig_ip_bytes" : 52,
  "resp_pkts" : 0,
  "resp_ip_bytes" : 0}
 {
  "ts" : "2020-10-10T12:00:00.461880Z",
  "uid" : "CdoiLnRscrxO1BSYb",
  "id.orig_h" : "10.10.17.777",
  "id.orig_p" : 40433,
  "id.resp_h" : "172.10.10.77",
  "id.resp_p" : 443,
  "version" : "TLSv12",
  "cipher" : "TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384",
  "curve" : "secp777r1",
  "server_name" : "connect-stackoverflow.questions.com",
  "resumed" : false,
  "established" : true,
  "cert_chain_fuids" : [ "FR84qjkl2342SZLwV7", "Ffweqiof48b8j" ],
  "client_cert_chain_fuids" : [ ],
  "subject" : "CN=connect-ulastm.bentley.com",
  "issuer" : "CN=Let's Encrypt Authority X3,O=Let's Encrypt,C=US",
  "validation_status" : "ok"}

I want to specifically route on the $.conn_state attribute but it is not working. I have tried to match the expression with the evaluateJSONpath processor and passed it to routeOnAttribute. Here are my settings:

evaluateJSONpath processor:

The above processor does not match the json and forward the document

and the evaluateJSONpath is followed by the routeOnAttribute processor:

I have attempted to routeOnAttribute directly from my jsonRecord, but it does not appear to pull out or identify the attribute for routing...

How would I do this?

looks like json is not valid....are you getting the value for conn.state in conn.log as attribute (from json above I think it will be null )..when process json through EvaluateJsonPath? — Taha Naqvi
I do not think I am getting the attribute... I updated to split the json out of the file, but the attribute is still not being found. — Advent United

steven-matison steven-matison · Accepted Answer · 2020-08-23T15:14:25

@Advent United

The JSON sample is more than one object, your evaluateJson is going to expect a single record in matching $.conn_state. For multiple record work against json or any data stream you should use QueryRecord. Once configured w/ record reader and record writer within in that processor you just click + to create the route and the value is the select statement where conn_state is not null. Then you can drag that route to the next processor.

NiFi Route on JSON Attribute

1 Answers