NiFi: ReplaceText alternatives to modify JSON

0
votes

My NiFi application receives two kinda different types of JSON's. First of them looks like:

[
  {
    "campaign": {
      "resourceName": "customers/8952771329/campaigns/11381694617",
      "status": "ENABLED",
      "name": "Saint_Spring_Active Minerals_oct-nov_2020_trueview_skip_5766500views",
      "id": "11381694617"
    },
    "metrics": {
      "interactionEventTypes": [
        "VIDEO_VIEW"
      ],
      "clicks": "6",
      "videoQuartileP100Rate": 0.44493171079034244,
      "videoQuartileP25Rate": 0.9747718298919024,
      "videoQuartileP50Rate": 0.7339309987701469,
      "videoQuartileP75Rate": 0.5337562301767105,
      "videoViewRate": 0.4471109114825628,
      "videoViews": "27872",
      "viewThroughConversions": "0",
      "contentBudgetLostImpressionShare": 0.0000013066088274492382,
      "contentImpressionShare": 0.0999,
      "contentRankLostImpressionShare": 0.9001,
      "conversionsValue": 0,
      "conversions": 0,
      "costMicros": "9338700950",
      "ctr": 0.00009624947864865732,
      "currentModelAttributedConversions": 0,
      "currentModelAttributedConversionsValue": 0,
      "engagementRate": 0,
      "engagements": "0",
    },
    "segments": {
      "device": "CONNECTED_TV",
      "date": "2020-12-20"
    }
  }
]

And second:

[
  {
    "adGroup": {
      "resourceName": "customers/5404177717/adGroups/110501283582",
      "campaign": "customers/5404177717/campaigns/11628802542"
    },
    "metrics": {
      "interactionEventTypes": [
        "CLICK"
      ],
      "clicks": "1",
      "averageCpm": 95497428.02172929,
      "gmailForwards": "0",
      "gmailSaves": "0",
      "gmailSecondaryClicks": "0",
      "impressions": "4418",
      "interactionRate": 0.00022634676324128565,
      "interactions": "1"
    },
    "adGroupAd": {
      "resourceName": "customers/5404177717/adGroupAds/110501283582~480227690139",
      "status": "ENABLED",
      "ad": {
        "resourceName": "customers/5404177717/ads/480227690139",
        "id": "480227690139",
        "name": "20 sec perek"
      },
      "adGroup": "customers/5404177717/adGroups/110501283582"
    },
    "segments": {
      "device": "DESKTOP",
      "date": "2020-11-21"
    }
  }
]

I already have 2 tables in my database to save this data. I have an attribute table.name just to not create same block where's only table name is different.

My next block is FlattenJson. After this i'm using ReplaceText with search value (replacement value is empty string): (customers\\\/${client.customer.id}\\\/campaigns\\\/|customers\\\/${client.customer.id}\\\/adGroups\\\/).

Why this? From this line: "adGroup": "customers/5404177717/adGroups/110501283582" i only need last value 110501283582 as ad_group_id. And from this line: "campaign": "customers/5404177717/campaigns/11628802542" i only need 11628802542. ${client.customer.id} can be different, so i'm using EL features.

Also i need to change json value name adGroup to ad.group.id, for this i'm also using ReplaceText.

Can i do it faster without two ReplaceText processors?

1
apache-nifi

1 Answers