Example of using operator for filter in dataweave 2

0
votes

I am using dataweave 2 for transformation. I need to use filter operator to filter payload depending upon the value in the request body. The reason for using 'using' is because the value is optional and in that scenario I do not want to filter the payload. My transformation -

%dw 2.0
output application/json skipNullOn="everywhere"
fun toDate(v) = (v as String{format:'yyyy-MM-dd'} as Date) default null
---
    
if (payload is Array)
    flatten (
        payload **//need to put 2 filters here on  $.CloudAccountDetail.status and $.CloudAccountDetail.clouAccountId //**  map (item, order) -> (
            flatten ( [
                item.NamedSubscriptions map {
                    key: $.SubscriptionId,
                    expiryDate: toDate($.ExpiryDate)
                    //nextPaymentDate: toDate($.NextPaymentDueDate)
                } 
                , 
                item.UnnamedSubscriptions map {
                    key: $.SubscriptionId,
                    expiryDate: toDate($.ExpiryDate),
                    cloudPublicName: item.CloudAccountDetail.CloudAccountPublicName,
                    cloudAccountId: $.EskoCloudAccount
                }
            ] )
        )  
    )
else []

Input -

[
  {
    "CloudAccountDetail": {
      "CompanyCode": "B_B001",
      "CloudAccountId": "a-t-1000-5001-0687-0024",
      "CloudAccountPublicName": "dodp-testcloud",
      "Status": "SUBSCRIBED",
      "ClusterName": "ci.dev.cloudi.city"
    },
    "NamedSubscriptions": [
      {
        "SubscriptionId": "8f724cc9-bb21-43bc-8911-94d267eb7917",
        "OrderNumber": "TEST708259",
        "StartDate": "2020-01-07T12:13:19.507",
        "ExpiryDate": "2021-01-16T00:00:00",
        "CanceledDate": null,
        "ProductCode": "ECL-SA-PUSR",
        "ProductName": "Esko Cloud Power User",
        "Assignee": "[email protected]",
        "AssignedBy": "LicenseService",
        "AssignedDate": "2020-01-07T11:16:52.723",
        "EskoCloudAccount": "a-t-1000-5001-0687-0024",
        "Status": "ACTIVE"
      },
      {
        "SubscriptionId": "5c06112a-90a4-4e76-b2f4-bd19a1ce92b3",
        "OrderNumber": "TEST708259",
        "StartDate": "2020-01-07T12:13:19.507",
        "ExpiryDate": "2021-01-16T00:00:00",
        "CanceledDate": null,
        "ProductCode": "ECL-SA-PUSR",
        "ProductName": "Esko Cloud Power User",
        "Assignee": null,
        "AssignedBy": null,
        "AssignedDate": null,
        "EskoCloudAccount": "a-t-1000-5001-0687-0024",
        "Status": "ACTIVE"
      },
      {
        "SubscriptionId": "a258a7fc-cff6-4b53-b19d-4e25a8dbcee6",
        "OrderNumber": "TEST708259",
        "StartDate": "2020-01-07T12:13:19.507",
        "ExpiryDate": "2021-01-16T00:00:00",
        "CanceledDate": null,
        "ProductCode": "ECL-SA-PUSR",
        "ProductName": "Esko Cloud Power User",
        "Assignee": null,
        "AssignedBy": null,
        "AssignedDate": null,
        "EskoCloudAccount": "a-t-1000-5001-0687-0024",
        "Status": "ACTIVE"
      },
      {
        "SubscriptionId": "e30f16af-de60-4b84-bdae-aa7b6523c316",
        "OrderNumber": "TEST708259",
        "StartDate": "2020-01-07T12:13:19.507",
        "ExpiryDate": "2021-01-16T00:00:00",
        "CanceledDate": null,
        "ProductCode": "ECL-SA-PUSR",
        "ProductName": "Esko Cloud Power User",
        "Assignee": null,
        "AssignedBy": null,
        "AssignedDate": null,
        "EskoCloudAccount": "a-t-1000-5001-0687-0024",
        "Status": "ACTIVE"
      },
      {
        "SubscriptionId": "5ee9eeb6-9d65-42b8-b594-68c9528ff25b",
        "OrderNumber": "TEST708259",
        "StartDate": "2020-01-07T12:13:20.507",
        "ExpiryDate": "2021-01-16T00:00:00",
        "CanceledDate": null,
        "ProductCode": "ECL-SA-PUSR",
        "ProductName": "Esko Cloud Power User",
        "Assignee": null,
        "AssignedBy": null,
        "AssignedDate": null,
        "EskoCloudAccount": "a-t-1000-5001-0687-0024",
        "Status": "ACTIVE"
      },
      {
        "SubscriptionId": "818b38f1-0a10-45dc-ae16-f7d804b31cbc",
        "OrderNumber": "TEST708259",
        "StartDate": "2020-01-07T12:13:20.507",
        "ExpiryDate": "2021-01-16T00:00:00",
        "CanceledDate": null,
        "ProductCode": "ECL-SA-PUSR",
        "ProductName": "Esko Cloud Power User",
        "Assignee": null,
        "AssignedBy": null,
        "AssignedDate": null,
        "EskoCloudAccount": "a-t-1000-5001-0687-0024",
        "Status": "ACTIVE"
      },
      {
        "SubscriptionId": "cf5fc421-8ee0-4f1f-a1ca-e64daef2e0c6",
        "OrderNumber": "TEST708259",
        "StartDate": "2020-01-07T12:13:20.507",
        "ExpiryDate": "2021-01-16T00:00:00",
        "CanceledDate": null,
        "ProductCode": "ECL-SA-PUSR",
        "ProductName": "Esko Cloud Power User",
        "Assignee": null,
        "AssignedBy": null,
        "AssignedDate": null,
        "EskoCloudAccount": "a-t-1000-5001-0687-0024",
        "Status": "ACTIVE"
      }
    ],
    "UnnamedSubscriptions": [
      {
        "SubscriptionId": "c56c5d21-83dd-4df5-851c-52fd5686240b",
        "OrderNumber": "TEST708259",
        "StartDate": "2020-01-07T13:23:02.593",
        "ExpiryDate": "2021-01-16T00:00:00",
        "CanceledDate": null,
        "ProductCode": "ECL-SA-SITE",
        "ProductName": "Share & Approve Site",
        "EskoCloudAccount": "a-t-1000-5001-0687-0024",
        "Status": "ACTIVE"
      },
      {
        "SubscriptionId": "1d5bf54c-e812-4c17-830d-d3a1104d66a5",
        "OrderNumber": "TEST708259",
        "StartDate": "2020-01-07T13:23:02.593",
        "ExpiryDate": "2021-01-16T00:00:00",
        "CanceledDate": null,
        "ProductCode": "ECL-SA-SITE",
        "ProductName": "Share & Approve Site",
        "EskoCloudAccount": "a-t-1000-5001-0687-0024",
        "Status": "ACTIVE"
      },
      {
        "SubscriptionId": "fbe0bbef-3cc2-451a-9866-0bb31a829850",
        "OrderNumber": "TEST708259",
        "StartDate": "2020-01-07T13:23:02.593",
        "ExpiryDate": "2021-01-16T00:00:00",
        "CanceledDate": null,
        "ProductCode": "ECL-SA-SITE",
        "ProductName": "Share & Approve Site",
        "EskoCloudAccount": "a-t-1000-5001-0687-0024",
        "Status": "ACTIVE"
      },
      {
        "SubscriptionId": "368fa144-5a94-4ead-ba69-502ca6419de4",
        "OrderNumber": "TEST708259",
        "StartDate": "2020-01-07T13:23:02.593",
        "ExpiryDate": "2021-01-16T00:00:00",
        "CanceledDate": null,
        "ProductCode": "ECL-SA-SITE",
        "ProductName": "Share & Approve Site",
        "EskoCloudAccount": "a-t-1000-5001-0687-0024",
        "Status": "ACTIVE"
      },
      {
        "SubscriptionId": "3404fb6e-d234-4dac-a5a1-96c7534ccf21",
        "OrderNumber": "TEST708259",
        "StartDate": "2020-01-07T13:23:03.593",
        "ExpiryDate": "2021-01-16T00:00:00",
        "CanceledDate": null,
        "ProductCode": "ECL-SA-SITE",
        "ProductName": "Share & Approve Site",
        "EskoCloudAccount": "a-t-1000-5001-0687-0024",
        "Status": "ACTIVE"
      },
      {
        "SubscriptionId": "71d88ad6-3108-499e-8be9-3638c7298770",
        "OrderNumber": "TEST708259",
        "StartDate": "2020-01-07T13:23:03.593",
        "ExpiryDate": "2021-01-16T00:00:00",
        "CanceledDate": null,
        "ProductCode": "ECL-SA-SITE",
        "ProductName": "Share & Approve Site",
        "EskoCloudAccount": "a-t-1000-5001-0687-0024",
        "Status": "ACTIVE"
      },
      {
        "SubscriptionId": "992b551e-1499-4971-bc9b-34075820faa7",
        "OrderNumber": "TEST708259",
        "StartDate": "2020-01-07T13:23:03.593",
        "ExpiryDate": "2021-01-16T00:00:00",
        "CanceledDate": null,
        "ProductCode": "ECL-SA-SITE",
        "ProductName": "Share & Approve Site",
        "EskoCloudAccount": "a-t-1000-5001-0687-0024",
        "Status": "ACTIVE"
      }
    ],
    "StorageSubscriptions": [
      {
        "EntitlementId": "5ceae026-5c3e-40df-a9f8-2de4f4570566",
        "OrderNumber": "TEST708259",
        "StorageInGb": 50,
        "ExpiryDate": "2021-01-07T00:00:00",
        "StartDate": "2020-01-07T12:37:12.697",
        "CanceledDate": null,
        "ProductCode": "ECL-ECL-ST-ST50",
        "ProductName": "Storage Tier 1 (50GB)",
        "EskoCloudAccount": "a-t-1000-5001-0687-0024",
        "Status": "ACTIVE"
      }
    ],
    "TransactionSubscriptions": [
      {
        "EntitlementId": "6eaf4b93-8fef-4423-88fd-7e83d790f6f7",
        "OrderNumber": "TEST708259",
        "NumberOfTransactions": 12000,
        "ExpiryDate": "2021-01-07T00:00:00",
        "StartDate": "2020-01-07T12:17:56.257",
        "CanceledDate": null,
        "ProductCode": "ECL-ECL-BTR-BTR1000",
        "ProductName": "Basic Transactions Tier 2 (1000/m)",
        "EskoCloudAccount": "a-t-1000-5001-0687-0024",
        "Status": "ACTIVE"
      }
    ]
  },
  {
    "CloudAccountDetail": {
      "CompanyCode": "B_B001",
      "CloudAccountId": "a-t-1001-0001-0039-0024",
      "CloudAccountPublicName": "Test-Tom-Desmet",
      "Status": "TRIAL",
      "ClusterName": "ci.dev.cloudi.city"
    },
    "NamedSubscriptions": [
      {
        "SubscriptionId": "3dc289af-d469-4dba-875e-354c75d2a31a",
        "OrderNumber": "TEST709894",
        "StartDate": "2020-06-30T14:21:34.773",
        "ExpiryDate": "2021-07-09T00:00:00",
        "CanceledDate": null,
        "ProductCode": "ECL-SA-PUSR",
        "ProductName": "Esko Cloud Power User",
        "Assignee": "[email protected]",
        "AssignedBy": "LicenseService",
        "AssignedDate": "2020-06-30T13:31:01.777",
        "EskoCloudAccount": "a-t-1001-0001-0039-0024",
        "Status": "ACTIVE"
      },
      {
        "SubscriptionId": "3e6818fb-1c4f-4046-9783-5b76d3050eec",
        "OrderNumber": "TEST709894",
        "StartDate": "2020-06-30T14:21:35.773",
        "ExpiryDate": "2021-07-09T00:00:00",
        "CanceledDate": null,
        "ProductCode": "ECL-SA-PUSR",
        "ProductName": "Esko Cloud Power User",
        "Assignee": null,
        "AssignedBy": null,
        "AssignedDate": null,
        "EskoCloudAccount": "a-t-1001-0001-0039-0024",
        "Status": "ACTIVE"
      }
    ],
    "UnnamedSubscriptions": [
      {
        "SubscriptionId": "baf3d5c7-b53e-460e-93a5-311b95924fca",
        "OrderNumber": "TEST709894",
        "StartDate": "2020-06-30T15:18:15.593",
        "ExpiryDate": "2021-07-09T00:00:00",
        "CanceledDate": null,
        "ProductCode": "ECL-SA-SITE",
        "ProductName": "Share & Approve Site",
        "EskoCloudAccount": "a-t-1001-0001-0039-0024",
        "Status": "ACTIVE"
      },
      {
        "SubscriptionId": "69071fd1-59b5-44ef-a594-95141d68ef4c",
        "OrderNumber": "TEST709894",
        "StartDate": "2020-06-30T15:18:16.593",
        "ExpiryDate": "2021-07-09T00:00:00",
        "CanceledDate": null,
        "ProductCode": "ECL-SA-SITE",
        "ProductName": "Share & Approve Site",
        "EskoCloudAccount": "a-t-1001-0001-0039-0024",
        "Status": "ACTIVE"
      }
    ],
    "StorageSubscriptions": [
      {
        "EntitlementId": "f6f444e3-4dea-43d9-afc0-19d24200cf46",
        "OrderNumber": "TEST709894",
        "StorageInGb": 50,
        "ExpiryDate": "2021-06-30T00:00:00",
        "StartDate": "2020-06-30T14:57:54.05",
        "CanceledDate": null,
        "ProductCode": "ECL-ECL-ST-ST50",
        "ProductName": "Storage Tier 1 (50GB)",
        "EskoCloudAccount": "a-t-1001-0001-0039-0024",
        "Status": "ACTIVE"
      }
    ],
    "TransactionSubscriptions": [
      {
        "EntitlementId": "22d25390-4183-4c89-8461-38ebf048812f",
        "OrderNumber": "TEST709894",
        "NumberOfTransactions": 12000,
        "ExpiryDate": "2021-06-30T00:00:00",
        "StartDate": "2020-06-30T14:24:10.017",
        "CanceledDate": null,
        "ProductCode": "ECL-ECL-BTR-BTR1000",
        "ProductName": "Basic Transactions Tier 2 (1000/m)",
        "EskoCloudAccount": "a-t-1001-0001-0039-0024",
        "Status": "ACTIVE"
      }
    ]
  }
]

Output I need after transformation. Just take note I have not considered filtering in the ouput. -

[
  {
    "key": "8f724cc9-bb21-43bc-8911-94d267eb7917",
    "expiryDate": "2021-01-16"
  },
  {
    "key": "5c06112a-90a4-4e76-b2f4-bd19a1ce92b3",
    "expiryDate": "2021-01-16"
  },
  {
    "key": "a258a7fc-cff6-4b53-b19d-4e25a8dbcee6",
    "expiryDate": "2021-01-16"
  },
  {
    "key": "e30f16af-de60-4b84-bdae-aa7b6523c316",
    "expiryDate": "2021-01-16"
  },
  {
    "key": "5ee9eeb6-9d65-42b8-b594-68c9528ff25b",
    "expiryDate": "2021-01-16"
  },
  {
    "key": "818b38f1-0a10-45dc-ae16-f7d804b31cbc",
    "expiryDate": "2021-01-16"
  },
  {
    "key": "cf5fc421-8ee0-4f1f-a1ca-e64daef2e0c6",
    "expiryDate": "2021-01-16"
  },
  {
    "key": "c56c5d21-83dd-4df5-851c-52fd5686240b",
    "expiryDate": "2021-01-16",
    "cloudPublicName": "dodp-testcloud",
    "cloudAccountId": "a-t-1000-5001-0687-0024"
  },
  {
    "key": "1d5bf54c-e812-4c17-830d-d3a1104d66a5",
    "expiryDate": "2021-01-16",
    "cloudPublicName": "dodp-testcloud",
    "cloudAccountId": "a-t-1000-5001-0687-0024"
  },
  {
    "key": "fbe0bbef-3cc2-451a-9866-0bb31a829850",
    "expiryDate": "2021-01-16",
    "cloudPublicName": "dodp-testcloud",
    "cloudAccountId": "a-t-1000-5001-0687-0024"
  },
  {
    "key": "368fa144-5a94-4ead-ba69-502ca6419de4",
    "expiryDate": "2021-01-16",
    "cloudPublicName": "dodp-testcloud",
    "cloudAccountId": "a-t-1000-5001-0687-0024"
  },
  {
    "key": "3404fb6e-d234-4dac-a5a1-96c7534ccf21",
    "expiryDate": "2021-01-16",
    "cloudPublicName": "dodp-testcloud",
    "cloudAccountId": "a-t-1000-5001-0687-0024"
  },
  {
    "key": "71d88ad6-3108-499e-8be9-3638c7298770",
    "expiryDate": "2021-01-16",
    "cloudPublicName": "dodp-testcloud",
    "cloudAccountId": "a-t-1000-5001-0687-0024"
  },
  {
    "key": "992b551e-1499-4971-bc9b-34075820faa7",
    "expiryDate": "2021-01-16",
    "cloudPublicName": "dodp-testcloud",
    "cloudAccountId": "a-t-1000-5001-0687-0024"
  },
  {
    "key": "3dc289af-d469-4dba-875e-354c75d2a31a",
    "expiryDate": "2021-07-09"
  },
  {
    "key": "3e6818fb-1c4f-4046-9783-5b76d3050eec",
    "expiryDate": "2021-07-09"
  },
  {
    "key": "baf3d5c7-b53e-460e-93a5-311b95924fca",
    "expiryDate": "2021-07-09",
    "cloudPublicName": "Test-Tom-Desmet",
    "cloudAccountId": "a-t-1001-0001-0039-0024"
  },
  {
    "key": "69071fd1-59b5-44ef-a594-95141d68ef4c",
    "expiryDate": "2021-07-09",
    "cloudPublicName": "Test-Tom-Desmet",
    "cloudAccountId": "a-t-1001-0001-0039-0024"
  }
]
2
muledataweave

2 Answers

1
votes

You should first apply the filter on the whole array, and then do the rest of the transformation:

%dw 2.0
output application/json skipNullOn="everywhere"
fun toDate(v) = (v as String{format:'yyyy-MM-dd'} as Date) default null
var status = 'SUBSCRIBED'
var accountId = 'a-t-1000-5001-0687-0024'
---
if (payload is Array)
  flatten(payload filter ($.CloudAccountDetail.CloudAccountId == accountId and $.CloudAccountDetail.Status == status) map(item, value) -> 
flatten ( [
                item.NamedSubscriptions map {
                    key: $.SubscriptionId,
                    expiryDate: toDate($.ExpiryDate)
                    //nextPaymentDate: toDate($.NextPaymentDueDate)
                } 
                , 
                item.UnnamedSubscriptions map {
                    key: $.SubscriptionId,
                    expiryDate: toDate($.ExpiryDate),
                    cloudPublicName: item.CloudAccountDetail.CloudAccountPublicName,
                    cloudAccountId: $.EskoCloudAccount
                }
            ] ))
else []

The above dataweave expression applied to the provided input results in:

[
  {
    "key": "8f724cc9-bb21-43bc-8911-94d267eb7917",
    "expiryDate": "2021-01-16"
  },
  {
    "key": "5c06112a-90a4-4e76-b2f4-bd19a1ce92b3",
    "expiryDate": "2021-01-16"
  },
  {
    "key": "a258a7fc-cff6-4b53-b19d-4e25a8dbcee6",
    "expiryDate": "2021-01-16"
  },
  {
    "key": "e30f16af-de60-4b84-bdae-aa7b6523c316",
    "expiryDate": "2021-01-16"
  },
  {
    "key": "5ee9eeb6-9d65-42b8-b594-68c9528ff25b",
    "expiryDate": "2021-01-16"
  },
  {
    "key": "818b38f1-0a10-45dc-ae16-f7d804b31cbc",
    "expiryDate": "2021-01-16"
  },
  {
    "key": "cf5fc421-8ee0-4f1f-a1ca-e64daef2e0c6",
    "expiryDate": "2021-01-16"
  },
  {
    "key": "c56c5d21-83dd-4df5-851c-52fd5686240b",
    "expiryDate": "2021-01-16",
    "cloudPublicName": "dodp-testcloud",
    "cloudAccountId": "a-t-1000-5001-0687-0024"
  },
  {
    "key": "1d5bf54c-e812-4c17-830d-d3a1104d66a5",
    "expiryDate": "2021-01-16",
    "cloudPublicName": "dodp-testcloud",
    "cloudAccountId": "a-t-1000-5001-0687-0024"
  },
  {
    "key": "fbe0bbef-3cc2-451a-9866-0bb31a829850",
    "expiryDate": "2021-01-16",
    "cloudPublicName": "dodp-testcloud",
    "cloudAccountId": "a-t-1000-5001-0687-0024"
  },
  {
    "key": "368fa144-5a94-4ead-ba69-502ca6419de4",
    "expiryDate": "2021-01-16",
    "cloudPublicName": "dodp-testcloud",
    "cloudAccountId": "a-t-1000-5001-0687-0024"
  },
  {
    "key": "3404fb6e-d234-4dac-a5a1-96c7534ccf21",
    "expiryDate": "2021-01-16",
    "cloudPublicName": "dodp-testcloud",
    "cloudAccountId": "a-t-1000-5001-0687-0024"
  },
  {
    "key": "71d88ad6-3108-499e-8be9-3638c7298770",
    "expiryDate": "2021-01-16",
    "cloudPublicName": "dodp-testcloud",
    "cloudAccountId": "a-t-1000-5001-0687-0024"
  },
  {
    "key": "992b551e-1499-4971-bc9b-34075820faa7",
    "expiryDate": "2021-01-16",
    "cloudPublicName": "dodp-testcloud",
    "cloudAccountId": "a-t-1000-5001-0687-0024"
  }
]
0
votes

Much cleaner example. Instead of using flatten, try using a function to populate the values.

Example:

%dw 2.0
var accountStatus=["SUBSCRIBED"]
var accountId=["a-t-1000-5001-0687-0024"]
fun getData(items, account, isUnnamedSubscriptions) = 
    (items default []) map {
        "key": $.SubscriptionId,
        "expiryDate": $.ExpiryDate as Date,
        ("cloudPublicName": account.CloudAccountPublicName) if (isUnnamedSubscriptions),
        ("cloudAccountId": account.CloudAccountId) if (isUnnamedSubscriptions)
    }

output application/json
---

(payload[?((accountId contains $.CloudAccountDetail.CloudAccountId) 
    and (accountStatus contains $.CloudAccountDetail.Status) )] map (
        getData($.NamedSubscriptions, $.CloudAccountDetail, false)
        ++
       getData($.UnnamedSubscriptions, $.CloudAccountDetail, true)
) flatMap () -> $) default []

accountId variable is an Array and you can replace that as a string as well and similar to status in line 2 and line 3.

If you have multiple accountId to filter, just add another string in the accountId, same as the status.

If you want to exclude filter if the CloudAccountDetail.Status is Null and CloudAccountDetail.CloudAccountId is Null

You can create a function to return boolean. the funciton should be like this

fun isValidFilter(id, stat) = 
    if ( id !=null and stat != null) 
       ((accountId contains id) and (accountStatus contains stat))
    else (true)

and the pass the value to the function isValidFilter

Example:

payload[?( (isValidFilter($.CloudAccountDetail.CloudAccountId, 
  $.CloudAccountDetail.Status)) )] map ...