Remove all flow file attributes except the ones explicitly defined

Question

I want to remove all attributes from a given flow file except the ones I explicitly define to keep.

Given the following sample flow file attributes:

name: aaa
Place: bbb
Host: ccc
JsonAttribute: {
    "A": "a",
    "B": "b"
}
data: ddd

And I want to keep Host and JsonAttribute only.

Thus, the resulting flow file attributes should be:

Host: ccc
JsonAttribute: {
    "A": "a",
    "B": "b"
}

How can I achieve this using the standard processors NiFi provides?

renaming the attributes

Is it possible to rename the attributes using the same procedure? E.g. I would like to keep the attributes like above but rename JsonAttribute into customName

Endzeit Endzeit · Accepted Answer · 2019-11-18T22:54:46

The UpdateAttribute processor does not only allow to add / set flow file attributes but also enables you to delete existing attributes from a flow file.

To do so you have to pass a regular expression matching all attributes you want to remove to the property Delete Attributes Expression.

In your case you can use a negative look-around to match every attribute other than the ones you want to keep.

^((?!Host)(?!JsonAttribute).)*$

So no additional processor is needed.

keeping Host attribute, moving JsonAttribute to a different attribute name

You should be able to also achieve the second behaviour using only a single UpdateAttribute processor.

Simply add a property, e.g. customName, to the processor and reference the old attribute using the NiFi Expression Language:

${JsonAttribute}

In that case you may also simplify your deletion regex to also delete the JsonAttribute flow file attribute:

^((?!Host).)*$

Remove all flow file attributes except the ones explicitly defined

renaming the attributes

3 Answers

keeping Host attribute, moving JsonAttribute to a different attribute name