ReplaceText of NiFi does not parse regex being passed through an Attribute (even though it supports Expression Language)

0
votes

I'm having the same issue.

The following works great in the NiFi ReplaceText Processor

Flowfile Content:

US0706003247984600Z1Z000123371K US0706003247984600Z1Z000125491K US0706003247984600Z1Z000125596K

Search Value: (.{2})(?:.{4})(.{6})(.{2})(.{4})(.{6})(.{6})(.{1})

Replacement Value: {col_foo1:$1,col_foo3:$2,col_foo4:$3,col_foo5:$4,col_foo6:$5,col_foo7:$6,col_foo8:$7},

Output: {col_foo1:US,col_foo3:003247,col_foo4:98,col_foo5:4600,col_foo6:Z1Z000,col_foo7:123371,col_foo8:K}, {col_foo1:US,col_foo3:003247,col_foo4:98,col_foo5:4600,col_foo6:Z1Z000,col_foo7:125491,col_foo8:K}, {col_foo1:US,col_foo3:003247,col_foo4:98,col_foo5:4600,col_foo6:Z1Z000,col_foo7:125596,col_foo8:K},

however

I need to store the Search Value in an Attribute (e.g. search.value) and the Replacement Value in an Attribute (e.g. replace.value), which will be passed in a via a configuration file.

Flowfile Content:

US0706003247984600Z1Z000123371K US0706003247984600Z1Z000125491K US0706003247984600Z1Z000125596K

Search Value: ${search.value}

search.value Attribute: (.{2})(?:.{4})(.{6})(.{2})(.{4})(.{6})(.{6})(.{1})

Replacement Value: ${replacement.value}

replacement.value Attribute: {col_foo1:$1,col_foo3:$2,col_foo4:$3,col_foo5:$4,col_foo6:$5,col_foo7:$6,col_foo8:$7},

Output: US0706003247984600Z1Z000123371K US0706003247984600Z1Z000125491K US0706003247984600Z1Z000125596K

which appears to indicate that the regex content of each of the Attribute values is not being evaluated properly.

Any ideas are greatly appreciated.

1
regexapache-nifi

1 Answers

0
votes

In general, expression language in NiFi can be evaluated with and without a flow file, and is up to the processor to determine how to use it.

For example, there may be a URL property on some processor which is used to make a connection when the processor starts, this can support expression language against system properties and environment variables, but can't be evaluated against a flow file because there is no flow file when the processor is started.

From a quick glance at the code for ReplaceText, there may be an inconsistency which is causing the problem you described.

I created this JIRA: https://issues.apache.org/jira/browse/NIFI-4417

This may also be related: https://issues.apache.org/jira/browse/NIFI-4272