nifi publishKafka processor can not publish on kafka 2.5

1
votes

I'm trying to publish a simple json to a kafka topic. I'm using the dockerized version of both apache nifi and apache kafka (latest of both). the kafka version is 2.5 but there is no processor to connect to this version.

When I use the other publishKafka processors I get this error:"can not update metadata in 5000 ms".

Can anyone help me about the problem?

1
apache-kafkaapache-nifikafka-producer-api
That error generally implies that the topic does not exist. Are you sure the topic exists? - Matthew Formosa
thanks matthew. yes I created the topic before running the flow. Also I checked it in the topics list and it's available. Also I published some message manually on it - hossein shemshadi
It could also be the Delivery Guarantee property in the Publish Kafka processor. Is it set to Best Effort? I do not believe it will work using a Docker instance if it is set otherwise. - Matthew Formosa
I set the transaction to false and so it doesn't need the guarantee property now. Could the problem refer to this transaction? - hossein shemshadi
Just to confirm, you are using PublishKafkaRecord_2_0, nifi.apache.org/docs/nifi-docs/components/nifi-docs/components/… - mazaneicha

1 Answers

0
votes

Kafka up to 2.6 is now supported in NiFi 1.12.X. You need to upgrade and switch to the new Kafka 2.6 processors.