Multiple kafka topics in publishKafka processor in Apache Nifi

Question

I am trying to have three different data sources from Twitter(GetTwitter Processor) in Apache Nifi and only one PublishKafka_0_10 processor. Can I put data from three different data sources to single Publish kafka processor in separate topics?

if yes can anyone provide me any links for it?

Answer 1

You can do two things in this case:

1. You can either use three PublishKafka_0_10 processors and configure them with three different topic names individually.
2. If you want to stick with only one PublishKafka_0_10 processor then you can leverage the ExpressionLanguage support that the Topic property offers in PublishKafka_0_10 processor.

In the latter case, you need to connect the three GetTwitter processors to three UpdateAttribute processors individually. Configure UpdateAttribute processor and add a new property, say topicName and give them the relevant topic names. For ex, if you want the first GetTwitter processor to write to politics topic, configure it with that name. Similarly for the other two, give the relevant names. Then connect the three success relationships from the UpdateAttribute processors to the PublishKafka_0_10 processor. Configure the PublishKafka_0_10 and set the Topic property to ${topicName}