Kafka Streams Add New Source to Running Application

Question

Is it possible to add another source topic to an existing topology of a running kafka streams java application. Based on the javadoc (https://kafka.apache.org/23/javadoc/org/apache/kafka/streams/KafkaStreams.html) I am guessing the answer is no.

My Use Case: REST api call triggers a new source topic should be processed by an existing processor. Source topics are stored in a DB and used to generate the topology.

I believe the only option is to shutdown the app and restart it allowing for the new topic to be picked up.

Is there any option to add the source topic without shutting down the app?

Answer 1

You cannot modify the program while it's running. As you point out, to change anything, you need to stop the program and create a new Topology. Depending on your program and the change, you might actually need to reset the application before restarting it. Cf. https://docs.confluent.io/current/streams/developer-guide/app-reset-tool.html