How to unit test kafka streams dsl when using schema registry

Question

Lets say I want write a unit test for the example show here :

https://github.com/confluentinc/kafka-streams-examples/blob/5.1.2-post/src/main/java/io/confluent/examples/streams/WikipediaFeedAvroLambdaExample.java

I tried the following methods, both which did not work out for me:

1) Use TopologyTestDriver.

This class is pretty useful as long as schema registry is not involved. I tried making use of MockSchemaRegistryClient but it didn't work out. And even if it does work out, it requires that I create my own serializers which kind of defeats the purpose of schema registry.

2) Use EmbeddedSingleNodeKafkaCluster defined in the same project.

https://github.com/confluentinc/kafka-streams-examples/blob/5.1.2-post/src/test/java/io/confluent/examples/streams/kafka/EmbeddedSingleNodeKafkaCluster.java

Now this class is really handy and seems to have embedded kafka cluster and schema registry. But it does not seem to be available in any artifact. Consequently I tried copying the class but ran into further import issues. Unable to download this particular artifact : io.confluent:kafka-schema-registry-client:5.0.0:tests

Has anyone able to make progress with the above mentioned options? Or even a completely different solution?

Answer 1

For doing this I ended up doing this small test library based on testcontainers: https://github.com/vspiliop/embedded-kafka-cluster. Starts a fully configurable docker based Kafka cluster (broker, zookeeper and Confluent Schema Registry) as part of your tests. Check out the example unit and cucumber tests.