Unit Testing Flink Streams with Multiple Event Streams

Question

I have 2 DataStreams that I connect and feed into a CoFlatMap function. I need to be able to test generating messages on two different DataStreams, but coordinating when the messages arrive. How does one go about doing that in Flink?

Answer 1

I ended up going with: https://github.com/ottogroup/flink-spector/

The documentation could use a little updating, but overall it works really well. Here's the pom.xml entry:

    <dependency>
        <groupId>io.flinkspector</groupId>
        <artifactId>flinkspector-datastream_2.11</artifactId>
        <version>0.8.3</version>
        <scope>test</scope>
    </dependency>

Answer 2

Take a look on the official unit testing guide. Check especially last paragraphs regarding using TestHarness. In your case it seems like you should use TwoInputStreamOperatorTestHarness. You can find example how to use it in CoProcessOperatorTest