Apache Beam Wait.on JdbcIO.write with unbounded PCollection issue

Question

I am trying use the below scenario with unbounded pCollection datasource(PubSub). https://issues.apache.org/jira/browse/BEAM-6732

I am able to write to DB1. DB2 is having a Wait.on DB1 (PCollection .withResults). But unfortunately DB2 is not getting updated.

When I change the source to a bounded dummy PCollection, it works. Any input is appreciated.

Answer 1

As I mentioned on Jira, do you have any Windows using in your unbounded pipeline? The writing to another database starts only after the respective window has been fully written to the first database. Otherwise, it will wait forever.