Why Flink Stream not support left join expression?

0
votes

Flink Stream support inner join expressions like window-join, interval-join. But not support left join / full join expressions. It's surely the window-cogroup expression can implement the same semantics which have to wait a completely window-size time even if events have joined immediately. My Question is that:

  1. How to explain Flink Stream does not support left join / full join expresiions from a design point of view ?
  2. How could I achive it by Flink DataStream API(It's better if can forward joined-event immediately) ?
  3. Is there a way to extend Flink DataStream API to support left join like:
.leftJoin()
.where()
.window()
.apply()
1
flink-streaming

1 Answers

0
votes

The difference between temporally constrained joins like windowed or interval joins, and regular joins, is that in a streaming context, regular joins require indefinite state retention.

Regular left/full joins are available using Flink's Table and SQL APIs. The direction the Flink community has been going is to not put any further effort into developing relational operations with the DataStream API, but to instead improve the interoperability between the DataStream and Table APIs. Flink 1.13 marked a new milestone in making it even easier to convert between streams and tables and back again, and this is the recommended approach whenever relational operations on DataStreams are required.