Apache Flink for a real-time processing of finite streams

0
votes

I am considering how to use Apache Flink for a voting system I’m currently developing. I am a complete newbie to Flink and any help would be appreciated.

The requirements are:

  • A user with some privilege can start a vote on an arbitrary issue. The user can close the vote any time they like.
  • As much as hundreds of thousands of people possibly join a vote
  • Counting of votes should be started immediately after a vote is started, and the intermediate results should be updated over time, so that it can be shown to the participants.
  • When the system finishes counting after a vote has been closed, it should notify the participants of the final result.

In my understanding, Flink’s stream processing is for a real-time processing of infinite streams, while batch processing is for a non-real-time processing of finite streams.
How can I apply Flink to my requirement, which is a real-time processing of finite streams?

1
apache-flink

1 Answers

1
votes

Flink's DataStream API can process events of finite streams without any problems. The DataStream program will simply terminate when the stream reached its end.

You can simulate this behavior if you use a SocketTextStreamFunction to read text data from a socket. Once you close the socket, the program will terminate. Alternatively, you can also read data from a file which is also some kind of finite stream. However, keep in mind that incomplete windows will not be automatically evaluated. So you have to make sure that you do not lose data in windows if you use them.