How to stop Flink application in live production environment without loosing streaming data

Question

I have a flink application reading data from AWS Kinesis Stream. The producer application writes to the Kinesis stream at the rate of 600 records per sec.

I wanted to know what are the best practices around stopping the flink application in live production environment while the producer is still writing to streams without loosing the input stream data.

I see the need to stop the application for scheduled maintenance, application deployment changes etc.

Answer 1

You should use savepoints, which enables you to store the internal state of your flink application and further restart your application from that point in time.

For more info see the docs