We are testing Flink's local-recovery option to achieve fast recovery for our large keyed state. We canceled our current running job and then restart it from the last checkpoint and we found that the previous state remained in the file system. We want to ask if the state files would not be deleted even if we already resumed it. We would not want our local task's disk usage getting unlimited increased.
0
votes
1 Answers
0
votes
The state files will not be deleted because there is an new jobid assigned to the resumed job, so Flink will create a new directory to store the checkpoint files. And it totally makes sense to me.
Let's assume that if Flink deletes the state files after recovery, what are you going to do if the program fails again?
