Flink - how to solve error This job is not stoppable

Question

I tried to stop a job through flink stop

flink stop [jobid]

However the CLI throws error and does not allow me to stop the job. I could cancel it. What could be the reason here?

Stopping job c7196bb1d21d679efed73770a4e4f9ed.

------------------------------------------------------------ The program finished with the following exception:

org.apache.flink.util.FlinkException: Could not stop the job c7196bb1d21d679efed73770a4e4f9ed. at org.apache.flink.client.cli.CliFrontend.lambda$stop$5(CliFrontend.java:557) at org.apache.flink.client.cli.CliFrontend.runClusterAction(CliFrontend.java:988) at org.apache.flink.client.cli.CliFrontend.stop(CliFrontend.java:550) at org.apache.flink.client.cli.CliFrontend.parseParameters(CliFrontend.java:1065) at org.apache.flink.client.cli.CliFrontend.lambda$main$11(CliFrontend.java:1129) at org.apache.flink.runtime.security.NoOpSecurityContext.runSecured(NoOpSecurityContext.java:30) at org.apache.flink.client.cli.CliFrontend.main(CliFrontend.java:1129) Caused by: java.util.concurrent.ExecutionException: org.apache.flink.runtime.rest.util.RestClientException: [Job termination (STOP) failed: This job is not stoppable.] at java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:357) at java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1915) at org.apache.flink.client.program.rest.RestClusterClient.stop(RestClusterClient.java:392) at org.apache.flink.client.cli.CliFrontend.lambda$stop$5(CliFrontend.java:555) ... 6 more Caused by: org.apache.flink.runtime.rest.util.RestClientException: [Job termination (STOP) failed: This job is not stoppable.] at org.apache.flink.runtime.rest.RestClient.parseResponse(RestClient.java:351) at org.apache.flink.runtime.rest.RestClient.lambda$submitRequest$3(RestClient.java:335) at java.util.concurrent.CompletableFuture.uniCompose(CompletableFuture.java:952) at java.util.concurrent.CompletableFuture$UniCompose.tryFire(CompletableFuture.java:926) at java.util.concurrent.CompletableFuture$Completion.run(CompletableFuture.java:442) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748)

David Anderson David Anderson · Accepted Answer · 2018-12-16T14:41:48

For a Flink job to be stoppable, all of its sources have to implement the Stoppable interface, and at this point it is still the case that very few do. Among the source connectors bundled with Flink, only the Storm SpoutWrapper, the TwitterSource, and the NifiSource support stop.

It may appear that you are meant to use stop, but for most purposes, the cancel command is more appropriate. The only conceivable benefit to stopping, as opposed to canceling, is that you might avoid lost or duplicated results. But if you are using connectors that implement the checkpointing interface, such as the Kafka connector, then it doesn't matter how you shutdown a job -- you can achieve exactly-once semantics regardless.

Update: the Stoppable interface was removed in Flink 1.9, and the stop command was reimplemented, and is now preferred in most cases over the cancel command. See the documentation for more details.

Flink - how to solve error This job is not stoppable

2 Answers