How to get ec2 EMR instance from terminating after job complete

Question

I want to use the spark-shell after job completes so i dont want my EC2 EMR cluster to stop which it does after it comlpletes the job.

How can i keep the cluster running even thought the job completed?

AWS disable auto terminate?

AWS emr wait ?? any ideas?

Answer 1

You can disable Auto-Termination.

You have cluster termination options. Specifically you can provide options at cluster launch time that will prevent the cluster from auto-terminating.

Note that Termination Protection and Auto-Termination are two different things. The former relates to accidental termination while the latter relates to the transient nature of the cluster.

For more detail, it's worth understanding the EMR Cluster Lifecycle.

Answer 2

Best way to do this is to add KeepJobFlowAliveWhenNoSteps