Spark History Server .... list of running jobs

2
votes

I am using Cloudera 5.4.1 with Spark 1.3.0. When I go to spark history server, I can see list of completed jobs and list of incomplete jobs.

However many jobs listed as incomplete are the ones which were killed.

So how does one see list of "running" jobs. Not the ones which were killed.

also how does one kill a running spark job by taking the application id from the history server?

1
apache-spark

1 Answers

1
votes

Following is from Cloudera documentation:

To access the web application UI of a running Spark application, open http://spark_driver_host:4040 in a web browser. If multiple applications are running on the same host, the web application binds to successive ports beginning with 4040 (4041, 4042, and so on). The web application is available only for the duration of the application.

For 5.4x For 5.9x

Answer for your second question: You can use yarn CLI to kill the Spark application. Ex: yarn application -kill <application ID>