Spark History Server not showing completed Jobs

1
votes

Context:

  • Spark version 3 here (latest as of posting)
  • EMR 6.1

1 - Settings are correctly filled - according to the doc : (/etc/spark/conf/spark-defaults.conf)

enter image description here

2 - History Server Service is Up

enter image description here

3 - Log Files are produced & present, with open rights

enter image description here

4 - But still that stubborn History Server UI refuses to display stuff, although the log files for the jobs are produced

enter image description here

I restarted the hsitory server many time, but no luck
I am seriously out of options here...
Any Ideas what could go wrong ?

1
apache-sparkpyspark

1 Answers

0
votes

So... Instead of restarting the spark history server, I just stopped it.

enter image description here

And guess, what... that web uisucker was still there !

enter image description here

So I headed to the terminal, armed to kill, and found that mastard and gave it a headshot.

enter image description here

I refreshed... and voilà !

enter image description here

Sometimes the usage of brute force is the only way out ... I still wonder what really happened exactly -technically speaking, it is still kinda not really clear to me...