Azure HDInsight Jupyter notebook not working

1
votes

I have deployed a HDInsight 3.6 Spark (2.3) cluster on Microsoft Azure with the standard configurations (Location = Central US, Head Nodes = D12 v2 (x2) - 8 cores, Worker Nodes = D13 v2 (x4)) - 32 cores).

Launch Jupyter notebook and when selecting spark notebook gives a weird error which i am not able to figure out .

enter image description here

4
jupyter-notebookazure-hdinsight
Welcome to Stackoverflow! Please copy-paste the text of the error and/or provide a sample code. Indeed, the image is not easy to read and text instead of image could improve the visibilty of your question, thus helping other user facing the same issue. - francis
Used inline image - Chris Schaller
Facing the same issue myself. What does azure test ! A jupyter notebook which is the starting point itself is not working. - Krishan Subudhi

4 Answers

0
votes

I am on the same boat as you and it was frustrating. I couldn't make the jupyter notebook work on azure . But I was able to run spark commands through pyspark shell. My experience in this blog

https://krishansubudhi.github.io/datascience/2020/01/12/hdinsight.html

However, I was able to open jupyter notebook using AWS. There is a section on this in the end of the blog post.

0
votes

The HDInsight team is in the process of rolling out a self-healing action for this issue. Once the self-healing action has been deployed, the Jupyter service on the affected clusters will have to be started again for the changes to take effect. I will update this thread as the fix is deployed.

0
votes

I resolved the same issue with 2 steps:

  1. Connect to headnode via ssh and change content of file - /usr/bin/anaconda/lib/python2.7/site-packages/nbformat/_version.py. replace 5 to 4(by sudo vim..)

enter image description here

  1. Restart Jupyter service via Ambari

enter image description here

-1
votes

The code is converting jupyter notebook to v5 format which is supported only in Python 3.

You need to use python 3.5+ built-in anaconda environment or setup virtual python 3 environmet