zeppelin-0.7.3 Interpreter pyspark not found

Question

I get the below error when I use pyspark via Zeppelin.

The python & spark interpreters work and all environment variables are set correctly.

print os.environ['PYTHONPATH']

/x01/spark_u/spark/python:/x01/spark_u/spark/python/lib/py4j-0.10.4-src.zip:/x01/spark_u/spark/python:/x01/spark_u/spark/python/lib/py4j-0.10.4-src.zip:/x01/spark_u/spark/python/lib/py4j-0.10.4-src.zip:/x01/spark_u/spark/python/lib/pyspark.zip:/x01/spark_u/spark/python:/x01/spark_u/spark/python/pyspark:/x01/spark_u/zeppelin/interpreter/python/py4j-0.9.2/src:/x01/spark_u/zeppelin/interpreter/lib/python

zepplin-env.sh is set with the below vars

export PYSPARK_PYTHON=/usr/local/bin/python2
export PYTHONPATH=${SPARK_HOME}/python:${SPARK_HOME}/python/lib/py4j-0.10.4-src.zip:${PYTHONPATH}

export SPARK_YARN_USER_ENV="PYTHONPATH=${PYTHONPATH}"

See the below log file

INFO [2017-11-01 12:30:42,972] ({pool-2-thread-4} RemoteInterpreter.java[init]:221) - Create remote interpreter org.apache.zeppelin.spark.PySparkInterpreter org.apache.zeppelin.interpreter.InterpreterException: paragraph_1509038605940_-1717438251's Interpreter pyspark not found

Thank you in advance

sankad sankad · Accepted Answer · 2017-11-21T17:38:38

I found a workaround for the above issue.The interpreter not found issue does not happen when I create note inside a directory.The issue only happens when I use notes at toplevel.Addionally I foud out that this issue does not happen in 0.7.2 version Ex : enter image description here

zeppelin-0.7.3 Interpreter pyspark not found

1 Answers