why cannot run pyspark in jupyter notebook ,mac os

0
votes

I downloaded apache pyspark through brew download and also tried download directly from https://spark.apache.org/. But after download I tried type "pyspark" directly in terminal and countered this errors. enter image description here

I also tried all the steps that are available in this manual: https://medium.com/@GalarnykMichael/install-spark-on-mac-pyspark-453f395f240b. But also get error when run spark in jupyter notebook, which are:

enter image description here

1
macosapache-sparkpysparkjupyter-notebook
from my experience the easiest way is to install pyspark via pip (even in anaconda). This has always worked for me and is very convenient - Paul

1 Answers

1
votes

in your screenshot the error is at line 14, you have not imported sparksession. Sparksession in not a pyspark function.

Add : from pyspark.sql import SparkSession to your code, it might give others as per your script but will fix this one. Please share your script as well.