I am trying to install apache-spark on my iMac, using the instructions on these 2 pages:
https://medium.com/macoclock/how-to-install-apache-pyspark-on-macbook-pro-4a9249f0d823
https://www.freecodecamp.org/news/installing-scala-and-apache-spark-on-mac-os-837ae57d283f/
During installation, I couldn't run
brew cask install java
so instead I ran
brew install java
I think I managed to install everything, including apache-spark. But when I ran the commands spark-shell or pyspark, I keep running into the following errors:
For spark-shell:
/usr/local/Cellar/apache-spark/3.0.1/libexec/bin/spark-shell: line 60: /usr/local/Cellar/apache-spark/2.0.1/libexecexport/bin/spark-submit: No such file or directory
For pyspark:
/usr/local/Cellar/apache-spark/3.0.1/libexec/bin/pyspark: line 24: /usr/local/Cellar/apache-spark/2.0.1/libexecexport/bin/load-spark-env.sh: No such file or directory /usr/local/Cellar/apache-spark/3.0.1/libexec/bin/pyspark: line 68: /usr/local/Cellar/apache-spark/2.0.1/libexecexport/bin/spark-submit: No such file or directory /usr/local/Cellar/apache-spark/3.0.1/libexec/bin/pyspark: line 68: exec: /usr/local/Cellar/apache-spark/2.0.1/libexecexport/bin/spark-submit: cannot execute: No such file or directory
My iMac is running Big Sur version 11.1. Appreciate any help, thanks.
