intellij idea with scala error on : import org.apache.spark.{SparkContext, SparkConf}

Question

I am using intelli j idea (like eclipse)with scala integration , i create a scala sbt project , i use spark 1.4.0 and scala 2.11.6 , I am getting error on : import org.apache.spark.{SparkContext, SparkConf}

the file buid.sbt contains this code :

name := "simple"

version := "1.0"

scalaVersion := "2.11.6"

libraryDependencies += "org.apache.spark" % "spark-core_2.10" % "1.4.0"

when i use maven to build java application i have not problem just the problem is when i try to build scala application with sbt using intellij idea

Answer 1

Your scala version and spark scala version are wrong. Change your spark dependency:

libraryDependencies += "org.apache.spark" %% "spark-core" % "1.4.0"

Answer 2

You can't use spark-core_2.10 with scala version 2.11.6.

2.10 and 2.11 are not binary compatible.

You probably want spark-core_2.11

Answer 3

maybe you need read this:https://github.com/mpeltonen/sbt-idea. you need to run sbt gen-idea after config plugins.sbt and build.sbt