Maven setup for spark project in Eclipse Scala IDE

0
votes

Im using Eclipse for Scala IDE to develop a Spark application. Im using a Maven project, but when i try to import sparkcon like : 

import org.apache.spark.SparkConf

I have the error :

object apache is not a member of package org can you help me to setup spark dependancies

2
eclipsescalamaven
Have you compile your meven? – SharpLu

2 Answers

2
votes

You need to put the Spark dependency in your pom.xml file. As of now, the latest version of Spark can be obtained by putting:

<dependency>
    <groupId>org.apache.spark</groupId>
    <artifactId>spark-core_2.10</artifactId>
    <version>2.2.0</version>
</dependency>

in the dependencies in your pom.xml.