Apache Spark Mlib

3
votes

I'm trying to use the MLlib from Spark to implement KMeans on Java and I've stumbled upon a problem, which is that, despite the fact that I've imported the correct jar's my compiler will not recognise this line

// Cluster the data into two classes using KMeans
    int numClusters = 2;
    int numIterations = 20;
    KMeansModel clusters = KMeans.train(parsedData.rdd(), numClusters, numIterations);

The error I get is: The method train(<RDD> vector, int, int) is undefined for the type KMeans() Which doesn't make any sense since, I've downloaded the latest apache MLlib(1.5.2) jar and also it is defined in the Javadoc.

Any ideas? Has anyone encountered this sort of problem before?

1
javaapache-sparkk-meansapache-spark-mllib
You added the jar dependency to your project as well, yes? - Leet-Falcon
I've never heard of the term. could you briefly enlighten me? - Jack
Project-->Properties-->Build-->Add Jar - Leet-Falcon
Yes. ofcourse! I've added the most recent JAR. (1.5.2) as I said in the description - Jack
You seem to be missing basics concerning Java applications and build tools. Before you get your hands dirty with Spark, you should start with those topics. - eliasah

1 Answers

1
votes

I was getting the similar issue and it was resolved by importing the correct libraries,

import org.apache.spark.mllib.clustering.KMeans;
import org.apache.spark.mllib.clustering.KMeansModel;

instead of

import org.apache.spark.ml.clustering.KMeans;
import org.apache.spark.ml.clustering.KMeansModel;