Can't convert Dataframe to Labeled Point

Question

My program uses Spark.ML, I use logistic regression on dataframes. However I would like to use LogisticRegressionWithLBFGS too so I want to convert my dataframe into LabeledPoint.

The following code gives me an error

val model = new LogisticRegressionWithLBFGS().run(dff3.rdd.map(row=>LabeledPoint(row.getAs[Double]("label"),org.apache.spark.mllib.linalg.SparseVector.fromML(row.getAs[org.apache.spark.ml.linalg.SparseVector]("features")))))

Error :

org.apache.spark.ml.linalg.DenseVector cannot be cast to org.apache.spark.ml.linalg.SparseVector

So I changed SparseVector to DenseVector but it doesn't work :

org.apache.spark.ml.linalg.SparseVector cannot be cast to org.apache.spark.ml.linalg.DenseVector

Answer 1

Have you tried to use org.apache.spark.mllib.linalg.Vectors.fromML instead?

Note: This answer is a copy paste from the comments to allow it to be closed.