How to convert RDD like ((int, int), int) into 3 key-value pairs in one map function in spark?

1
votes

Right now I have a RDD in the following format

( (int, int), int )

and I'm trying to convert it into 3 key-value pairs like (int, 1).

I can do this by using 3 map functions then join them together, but it's definitely not the best way to implement it,

also by using case I can generate a list of ((int, 1), (int, 1), (int, 1)), but how can I generate a List of (int, 1)?

2
scalaapache-spark

2 Answers

2
votes

Perhaps you want flatMap?

rdd.flatMap { case ((x, y), z) => List((x, 1), (y, 1), (z, 1)) }

The above code produces RDD[(Int, Int)], expanding each of the nested tuples in your RDD into three separate RDD elements.

1
votes

It shouldn't take three map functions but a single map function as shown below

inputRDD.map({case ((x,y),z) => ((x,1),(y,1),(z,1))})