I have the following PySpark Input Dataframe:
+-------+------------+
| index | valuelist |
+-------+------------+
| 1.0 | [10,20,30] |
| 2.0 | [11,21,31] |
| 0.0 | [14,12,15] |
+-------+------------+
Where:
- Index: type Double
- Valuelist: type Vector. (it's NOT Array)
From the above Input Dataframe, I want to get the following Output Dataframe in PySpark
+-------+-------+
| index | value |
+-------+-------+
| 1.0 | 20 |
| 2.0 | 31 |
| 0.0 | 14 |
+-------+-------+
Logic:
for each row:
value = valuelist[index]