Does groupbykey functionality in Beam

Question

We need to use groupbykey to achive the functionality

KV < ID , < List of objects > > like

KV <1, < obj1,obj2 >>

Could you please tell if groupbykey works in Spark runner. According to capability matrix it is not supported but according the following link it is supported.

https://issues.apache.org/jira/browse/BEAM-799

Thanks

Answer 1

If you press "expand details" on the capability matrix, you can see that GroupByKey is supported in the Spark runner with the note:

Partially: fully supported in batch mode

Using Spark's groupByKey. GroupByKey with multiple trigger firings in streaming mode is a work in progress.