Structured Streaming in pyspark

Question

Am trying to stream data from another server to HBase and be able to define different column families in Python. I have looked around in the Spark docs and only seeing:

writestream.format('jdbc').start('jdbc:///')

How can I have the same implementations to write directly to HBase with the ability to map data to different column families?

Answer 1

You can write your data to HBase using foreach (Scala or Java): http://spark.apache.org/docs/latest/structured-streaming-programming-guide.html#using-foreach