Union of Streaming Dataframe and Batch Dataframe in Spark Structured Streaming

Question

How can I combine streaming dataframe and batch dataframe together in Spark Structured Streaming ?

Answer 1

You can do normal join:

val stream = spark.readStream(....)
val batch = spark.read....

val joinSandB = stream.join(batch, "someColumn")

See more in documentation