When are records written to write ahead log with spark streaming?

Question

I have writeAheadLog enabled in my application that uses spark streaming. When exactly will the records be written to the writeAheadLog? Is it when the store(...) is called or as soon as the record is received/accepted? As I am trying to implement a reliable receiver I want to first collect a bunch of records before actually storing them. Now can I be sure that the records I have in the buffer and have yet to call store() upon, will be present in the writeAheadLog should the application fail before storing?

Also, what would be the correct way to handle the records in the log when application is restarted. I would like those events to be stored, is there an easy way to access them?

Answer 1

I did some testing and it appears that records are written to writeAheadLog upon store(). This means that additional actions are required to make failsafe the buffer where records are collected to before storing.