I want to read multiple big files using Akka Streams to process each line. Imagine that each key consists of an (identifier -> value)
. If a new identifier is found, I want to save it and its value in the database; otherwise, if the identifier has already been found while processing the stream of lines, I want to save only the value. For that, I think that I need some kind of recursive stateful flow in order to keep the identifiers that have already been found in a Map
. I think I'd receive in this flow a pair of (newLine, contextWithIdentifiers)
.
I've just started to look into Akka Streams. I guess I can manage myself to do the stateless processing stuff but I have no clue about how to keep the contextWithIdentifiers
. I'd appreciate any pointers to the right direction.