How to get an iterator from an akka streams Source?

1
votes

I'm trying to create a flow that I can consume via something like an Iterator. I'm implementing a library that exposes an iterator-like interface, so that would be the simplest thing for me to consume.

My graph designed so far is essentially a Source<Iterator<DataRow>>. One thing I see so far is to flatten it to Source<DataRow> and then use http://doc.akka.io/japi/akka/current/akka/stream/javadsl/StreamConverters.html#asJavaStream-- followed by https://docs.oracle.com/javase/8/docs/api/java/util/stream/BaseStream.html#iterator--

But given that there will be lots potentially many rows, I'm wondering whether it would make sense to avoid the flattening step (at least within the akka streams context, I'm assuming there's some minor per-element overhead when passed via stages), or if there's a more direct way.

Also, I'm curious how backpressure works in the created stream, especially the child Iterator; does it only buffer one element?

1
javaakkaakka-streamakka-http
Could you elaborate on why you need to turn it into an Iterator? – Viktor Klang
I'm implementing an api which exposes a forward-only cursor over rows of data (essentially the iterator interface). Either I'll need to consume an iterator, or essentially implement one on top of something else. – Bwmat

1 Answers

2
votes

Flattening Step

Flattening a Source<Iterator<DataRow>> to a Source<DataRow> does add some amount of overhead since you'll have to use flatMapConcat which does eventually create a new GraphStage.

However, if you have "many" rows then this separate stage may come in handy since it will provide concurrency for the flattening step.

Backpressure

If you look at the code of StreamConverters.asJavaStream you'll see that there is a QueueSink that is spawning a Future to pull the next element from the akka stream and then doing an Await.result(nextElementFuture, Inf) to wait on the Future to complete so the next element can be forwarded to the java Stream.

Answering your question: yes the child Iterator only buffers one element, but the QueueSink has a Future which may also have the next DataRow. Therefore the javaStream & Iterator may have 2 elements buffered, on top of however much buffering is going on in your original akka Source.