Understanding how Akka provides back pressure

Question

So, we have a use case in our production systems where we could probably use Akka streams. To understand how Akka streams exactly provide back pressure, I would like to go a bit deeper into our requirements.

We have a Solr cluster that hosts some of our data. Next, we have a Play app that serves the front-end customer facing site. Every in-coming request ultimately boils down to fetching a good deal of data from Solr using the /sql handler that Solr provides. Once we fetch the entire dataset from Solr, we write it back after morphing it, to a Cassandra cluster. This can be converted into a problem which can be solved using Akka streams where the Solr stream from the /sql handler will be the akka Source and the Cassandra storage will be the Sink and everything in between will be custom Flows.

I was studying Akka streams and understand it's an implementation of the Reactive streams. Most notably, the way Akka streams provide back pressure to make sure the customer isn't overwhelmed by the producer. Now, with respect to my use case, I want to understand how Akka provides back pressure.

As I can see it, there's a reactive streams library for Cassandra. Since it's the consumer in our case, this driver will be capable of signalling to the producer about how much data it will be able to receive. That would mean, there has to be a corresponding driver on the producer side that can react to this signal and control the emitting of elements. Specifically, since the producer in our case is Solr, isn't it correct that I would also have to use a reactive-compliant Solr driver that I can use to fetch documents from Solr and stream it in my application? This driver would then be capable of controlling the rate at which it has to fetch the documents from the Solr cluster whenever the Cassandra reactive driver signals it to backpressure. Isn't this correct?

If that is indeed the case, will using Akka streams without a non-reactive driver on the producer side provide any benefits? Specifically, are there other ways that Akka publishers can provide back pressure capabilities in such cases when the driver isn't reactive-compliant?

Answer 1

For Solr, there's also a fully reactive Akka Streams implementation from the Alpakka project, so using that as the Source would handle backpressure, though it would mean not using the SQL interface for expressing the query.

On the other hand, since the Solr SQL interface is essentially a JDBC facade which uses Solr, it's possible to use the Alpakka Slick integration as long as you define an instance of slick.jdbc.JdbcProfile which uses the Solr JDBC driver.