Akka.net load balancing and span out processing

Question

I am looking to build a system that is able to process a stream of requests that needs a long processing time say 5 min each. My goal is to speed up request processing with minimal resource footprint which at times can be a burst of messages.

I can use something like a service bus to queue the request and have multiple process (a.k.a Actors in akka) that can subscribe for a message and start processing. Also can have a watchdog that looks at the queue length in the service bus and create more actors/ actor systems or stop a few.

if I want to do the same in the Actor system like Akka.net how can this be done. Say something like this:

I may want to spin up/stop new Remote Actor systems based on my request queue length
Send the message to any one of the available actor who can start processing without having to check who has the bandwidth to process on the sender side.
Messages should not be lost, and if the actor fails, it should be passed to next available actor.

can this be done with the Akka.net or this is not a valid use case for the actor system. Can some one please share some thoughts or point me to resources where I can get more details.

If you already have something like ServiceBus in front of your system, I don't see what advantage are you looking for. 1. Akka.NET is a toolkit / library. It's meant to be used from within your process, but it won't spawn other machines on demand. You need resource manager for that. 2. & 3. can be solved by service bus itself. — Bartosz Sypytkowski
@Horusiath: I would like to move out of Windows service bus 1.1 as soon as possible. its a pain to deploy as part of our solution especially when systems don't have access to internet. I see that there is something called akka.clustering Just digging to see if that can help here.. — Kiran

Bartosz Sypytkowski Bartosz Sypytkowski · Accepted Answer · 2016-08-26T15:37:36

While you may achieve some of your goals with Akka cluster, I wouldn't advise that. From your requirements it clearly states that your concerns are oriented about:

Reliable message delivery (where service buses and message queues are better option). There are a lot of solutions here, depending on your needs i.e. MassTransit, NServiceBus or queues (RabbitMQ).
Scaling workers (which is infrastructure problem and it's not solved by actor frameworks themselves). From what you've said, you don't even even need a cluster.

You could use akka for building a message processing logic, like workers. But as I said, you don't need it if your goal is to replace existing service bus.

Akka.net load balancing and span out processing

2 Answers