Handling Redundancy and Failover in Akka.Net

Question

I've implemented an Akka.Net actor system in a service that accepts remote requests and does processing and sends back responses. Some of the actors within the system have state that gets persisted to a database. When the actor system starts up, these actors get hydrated by reading their state from the database.

When a request comes in, actors are loaded lazily on demand and stay resident in memory to process further requests.

I'm now looking at how I can provide redundancy in the simplest way possible.

I'm guessing the simplest solution here is to have a second copy of the service on 'cold' standby. When the first node goes down, requests get routed to the second node - which will then start creating actors and getting their state from the database.

Then the original node, when it comes up, needs to recognize that its not the 'primary' and terminate all actors (so it can read them back with their state from disk when it becomes active)

What's the best of achieving this? I'm guessing I should use Akka.Cluster and listen for when nodes join or leave a cluster? But how do I say that all messages should essentially go to one node? Is there some sort of leader-election process but for individual roles instead of the cluster as a whole?

Is there a better/simpler way to do this?

Oberdan Nunes Oberdan Nunes · Accepted Answer · 2016-08-08T02:12:32

I'm a newbie using akka.net, but I think you can use Akka.Net Cluster + SingletonActor. I'm making some tests here and it seems to work very well.

Sample:

    private void Start() {
        var system = ActorSystem.Create("SingletonActorSystem");
        var cluster = Cluster.Get(system);
        cluster.RegisterOnMemberRemoved(() => MemberRemoved(system));

        var actor = system.ActorOf(ClusterSingletonManager.Props(
            singletonProps: Props.Create<ProcessorCoordinatorActor>(),
            terminationMessage: PoisonPill.Instance,
            settings: ClusterSingletonManagerSettings.Create(system)),
            name: "processorCoordinator");

        Console.ReadLine();

        cluster.Leave(cluster.SelfAddress);
        _leaveClusterEvent.WaitOne();
    }

    private async void MemberRemoved(ActorSystem actorSystem) {
        await actorSystem.Terminate();
        _leaveClusterEvent.Set();
    }

Configuration:

    akka {
        suppress-json-serializer-warning = on

        actor {
            provider = "Akka.Cluster.ClusterActorRefProvider, Akka.Cluster"
        }

        remote {
            helios.tcp {
                port = 0
                hostname = localhost
            }
        }

        cluster {
            auto-down-unreachable-after = 5s
            down-removal-margin = 5s
            seed-nodes = [ "akka.tcp://[email protected]:4053" ] 
            roles = [worker]

            singleton {
                singleton-name = "processorCoordinator"
                role = "worker"
                hand-over-retry-interval = 5s
            }
        }
    }

Handling Redundancy and Failover in Akka.Net

1 Answers