How do I resolve: Akka.Remote.EndpointDisassociatedException?

2
votes

I have some code that involves remote deploying actors onto a separate process.

I am getting: Akka.Remote.EndpointDisassociatedException

[WARNING][3/24/2017 1:54:32 PM][Thread 0008][[akka://system1/system/endpointMana ger/reliableEndpointWriter-akka.tcp%3A%2F%2Fsystem2%40localhost%3A8080-1#1408457 663]] Association with remote system akka.tcp://system2@localhost:8080 has faile d; address is now gated for 5000 ms. Reason is: [Akka.Remote.EndpointDisassociat edException: Disassociated at Akka.Remote.EndpointWriter.PublishAndThrow(Exception reason, LogLevel leve l, Boolean needToThrow) at Akka.Actor.ReceiveActor.ExecutePartialMessageHandler(Object message, Parti alAction1 partialAction) at Akka.Actor.ActorCell.<>c__DisplayClass114_0.<Akka.Actor.IUntypedActorConte xt.Become>b__0(Object m) at Akka.Actor.ActorBase.AroundReceive(Receive receive, Object message)
at Akka.Actor.ActorCell.ReceiveMessage(Object message) at Akka.Actor.ActorCell.AutoReceiveMessage(Envelope envelope) at Akka.Actor.ActorCell.Invoke(Envelope envelope)] [ERROR][3/24/2017 1:54:32 PM][Thread 0008][akka://system1/system/endpointManager /reliableEndpointWriter-akka.tcp%3A%2F%2Fsystem2%40localhost%3A8080-1/endpointWr iter] Disassociated Cause: Akka.Remote.EndpointDisassociatedException: Disassociated at Akka.Remote.EndpointWriter.PublishAndThrow(Exception reason, LogLevel leve l, Boolean needToThrow) at Akka.Actor.ReceiveActor.ExecutePartialMessageHandler(Object message, Parti alAction1 partialAction) at Akka.Actor.ActorCell.<>c__DisplayClass114_0.b__0(Object m) at Akka.Actor.ActorBase.AroundReceive(Receive receive, Object message)
at Akka.Actor.ActorCell.ReceiveMessage(Object message) at Akka.Actor.ActorCell.AutoReceiveMessage(Envelope envelope) at Akka.Actor.ActorCell.Invoke(Envelope envelope)

Here's the code that I execute in a separate process that triggers that error:

use system = ActorSystem.Create("system1", config)
let reply = system.ActorOf<ReplyActor>("reply")

let props1 = Props.Create(typeof<SomeActor>, [||])
let props2 = Props.Create(typeof<SomeActor>, [||])
let props3 = Props.Create(typeof<SomeActor>, [||])

let remote1 = system.ActorOf(props1.WithRouter(FromConfig.Instance), "remoteactor1")
let remote2 = system.ActorOf(props2.WithRouter(FromConfig.Instance), "remoteactor2")
let remote3 = system.ActorOf(props3.WithRouter(FromConfig.Instance), "remoteactor3")

let hashGroup = system.ActorOf(Props.Empty.WithRouter(ConsistentHashingGroup(config)))
Task.Delay(500).Wait();

let routee1 = Routee.FromActorRef(remote1);
hashGroup.Tell(new AddRoutee(routee1));

let routee2 = Routee.FromActorRef(remote2);
hashGroup.Tell(new AddRoutee(routee2));

let routee3 = Routee.FromActorRef(remote3);
hashGroup.Tell(new AddRoutee(routee3));

Task.Delay(500).Wait();

for i = 0 to 5 do
    for j = 0 to 7 do

        let message = new HashMessage(j, sprintf "remote message: %i" j);
        hashGroup.Tell(message, reply);

Console.ReadLine() |> ignore

Here's the configuration that my remote deploy code relies on:

open Akka.Configuration

let config = ConfigurationFactory.ParseString(@"
    akka {  
        log-config-on-start = on
        stdout-loglevel = DEBUG
        loglevel = DEBUG
        actor {
            provider = ""Akka.Remote.RemoteActorRefProvider, Akka.Remote""

            debug {  
              receive = on 
              autoreceive = on
              lifecycle = on
              event-stream = on
              unhandled = on
            }

            deployment {
                /localactor {
                    router = consistent-hashing-pool
                    nr-of-instances = 5
                    virtual-nodes-factor = 10
                }
                /remoteactor1 {
                    router = consistent-hashing-pool
                    nr-of-instances = 5
                    remote = ""akka.tcp://system2@localhost:8080""
                }
                /remoteactor2 {
                    router = consistent-hashing-pool
                    nr-of-instances = 5
                    remote = ""akka.tcp://system2@localhost:8080""
                }
                /remoteactor3 {
                    router = consistent-hashing-pool
                    nr-of-instances = 5
                    remote = ""akka.tcp://system2@localhost:8080""
                }
            }
        }
        remote {
            helios.tcp {
                port = 8090
                hostname = localhost
            }
        }
    }
    ")

Here's the C# code that actually works that my F# implementation is based off:

var config = ConfigurationFactory.ParseString(@"
akka {  
    log-config-on-start = on
    stdout-loglevel = DEBUG
    loglevel = DEBUG
    actor {
        provider = ""Akka.Remote.RemoteActorRefProvider, Akka.Remote""

        debug {  
          receive = on 
          autoreceive = on
          lifecycle = on
          event-stream = on
          unhandled = on
        }

        deployment {
            /localactor {
                router = consistent-hashing-pool
                nr-of-instances = 5
                virtual-nodes-factor = 10
            }
            /remoteactor1 {
                router = consistent-hashing-pool
                nr-of-instances = 5
                remote = ""akka.tcp://system2@localhost:8080""
            }
            /remoteactor2 {
                router = consistent-hashing-pool
                nr-of-instances = 5
                remote = ""akka.tcp://system2@localhost:8080""
            }
            /remoteactor3 {
                router = consistent-hashing-pool
                nr-of-instances = 5
                remote = ""akka.tcp://system2@localhost:8080""
            }
        }
    }
    remote {
        dot-netty.tcp {
            port = 8090
            hostname = localhost
        }
    }
}
");
            using (var system = ActorSystem.Create("system1", config))
            {
                var reply = system.ActorOf<ReplyActor>("reply");

                //create a remote deployed actor
                var remote1 = system.ActorOf(Props.Create(() => new SomeActor(null, 123)).WithRouter(FromConfig.Instance), "remoteactor1");
                var remote2 = system.ActorOf(Props.Create(() => new SomeActor(null, 456)).WithRouter(FromConfig.Instance), "remoteactor2");
                var remote3 = system.ActorOf(Props.Create(() => new SomeActor(null, 789)).WithRouter(FromConfig.Instance), "remoteactor3");

                var hashGroup = system.ActorOf(Props.Empty.WithRouter(new ConsistentHashingGroup(config)));

                Task.Delay(500).Wait();

                var routee1 = Routee.FromActorRef(remote1);
                hashGroup.Tell(new AddRoutee(routee1));

                var routee2 = Routee.FromActorRef(remote2);
                hashGroup.Tell(new AddRoutee(routee2));

                var routee3 = Routee.FromActorRef(remote3);
                hashGroup.Tell(new AddRoutee(routee3));

                Task.Delay(500).Wait();

                for (var i = 0; i < 5; i++)
                {
                    for (var j = 0; j < 7; j++)
                    {
                        var message = new SomeMessage(j, $"remote message: {j}");
                        hashGroup.Tell(message, reply);
                    }
                }

                Console.ReadLine();
            }
        }
    }
}

Can anyone provide guidance on why I'm getting this exception and how I can resolve it?

Hence, the F# implementation closely mirrors the working C# implementation. The F# code can be found on GitHub.

1
f#akka.netakka.net-cluster

1 Answers

3
votes

When you're starting your application, you may read an exact exception that causes node disassociation: Could not load file or assembly 'System1....

What you've defined in your routers configuration is remote deployment. This means, that from one system you're trying to create actors on another node and communicate with them as if they were available locally. While this is possible, there is one requirement: a destination actor system must know how to build an actor. Since your actors are defined in System1 and created in System2, which doesn't know anything about SomeActor it fails and causes actor system to disassociate.

You need to pass SomeActor class to shared assembly, available for both systems, in order for your scenario to work.