0
votes

Let's say I have 3 nodes and two possible ArangoDB configurations:

  • ActiveFailover:
    node 1: Agent
    node 2: Leader
    node 3: Follower

or

  • Cluster:
    node 1: 1 Agent 1 DBserver and 1 Coordinator
    node 2: 1 Agent 1 DBserver and 1 Coordinator
    node 3: 1 Agent 1 DBserver and 1 Coordinator

Now let's say nodes 1 and 2 go down. Which configuration is more resilient? Will the Follower in the ActiveFailover configuration be reachable, even though the node 1 running the Agent went down? Or would the Cluster configuration handle this case more gracefully, leaving node 3 available to handle reads and writes?

1

1 Answers

1
votes

Unfortunately, none of the two configurations can survive two nodes going down. The reason for this is that no configuration which favours consistency over availability will ever be able to continue with only a single node out of 3. This is because this one node might be the smaller half of a network split. If we would continue with a single node, then we could face split-brain scenarios, in which the other 2-node half continue as well and we would end up with an inconsistent state.

In ArangoDB we have chosen consistency over availability. Therefore, in any 3 node ensemble in which 2 nodes go down the surviving node will not continue service but rather wait it out until at least one is back.

Now let's compare the two configurations for the case of one node failure: The ActiveFailover configuration will continue to work, because at least one of the two data nodes (Leader and Follower) will survive, and together with the third running only an Agent, they can elect a leader and make the surviving data node the Leader. If the only-Agent node fails, then the leader election still works, since the two other nodes actually also run an Agent. Note however, that if the current Leader fails, a failover happens, but since the replication is only asynchronous, some committed data might be lost!

The cluster essentially behaves the same way, except that it has a more symmetric setup. If any node fails, the two others can continue. Provided all collections have at least replicationFactor 2, the failover can move leadership for each shard to a surviving node. Since replication is synchronous, no committed data is lost. This is the advantage of the cluster over the ActiveFailover setup. However, note that the latency of operations can be higher because of the synchronous replication, and since not all data is concentrated on a single node, performance of some operations can be worse since we have less data-locality. In any case, there is no such thing as a free lunch, one eventually has to pay.