How build a very large cluster with Erlang?

Question

I'm a newbie to Erlang. In Erlang, node is represented by an atom, like 'name@host'. I want to ask how a node can communicate with the other nodes without increasing the number of atoms of it?

I want to build a very distributed storage system which may contain thousands of nodes. For a specified node A, it can send/receive messages to/from any other nodes in the cluster, for example:

rpc:call(Node, Module, Method, [])

But with the node joining and leaving the cluster, node A may have communicated to thousands of nodes, in this way, the number of atom of node A will keep increasing and finally reach the limit. How to avoid this from happending? If I use the Pid instead of Node to communicate, for example,

Pid ! Message

Will this way increase the number of atoms in node A? It is said that Pid contains the information of a remote node.

Please note that tags are not keywords. Stuffing the tag list full of the same words that are in your question (scale, large, numbers) will not help categorize it. Always be sure to read the descriptions that appear when selecting tags! — Charles
"Cluster" has a very specific meaning in Erlang. You could use TCP or UDP to network your nodes instead of clustering them. — Nathaniel Waisbrot

David Budworth David Budworth · Accepted Answer · 2014-05-11T13:08:47

The maximum number of atoms is 1048576 and you can raise it with +t. see: erlang docs

There's no chance you hit the limit with erlang clustering. If you were to scale to the 10s of thousands to millions range, you are very likely to have several separate clusters.

Distributed Erlang keeps the cluster alive with tcp heartbeats between nodes. You probably don't really want a single cluster more than a few thousand nodes.

How build a very large cluster with Erlang?

1 Answers