Erlang ensure Mnesia schema replication

Question

I have a distributed application. In this, a Master node starts a mnesia schema with 4 tables. Some of them are replicated to other nodes, some are not. When a node spawns, it registers at the master node and is added to the schema and the data are replicated to this node.

How can I ensure that my replication is finished? I tried the following:

Timeout=60000,
TabList = [tab1, tab2, tab3, tab4],
mnesia:wait_for_tables(TabList, Timeout).

However, it does not take 60 seconds, not even 5 seconds until I get an error:

{{badmatch,{aborted,{no_exists,tab1}}}

Obviously it does not work..

When a new node joins a cluster, a rpc call from the master node performs the following function on the new node:

start_Mnesia(MasterNode) ->
mnesia:start(),
mnesia:change_config(extra_db_nodes, [MasterNode]),
Tabs=mnesia:system_info(tables) -- [schema], 
[mnesia:add_table_copy(Tab, node(), ram_copies) || Tab <- Tabs].

Is it also waiting until it is written to ram_copies?

Thanks.

Synchronisation and replication are sort of two different things (replication doesn't have to be synchronised with anything in any way as such for example; it can be asynchronous). Do you actually care when the replication has completed? Or do you care about synchronisation from a transactional point of view (i.e. you want to safely carry out DB transactions)? — Michael
Well, my use case is a distributed firewall. When a new node joins the cluster it has to know which rules are set to decide if a packet may pass or not. Thus, I initially need all data at the new node before traffic is inspected. — muehsi
You don't, when your new node has joined the cluster, it can access the data in the tables, the data doesn't have to be copied locally. — Michael

Michael Michael · Accepted Answer · 2016-06-26T14:34:59

When a node joins your mnesia cluster it is already synchronised, regardless of copies of tables it, or other nodes, do or do not have.

You should see that a new node, after registering with your master and being added to the cluster, can already access all your tables. Adding a copy of a table doesn't change that, regardless of the state/stage of that copy.

When you add a copy of a table on your new node you can continue to run transactions, during and after replication, and the data that is or is not replicated to the node originating the transaction will make no difference to the correctness of the result.

So, if you are concerned just with synchronisation in terms of keeping your transactions ACID, then don't worry.

If you are concerned when when your data is actually replicated and stored safely on the other nodes, that's a different thing. In this case, I have observed that when you runmnesia:add_table_copy(Table, NewNode, disc_copies) it blocks, and returns only when NewNode has copied the data to the filesystem.

Remember though, unless you run mnesia:sync_transaction/3 all the time you don't have guarantees about data actually being on any disc after a transaction completes anyway.

Erlang ensure Mnesia schema replication

1 Answers