Make slaves wait for each other in MPI

Question

I have a master-slave model in my MPI program. I want to make slaves wait for each other before going to the next iteration.

if (rank == 0) {
    // master process
} else {
    // slave process
    for (int i = 0; i < 10; i++) {
        // do stuff
        // wait for all slaves to end iteration i
    }
}

Basically, I don't want any processor to go into next iteration without all other slaves complete their current iteration. How can I do this? With MPI_Barrier?

Gilles Gilles · Accepted Answer · 2016-01-10T13:21:13

You can create a communicator comprising with all slave processes and use it on a MPI_Barrier(). Fro creating this communicator, the simplest / safest is to use MPI_Comm_split() this way:

MPI_Comm slaves;
MPI_Comm_split( MPI_COMM_WORLD, ( rank == 0 ), rank, &slaves );

This will actually globally create 2 communicators: one comprising only the master process and one comprising all processes but the master.

For actual use, you can do it this way:

if (rank == 0) {
    // master process
} else {
    // slave process
    for (int i = 0; i < 10; i++) {
        // do stuff
        // wait for all slaves to end iteration i
        MPI_Barrier( slaves );
    }
}

Make slaves wait for each other in MPI

1 Answers