MPI_Barrier in C

Question

I'm trying to implement a program using MPI, for which I need to have a block of code to be executed in a particular processor and until the execution completes other processors must wait.I thought it can be achieved using MPI_Barrier (though I'm not clear with its actual functionality) and tried out the following program.

#include<mpi.h>
#include<stdio.h>
int main(int argc, char **argv) {
    int rank=0,size;
    MPI_Init(&argc,&argv);
    MPI_Comm_rank(MPI_COMM_WORLD,&rank);
    MPI_Comm_size(MPI_COMM_WORLD,&size);
    if(rank == 0){                             //Block 1
        printf("\nRank 0 Before Barrier");
    }
        MPI_Barrier(MPI_COMM_WORLD);
    if(rank==1){
        printf("\nRank 1 After Barrier");
        printf("\nRank 1 After Barrier");
    }
    if(rank==2){
        printf("\nRank 2 After Barrier");
    }
    MPI_Finalize();
}

I got the following output when I executed with np as 3

Rank 1 After Barrier

Rank 0 Before BarrierRank 2 After BarrierRank 1 After Barrier

How could I possibly make the other processors to wait until Block 1 completes its execution in the processor with Rank 0?

Intended Output

Rank 0 Before Barrier
Rank 1 After Barrier //After this, it might be interchanged
Rank 1 After Barrier
Rank 2 After Barrier

MPI_Barrier() blocks the all process until all of them reaches to barrier. — Polymorphism
try adding fflush(stdout); after every printf statement. — eduffy
That or do fprintf(stderr...). stderr is unbuffered by default. — Aaron Altman

Aaron Altman Aaron Altman · Accepted Answer · 2013-12-30T21:51:05

Besides the issue with concurrent writing to stdout that eduffy pointed out in a comment, the barrier you've used is only part of what you need to do to ensure ordering. Once all 3 or more ranks pass the one barrier you've inserted, all possible interleavings of rank 1 and 2 are allowed:

Rank 1 After Barrier
Rank 1 After Barrier
Rank 2 After Barrier

or:

Rank 1 After Barrier
Rank 2 After Barrier
Rank 1 After Barrier

or:

Rank 2 After Barrier
Rank 1 After Barrier
Rank 1 After Barrier

You need to do some kind of synchronization between ranks 1 and 2 after the barrier you've got now to make sure that rank 1 gets its first printf done before rank 2 can continue. That could be another barrier, a barrier on a smaller communicator just containing ranks 1 and 2 if you don't want to force other ranks to wait, a blocking MPI_Ssend/MPI_Recv pair with dummy data or similar.

MPI_Barrier in C

1 Answers