Disadvantages of MPI_Bsend?

5
votes

I recently encountered a deadlock that I have been able to resolve by using MPI_Bsend instead of MPI_Send. If I understand correctly MPI_Bsend is a non-blocking send and it is safe to modify the thing I send without worrying about when the send operation it complete, so 

double x = 1;
MPI_Bsend(&x,1,MPI_DOUBLE,master,1,MPI_COMM_WORLD);
x = 0;

will always result in x being sent at 1.

When reading the documentation on MPI send modes I encountered this warning about MPI_Bsend

A late add-on to the MPI specification. Should be used only when absolutely necessary.

My questions are:

  1. Why? Is it because Bsend may not be supported by all implementations?
  2. What is the oldest version of Open MPI that supports MPI_Bsend?
  3. Are there other cautions I should observe about using MPI_Bsend?
1
mpi
Make sure you read and understand the MPI_Bsend() semantic open-mpi.org/doc/v2.0/man3/MPI_Bsend.3.php mpi-forum.org/docs/mpi-1.1/mpi-11-html/node40.html – Gilles Gouaillardet
There is a high risk you end up reusing the buffer when it is not safe to do so, that is why most people recommend against using MPI_Bsend(). – Gilles Gouaillardet
@GillesGouaillardet what do you mean by reusing the buffer? I don't know what that term means in this context. Do you mean that if I change x later (in my example) it will affect what is sent? Or do you mean that a later send command may replace what I sent using Bsend in the buffer? – taciteloquence
please refer to @Zulan 's excellent answer. long story short MPI_Bsend() is a rope you will likely end up hanging yourself with, so you'd rather be safe not using it. For example, you can malloc() a buffer, MPI_Isend() it and free it after the related MPI_Wait() completes. – Gilles Gouaillardet

1 Answers

5
votes

Why is MPI_BSend problematic?

Buffered operations in MPI require the user to provide a large-enough buffer. MPI describes an operational model how much buffer may be used for each operation - so you can theoretically compute how much buffer will be needed in total. However, in in sufficiently complex applications it is not feasible to correctly compute the amount of buffer that is needed. Not having enough buffer space is an unrecoverable error. This error may be a very nasty heisenbug randomly occurring under specific circumstances.

Note that buffered mode and non-blocking mode is different (even orthogonal) in MPI. It is easier to write a correct MPI program using non-blocking primitives, i.e. using MPI_Isend. This is generally recommended.

What about MPI_BSend support?

The "late addition" quote is misleading. It was already in the very first MPI standard 24 years ago. I wouldn't worry about support by libraries, but rather the other issues I mentioned.