MPI: send and recv must share the same communicator. Why?

Question

I have found written

"A message can ONLY be received within the same communicator from which it was sent". However, if I look at this picture

https://imgur.com/a/hYz4dWd

and then analyze this code

Send and Receive operations between communicators in MPI

use mpi !instead of include 'mpif.h'
implicit none

integer :: tag,ierr,rank,numtasks,color,new_comm,inter1,inter2
integer :: sendbuf,recvbuf,stat(MPI_STATUS_SIZE)
integer :: irank
!
tag = 22
sendbuf = 222
!
call MPI_Init(ierr)
call MPI_COMM_RANK(MPI_COMM_WORLD,rank,ierr)
call MPI_COMM_SIZE(MPI_COMM_WORLD,numtasks,ierr)
!
if (rank < 2) then
    color = 0
else 
    color = 1
end if
!
call MPI_COMM_SPLIT(MPI_COMM_WORLD,color,rank,new_comm,ierr)
!
if (color .eq. 0) then
    call MPI_INTERCOMM_CREATE(new_comm,0,MPI_Comm_world,2,tag,inter1,ierr)
!
call MPI_COMM_RANK(inter1,irank,ierr)
if(irank==0)then
    call mpi_send(sendbuf,1,MPI_INT,0,tag,inter1,ierr)
end if
!
else if(color .eq. 1) then
    call MPI_INTERCOMM_CREATE(new_comm,0,MPI_COMM_WORLD,0,tag,inter2,ierr)
    call MPI_COMM_RANK(inter2,irank,ierr)
    if(irank==0)then
        call mpi_recv(recvbuf,1,MPI_INT,0,tag,inter2,stat,ierr)
        if(ierr/=MPI_SUCCESS)print*,'Error in rec '
        print*,'rec buff = ', recvbuf
    end if
end if
!
call MPI_finalize(ierr)
end program h

to me it seems that I am communicating between two different communicators: inter1 and inter2. Turning to the picture attached, I am communicating from comm1 towards comm2.

Gilles Gouaillardet Gilles Gouaillardet · Accepted Answer · 2019-01-16T07:39:36

The picture is unrelated to the sample code.

Looking at the code, one rank MPI_Send(..., inter1, ...) and an other MPI_Recv(..., inter2, ...).

What matters here is how inter1 and inter2 were created, and they both come from all the ranks invoking MPI_Intercomm_create(), so even if you use different variable names, they indeed refer to the same (and unique) inter-communicator.

Here is a more intuitive way the program could have been written

use mpi !instead of include 'mpif.h'
implicit none

integer :: tag,ierr,rank,numtasks,color,new_comm,inter,remote_leader
integer :: sendbuf,recvbuf,stat(MPI_STATUS_SIZE)
integer :: irank
!
tag = 22
sendbuf = 222
!
call MPI_Init(ierr)
call MPI_COMM_RANK(MPI_COMM_WORLD,rank,ierr)
call MPI_COMM_SIZE(MPI_COMM_WORLD,numtasks,ierr)
!
if (rank < 2) then
    color = 0
    remote_leader=2
else 
    color = 1
    remote_leader=0
end if
!
call MPI_COMM_SPLIT(MPI_COMM_WORLD,color,rank,new_comm,ierr)
!
call MPI_INTERCOMM_CREATE(new_comm,0,MPI_Comm_world,remote_leader,tag,inter,ierr)
call MPI_COMM_RANK(inter,irank,ierr)

if (irank.eq.0) then
    if(color.eq.0) then
        call mpi_send(sendbuf,1,MPI_INT,0,tag,inter,ierr)
    else if(color.eq.1) then
        call mpi_recv(recvbuf,1,MPI_INT,0,tag,inter,stat,ierr)
        if(ierr/=MPI_SUCCESS)print*,'Error in rec '
        print*,'rec buff = ', recvbuf
    end if
end if
!
call MPI_finalize(ierr)
end program

MPI: send and recv must share the same communicator. Why?

1 Answers