Why does MPI isend irecv not working?

2
votes

I have an array of user defined types. Each of the two processors writes uniquely to a portion of the array, and finally, I want to have the combined array. Why is my code generating a segmentation fault? 

program type_derived_recv_send
  use nodeinfo
  implicit none
  include 'mpif.h'
  integer(4) :: ierr
  integer(4) :: istatus1(MPI_STATUS_SIZE)
  integer(4) :: istatus2(MPI_STATUS_SIZE)
  integer(4) :: i,j,k,l
  integer(4) :: iblock(8)
  integer(4) :: idisp(8)
  integer(4) :: itype(8)
  integer(4) :: inewtype
  integer(4) :: iextent
  integer(4) :: itag, isend, irecv
  integer(4) :: n,m,p,q

    TYPE :: newXYZ
        real :: x, u
        real :: y, v
        real :: z, w
        real,dimension(3) :: uvw     
    END TYPE

    TYPE (newXYZ) :: xyz, xyz2

    TYPE (newXYZ), allocatable :: xyzArray(:,:,:,:) 
    n = 1
    m = 1 
    p = 1
    q = 2

    allocate( xyzArray(m,n,p,q) )


  call MPI_INIT(ierr)
  icomm = MPI_COMM_WORLD
  call MPI_COMM_SIZE(icomm,nproc,ierr)
  call MPI_COMM_RANK(icomm,iproc,ierr)

do l=1,q
    do k=1,p
        do j=1,n
            do i=1,m 
                    xyzArray(i,j,k,m)%x = 0
                    xyzArray(i,j,k,m)%y = 0
                    xyzArray(i,j,k,m)%z = 0
                    xyzArray(i,j,k,m)%u = 0
                    xyzArray(i,j,k,m)%v = 0
                    xyzArray(i,j,k,m)%w = 0
                    xyzArray(i,j,k,m)%uvw = (/0,0,0/)
end do
end do
end do
end do

    if (iproc == 1) then
        xyzArray(1,1,1,2)%x = 1.1
        xyzArray(1,1,1,2)%y = 2.1
        xyzArray(1,1,1,2)%z = 3.1
        xyzArray(1,1,1,2)%u = 4.1
        xyzArray(1,1,1,2)%v = 5.1
        xyzArray(1,1,1,2)%w = 6.1
        xyzArray(1,1,1,2)%uvw = (/10.1,10.1,10.1/)
    else
        xyzArray(1,1,1,1)%x = 0.1
        xyzArray(1,1,1,1)%y = 0.1       
        xyzArray(1,1,1,1)%z = 0.1
        xyzArray(1,1,1,1)%u = 0.1
        xyzArray(1,1,1,1)%v = 0.1       
        xyzArray(1,1,1,1)%w = 0.1 
        xyzArray(1,1,1,1)%uvw = (/0.1,0.1,0.1/)
    endif

    itag = 1
    xyz = xyzArray(1,1,1,2)
    xyz2 = xyzArray(1,1,1,1)

    if ( iproc == 1) call MPI_ISEND(xyzArray(1,1,1,2),36,MPI_INTEGER,0,1,icomm,isend,ierr)
    if ( iproc == 0) call MPI_IRECV(xyzArray(1,1,1,2),36,MPI_INTEGER,1,1,icomm,irecv,ierr)
    call MPI_WAIT(isend,istatus1,ierr)
    call MPI_WAIT(irecv,istatus2,ierr)

     if ( iproc == 0 )write(6,*)'iproc = ',iproc
     if ( iproc == 0 ) write(6,*)xyzArray

     if ( iproc == 1 )write(6,*)'iproc = ',iproc
     if ( iproc == 1 ) write(6,*)xyzArray

  call MPI_FINALIZE(ierr)
end program type_derived_recv_send

Output: 

--------------------------------------------------------------------------
 newtype extent =           36
 xyz =    0.000000       0.000000       0.000000       0.000000       0.000000       0.000000       0.000000       0.000000       0.000000    
 xyz2 =   0.1000000      0.1000000      0.1000000      0.1000000      0.1000000      0.1000000      0.1000000      0.1000000      0.1000000    
[west0334:09950] *** Process received signal ***
[west0334:09950] Signal: Segmentation fault (11)
[west0334:09950] Signal code: Address not mapped (1)
[west0334:09950] Failing at address: 0x40
[west0334:09950] [ 0] /lib64/libpthread.so.0 [0x3e00e0eb70]
[west0334:09950] [ 1] /n/sw/openmpi-1.2.5-gcc-4.1.2/lib64/libmpi.so.0(ompi_request_wait+0x17) [0x2b33f8628607]
[west0334:09950] [ 2] /n/sw/openmpi-1.2.5-gcc-4.1.2/lib64/libmpi.so.0(PMPI_Wait+0x88) [0x2b33f86566e8]
[west0334:09950] [ 3] /n/sw/openmpi-1.2.5-gcc-4.1.2/lib64/libmpi_f77.so.0(PMPI_WAIT+0x3a) [0x2b33f83eac8a]
[west0334:09950] [ 4] ./type_derived_recv_send.x(MAIN__+0x730) [0x401cd0]
 newtype extent =           36
 xyz =    1.100000       4.100000       2.100000       5.100000       3.100000       6.100000       10.10000       10.10000       10.10000    
 xyz2 =    0.000000       0.000000       0.000000       0.000000       0.000000       0.000000       0.000000       0.000000       0.000000    
[west0334:09951] *** Process received signal ***
[west0334:09951] Signal: Segmentation fault (11)
[west0334:09951] Signal code: Address not mapped (1)
[west0334:09951] Failing at address: 0x40
[west0334:09950] [ 5] ./type_derived_recv_send.x(main+0xe) [0x4049ae]
[west0334:09950] [ 6] /lib64/libc.so.6(__libc_start_main+0xf4) [0x3e0021d994]
[west0334:09950] [ 7] ./type_derived_recv_send.x [0x4014e9]
[west0334:09950] *** End of error message ***
[west0334:09951] [ 0] /lib64/libpthread.so.0 [0x3e00e0eb70]
[west0334:09951] [ 1] /n/sw/openmpi-1.2.5-gcc-4.1.2/lib64/libmpi.so.0(ompi_request_wait+0x17) [0x2b73c1aca607]
[west0334:09951] [ 2] /n/sw/openmpi-1.2.5-gcc-4.1.2/lib64/libmpi.so.0(PMPI_Wait+0x88) [0x2b73c1af86e8]
[west0334:09951] [ 3] /n/sw/openmpi-1.2.5-gcc-4.1.2/lib64/libmpi_f77.so.0(PMPI_WAIT+0x3a) [0x2b73c188cc8a]
[west0334:09951] [ 4] ./type_derived_recv_send.x(MAIN__+0x748) [0x401ce8]
[west0334:09951] [ 5] ./type_derived_recv_send.x(main+0xe) [0x4049ae]
[west0334:09951] [ 6] /lib64/libc.so.6(__libc_start_main+0xf4) [0x3e0021d994]
[west0334:09951] [ 7] ./type_derived_recv_send.x [0x4014e9]
[west0334:09951] *** End of error message ***
Nov 16 20:17:14 2012 9939 4 7.06 handleTSRegisterTerm(): TS reports task <0> pid <9950> on host<west0334> killed or core dumped
Job  /lsf/7.0/linux2.6-glibc2.3-x86_64/bin/openmpi_wrapper ./type_derived_recv_send.x
2
segmentation-faultfortranmpi

2 Answers

4
votes

The error is here:

if ( iproc == 1) call MPI_ISEND(xyzArray(1,1,1,2),36,MPI_INTEGER,0,1,icomm,isend,ierr)
if ( iproc == 0) call MPI_IRECV(xyzArray(1,1,1,2),36,MPI_INTEGER,1,1,icomm,irecv,ierr)
call MPI_WAIT(isend,istatus1,ierr)
call MPI_WAIT(irecv,istatus2,ierr)

The irecv request handle remains uninitialised in rank 0 while the isend request handle remains unitialised in rank 1. There are two options here - either group the MPI_WAIT call with its corresponding non-blocking MPI operation or initialise both isend and irecv to MPI_REQUEST_NULL before that.

Solution 1:

if ( iproc == 1) then
   call MPI_ISEND(xyzArray(1,1,1,2),36,MPI_INTEGER,0,1,icomm,isend,ierr)
   call MPI_WAIT(isend,istatus1,ierr)
end if
if ( iproc == 0) then
   call MPI_IRECV(xyzArray(1,1,1,2),36,MPI_INTEGER,1,1,icomm,irecv,ierr)
   call MPI_WAIT(irecv,istatus2,ierr)
end if

Solution 2 (simplified):

ireq = MPI_REQUEST_NULL
if ( iproc == 1) call MPI_ISEND(xyzArray(1,1,1,2),36,MPI_INTEGER,0,1,icomm,ireq,ierr)
if ( iproc == 0) call MPI_IRECV(xyzArray(1,1,1,2),36,MPI_INTEGER,1,1,icomm,ireq,ierr)
call MPI_WAIT(ireq,istatus,ierr)

You don't really need two separate request handles as well as two separate status arrays. With ireq initialised with a null handle, this code would also function correctly with more than 2 MPI processes.

Note that in order to be portable, you should construct a derived structure datatype with MPI_TYPE_CREATE_STRUCT and use it to send and receive elements of TYPE(newXYZ).

0
votes

Why are you sending a type with 9 real with an integer MPI datatype ? Why not sending 9 MPI_REAL or 36 MPI_BYTE