MPI shared memory with different window sizes in Fortran

Question

I am trying to adopt my code to exploit MPI shared memory.

To keep things as simple as possible, imagine I have two cores only. Core A needs an array of size_b from core b and core b needs an array of size_a from core a

When executing on core A, Can I do the following?

 call MPI_WIN_ALLOCATE_SHARED(size_a_in_BYTES, disp_unit, MPI_INFO_NULL, comm_shm, ptr, win_a, mpierr)
 call C_F_pointer(ptr, fptr, (/size_a/))

and for core B

 call MPI_WIN_ALLOCATE_SHARED(size_b_in_BYTES, disp_unit, MPI_INFO_NULL, comm_shm, ptr, win_b, mpierr)
 call C_F_pointer(ptr, fptrb, (/size_b/))

Now let assume I afterwards communicate by MPI_SEND/RECV win_a and win_b respectively to core B and core a. The logic would now be to make the two queries in following manner

Core A:

        call MPI_WIN_SHARED_QUERY( win_b
 &                               , rank_of_core_B
 &                               , size_b
 &                               , disp_unit
 &                               , ptr_buf
 &                               , mpierr )
        call C_F_pointer(ptr_buf, fptr_query_A, (/size_b/))

and visa versa for core B

Let say I know in Core B point my pointer fptrb to an array i.e.

  fptrb = GIVEN_ARRAY_OF_SIZEB

Would I then be able to retrieve the memory in CORE a by accessing fptr_query_A for example

   fptr_recv = fptr_query_A

where fptr_recv is a pointer declared in core A of the same KIND as fptr_query_A

I am asking this because it is not clear from the manual whether the routine MPI_WIN_ALLOCATE_SHARED, besides being collective for the comm_shm, has to have same win and size for all shared memory cores

Please while answering, I think some of my confusion stems from the following

1) When calling MPI_WIN_ALLOCATE_SHARED am I allocating a local memory from the calling processor to the shared environment i.e. the shared communication group.

1.a) If yes, It then means that each processor declares what it wants to share and from the receiver's (the processor that wants to access shared memory) point of review, it make a call to MPI_WIN_SHARED_QUERY to know where in memory a processor has stored its shareable data?

Finally, if these two latter points are overall valid, I am not sure I understand how to do the MPI_WIN_SHARED_QUERY call.

My understanding is MPI_WIN_SHARED_QUERY( win, rank, size,disp_unit, baseptr) that win is the window of whatever core we want to retrieve the data, similarly for the size and rank. So essentially does that mean I always need to communicate the win and size in order to make that call?

A2LBK A2LBK · Accepted Answer · 2019-11-04T14:02:22

I have meanwhile been able to answer the question myself, and the answers are

1) When calling MPI_WIN_ALLOCATE_SHARED am I allocating a local memory from the calling processor to the shared environment i.e. the shared communication group.

Yes each Rank will allocate a seperate buffer with a respective size that accessed by all other ranks within the shared communicator group. Even if rank does not need to allocate a shared buffer, it shall call the function with a zero size.

If yes, It then means that each processor declares what it wants to share and from the receiver's (the processor that wants to access shared memory) point of review, it make a call to MPI_WIN_SHARED_QUERY to know where in memory a processor has stored its shareable data?

That is how I understand it.

My understanding is MPI_WIN_SHARED_QUERY( win, rank, size,disp_unit, baseptr) that win is the window of whatever core we want to retrieve the data, similarly for the size and rank. So essentially does that mean I always need to communicate the win and size in order to make that call?

Since all MPI ranks must call MPI_WIN_ALLOCATE_SHARED each will have their own MPI handle win. Hence, you use this handle to call MPI_WIN_SHARED_QUERY to obtain a pointer that points to the shared buffer allocated by the MPI rank rank. This function also queries the size of the rank's shared memory.

MPI shared memory with different window sizes in Fortran

1 Answers