why this two MPI Sends behave differently?

Question

I have a pipeline like pattern communication between MPI processes. where processes Send message to each other as pipeline stages. The following example shows three processes communicating in such pattern.

    #include "mpi.h"
    #include <stdlib.h>
    #include <stdio.h>

    //declare stage function
    void* testcall(void* d);

    int main(int args, char** argv){
        int rank, size;
        MPI_Status status;
        MPI_Init(&args,&argv);
        MPI_Comm_rank(MPI_COMM_WORLD,&rank);
        MPI_Comm_size(MPI_COMM_WORLD,&size);

        if(rank==0){ 
            int k;  
            int x[3] = {10,11,12};
            void* data = malloc(sizeof(int));             
            for( k=0 ; k< 3;k++){           
                data = &x[k];           
                MPI_Send(data,4,MPI_BYTE,1,0,MPI_COMM_WORLD);   
            }
        }

        if(rank==1){                                
            void* rcv = malloc(sizeof(int));
            void* snd = malloc(sizeof(int));
            int k;
            for( k=0 ; k< 3;k++){
                MPI_Recv(rcv,4,MPI_BYTE,0,0,MPI_COMM_WORLD,&status);              
                snd = testcall(rcv);                        
                int z = *(int *) snd;  
                printf("RCV 1: %d \n",z);
                MPI_Send(&snd,4,MPI_BYTE,2,0,MPI_COMM_WORLD);
            }
        }

        if(rank==2){
            void* rcv2 = malloc(sizeof(int));
            void* snd2 = malloc(sizeof(int));
            int k;
            for( k=0 ; k< 3;k++){
                MPI_Recv(rcv2,4,MPI_BYTE,1,0,MPI_COMM_WORLD,&status);
                snd2 = testcall(rcv2);
                int z = *(int *) snd2;
                printf("RCV 2: %d \n",z);
            }
        }

        MPI_Finalize();
        return 0;
    }

    void* testcall(void* d){
        int z = *(int *) d;
        int k = z * 2;
        void* rslt = malloc(sizeof(int));
        rslt = &k;
        return rslt;
    }

output:

RCV1: 20

RCV1: 22

RCV1: 24

RCV: 2136566600

I have one problem with the code though. Send from process 0 to process 1 succeed and give me the correct when I print it.

while Send from process 1 to process 2 seems succeed, but when I try to print it is just an unexpected number (as shown in the output above).

I don't understand why this two sends behave differently. (both are sending a value pointed by a void pointer. why the second send is erroneous )??

help please.

Is there any particular reason why you're using void pointers and MPI_BYTE instead of int and MPI_INT? — suszterpatt
yes, I am trying to make the function that each stage call generic. Each function will accept void* argument and return void* The assumption is that a user will define a function with this prototype and also specify size of input and output in byte. — LeTex

Hristo Iliev Hristo Iliev · Accepted Answer · 2012-10-01T10:43:39

void* testcall(void* d){
    int z = *(int *) d;
    int k = z * 2;
    void* rslt = malloc(sizeof(int));
    rslt = &k;
    return rslt;
}

This code doesn't do what you expect. The rslt = &k; line overwrites the value of the rslt pointer with the address of k (and you do this in several other statements). First, this leads to memory leaks, as the address of the memory region, allocated by malloc, is lost. Second, k is an automatic (stack) variable at its location would get used for other purposes once testcall returns. It only works in your case because no other function call is present between testcall() and int z = *(int *) snd;. The correct function should be:

void* testcall(void* d){
    int z = *(int *) d;
    int k = z * 2;
    int* rslt = malloc(sizeof(int));
    *rslt = k;
    return rslt;
}

Then this line from the rank 1 code:

MPI_Send(&snd,4,MPI_BYTE,2,0,MPI_COMM_WORLD);

snd is a pointer itself. You are sending the address of the pointer and that's why rank 2 prints strange values. The correct statement should read:

MPI_Send(snd,4,MPI_BYTE,2,0,MPI_COMM_WORLD);

Output:

RCV 1: 20
RCV 1: 22
RCV 1: 24
RCV 2: 40
RCV 2: 44
RCV 2: 48

why this two MPI Sends behave differently?

1 Answers