Segmentation fault, multiplying matrices using MPI library

Question

I am writing a program that multiplies two matrices A and B which are stored in text files, and which size could be variant, so my program has to identify the size of matrix A and B, determine if they can be multiplied etc.

Well that's not the problem the real trouble is when I pass the data from master process to slave process, in my program I pass rows from master to slaves and the number of rows depends on the number of the rows of the matrix and the number of processes.

The matrix A is stored by rows, but the matrixB is stored by columns.

matrixA[ 0 ] ----------------

matrixA[ 1 ] ----------------

matrixA[ 2 ] ----------------

matrixB[ 0 ] matrixB[ 1 ] matrixB[ 2 ] .........
|           |         |     |
|           |         |     |
|           |         |     |    

You can find the text files here (for input): matrixA matrixB.

After several days of 80's style debugging (means not debugger at all), I think the problem (the segmentation fault I get as output) is in these code-lines ( from the slave function):

void slave( int id, int slaves, double **matrixA, double **matrixB, double **matrixC )
{
    int type, columnsA, columnsB, rowsA, rowsB, Btype, offset, rows, averageRows, extraRows;
    MPI_Status status;

    /* Recieves columns of A and B from master. */
    type = 3;

    MPI_Recv( &columnsA, 1, MPI_INT, 0, type, MPI_COMM_WORLD, &status );
    MPI_Recv( &rowsA, 1, MPI_INT, 0, type, MPI_COMM_WORLD, &status );
    MPI_Recv( &columnsB, 1, MPI_INT, 0, type, MPI_COMM_WORLD, &status );
    MPI_Recv( &rowsB, 1, MPI_INT, 0, type, MPI_COMM_WORLD, &status );
    printf( "%d slave recieved ColumnA = %d, RowsA = %d, ColumnB = %d, RowsB = %d.\n", id, columnsA, rowsA, columnsB, rowsB );


    /* Recieve from master. */
    type = 0;

    MPI_Recv( &offset, 1, MPI_INT, 0, type, MPI_COMM_WORLD, &status );
    MPI_Recv( &rows, 1, MPI_INT, 0, type, MPI_COMM_WORLD, &status );

    matrixAllocate( &matrixA, columnsA, rows );
    matrixAllocate( &matrixB, rowsB, columnsB );
    matrixAllocate( &matrixC, columnsB, rows );
    printf( "Correctly allocated.\n" );

    /* This part is only to see if the mem was correctly allocated.*/
    for( int i = 0; i < rows; i++ ){
        for( int j = 0; j < columnsA; j++)
            matrixA[ i ][ j ] = i + j;
    }

    for( int i = 0; i < columnsB; i++ ){
        for( int j = 0; j < rowsB; j++)
            matrixB[ i ][ j ] = i * j;
    }

    if ( id == 1 ){
        matrixPrinter( "matrixA", matrixA, rows, columnsA );
        matrixBPrinter( "matrixB", matrixB, rowsB, columnsB );
        matrixPrinter( "matrixC", matrixC, rows, columnsB );
    }

    MPI_Recv( &matrixA, ( rows * columnsA ) , MPI_DOUBLE, 0, type, MPI_COMM_WORLD, &status );
    MPI_Recv( &matrixB, ( rowsB * columnsB ), MPI_DOUBLE, 0, type, MPI_COMM_WORLD, &status );
    printf( "Correctly recieved.\n" );

    matrixPrinter( "matrixA", matrixA, rows, columnsA );
    matrixBPrinter( "matrixB", matrixB, rowsB, columnsB );
    matrixPrinter( "matrixC", matrixC, rows, columnsB );

    if ( id == 1 ){
        printf( "My id is %d.\n", id );
        for ( int i = 0; i < rows; i++ ){
            for( int j = 0; j < columnsA; j++ ){
                printf( "%lf    ", matrixA[ i ][ j ] );
            }
        printf( "\n" );
    }
}

The whole code can be found here. MPI matrix multiplier in C.

The output of the terminal is:

enter image description here

It seems rather unfriendly to include the (input) text as images. What if someone actually wanted to look at what this does? — sehe
When I encounter segfaults, I run the program using gdb. When the segfault occur, I type bt, which prints the stack trace, now it's very easy to spot the problem. — stdcall
@Mellowcandle I tried doing that, but mpi is kinda hard to "just" gdb or valgrind :) Did you? Start with sudo -E mpirun -np 2 xterm -e "gdb test multiply A B C", or use Bullet 6 in these debugging FAQ — sehe

taikou taikou · Accepted Answer · 2013-03-14T22:09:06

the problem is, the matrix is of type "double **" as allocated in "matrixAllocate". When sending and receiving data, MPI assumes that the buf contains the data continiously as an 1-d array, which, however, is not the case.(you can easily check that by printing out the address of each matrix entry)

I think it's a famous pitfall in C: pointer and array are different. If the matrix is a 2-d array, then all the entries are laid out continuously.

My suggestion is to allocate the matrix as 1-d and do not use multidim subscript.

Segmentation fault, multiplying matrices using MPI library

2 Answers