I'm running MPI code that looks like this (see below for explanation):
#include <mpi.h>
#include <sys/time.h>
#include <stdlib.h>
#include <stdio.h>
#include <string.h>
#include <unistd.h>
...
int main(int argc, char * argv[])
{
int i,j,local_N,num_procs = 0;
int N = 16; // width and height of matrix
int rank;
float ** A;
float ** local_A;
// MPI stuff
MPI_Init(&argc, &argv);
MPI_Comm_rank(MPI_COMM_WORLD, &rank);
MPI_Comm_size(MPI_COMM_WORLD, &num_procs); // "size" is number of processes
MPI_Status status;
// allocate and initialize A
if (rank == 0) {
A = allocate_matrix(N, N);
initialize_matrix(A, N, N);
}
// allocate local matrices
local_N = N / num_procs;
local_A = allocate_matrix(local_N, N);
//send/rcv pieces of matrix
for (i = 1; i < num_procs; i++) {
if (rank == 0) {
MPI_Send(A[i * local_N], N, MPI_FLOAT, i, 123, MPI_COMM_WORLD);
}
if (rank == i) {
MPI_Recv(local_A[0], N, MPI_FLOAT, 0, 123, MPI_COMM_WORLD, &status);
}
}
if (rank == 0) free_matrix(A, N, N);
free_matrix(local_A, local_N, N);
MPI_Finalize();
return 0;
}
and here are those helper methods:
float ** allocate_matrix(int rows, int cols) {
int i = 0;
float ** matrix = (float **) malloc(rows * sizeof(float *));
for (i = 0; i < cols; i++) {
matrix[i] = (float *) malloc(cols * sizeof(float));
}
return matrix;
}
void initialize_matrix(float ** matrix, int rows, int cols) {
int i, j = 0;
for (i = 0; i < rows; i++) {
for (j = 0; j < cols; j++) {
matrix[i][j] = ((float)(rand()%10000))/1000.0;
}
}
}
void free_matrix(float ** matrix, int rows, int cols) {
int i, j = 0;
for (j = 0; j < rows; j++) free(matrix[j]);
}
Basically, I'm splitting up a 16 by 16 matrix into (number of processes) chunks and sending one row from each chunk to each process. Printing the rows before and after the send and receive shows that they are being sent properly.
I've run this code over and over using mpirun -n 4 ./<name of executable>, and each time, something different happens. Either:
normal exit
mpirun noticed that process rank ... exited on signal 6mpirun noticed that process rank ... exited on signal 10mpirun noticed that process rank ... exited on signal 11
I saw the signal 6 and thought "double free()!", but I've tried removing the freeing methods, and the signal 6's persist, along with the other errors. Any idea why these errors are occurring, and why it's a different error each time? The debugger trace makes it look like the problem is happening in MPI_Finalize().