How can I initialize MPI in a function?

Question

I want to use multi-processes in a function and how can I make it.

As you know, MPI_Init needs two parameter: "int argc, char **argv". Does it mean that I must add these two parameters in the function definition?

My requirement is that I want to parallelize a step in function in stead of the step in main program.

For example,

func(mat &A, vec &x) {
  some computation on A;
  auto B = sub_mat(A, 0, 10);
  B*x; // I want to parallelize this computation
}
main(){
  mat A;
  vec x;
  func(A, x);
}

I just want to use MPI in B*x, but I don't know how to init MPI? By the way, if I can init MPI int func, does A exist in every processes at this time?

Help me & thanks!

You seem to have a fundamental mis understanding about what MPI is. MPI provides a communication and synchronization mechanism across ranks (i.e. where a rank is a process...at least in all the interesting implementations). In particular, the entire executable will need to be duplicated in each rank - both the "main" and "func". The kind of parallelism you are attempting is likely better achieved with threads, perhaps using OpenMP. — Stan Graves

Hristo Iliev Hristo Iliev · Accepted Answer · 2012-07-28T14:27:08

You do not need to pass argc and argv around since MPI-2 lifted the restriction in MPI-1 that compilant implementations may require arguments to MPI_Init to be the same as the arguments to main:

In MPI-2 implementations are not allowed to impose this requirement. Conforming implementations of MPI are required to allow applications to pass NULL for both the argc and argv arguments of main.

But you still have to test if MPI is already initialised since MPI_Init() (or MPI_Init_thread()) should be called no more than once. This is done using MPI_Initialized(), so your code should look like this:

int initialized, finalized;

MPI_Initialized(&initialized);
if (!initialized)
   MPI_Init(NULL, NULL);

// Perform work in parallel
...

// You also need this when your program is about to exit
MPI_Finalized(&finalized);
if (!finalized)
   MPI_Finalize();

Note that MPI can be initialised and then finalised only once for the entire lifetime of the application. That is surrounding a block of function code with MPI_Init() ... MPI_Finalize() won't work if the function is to be called multiple times, i.e. MPI doesn't function the same way as OpenMP with its parallel regions does.

By the way, if I can init MPI int func, does A exist in every processes at this time?

A running MPI program consists of multiple processes with their own private address spaces. Usually these are multiple copies of the same program code (the so-called Single Program Multiple Data or SPMD paradigm), but could also be multiple copies of several programs, written to work together (also called Multiple Programs Multiple Data or MPMD). SPMD is the more simple and more common case where all processes execute exactly the same code up to the point where their MPI rank is used to branch the execution into multiple directions. So yes, A exists in every process and if no (pseudo-)random numbers/events are involved in the preceding computations, then A would have the same value in every MPI process prior to the initialisation of the MPI library. Note that MPI_Init() is just a regular library call as any other library call. It doesn't change the content of user memory - it only makes the multitude of running MPI processes aware of one another and enables them to communicate with each other, thus enabling them to work collectively in order to solve the particular problem.

How can I initialize MPI in a function?

2 Answers