The same random numbers for each process in c++ code with MPI

Question

I have C++ MPI code that works, in that it compiles and does indeed launch on the specified number of processors (n). The problem is that it simply does the same calculation n times, rather than doing one calculation n times faster.

I have hacked quite a few examples I have found on various sites, and it appears I am missing the proper use of MPI_Send and MPI_Receive, but I can't find an instance these commands that takes a function as input (and am confused as to why these MPI commands would be useful for anything other than functions).

My code is below. Essentially it calls a C++ function I wrote to get Fisher's Exact Test p-value. The random-number bit is just something I put in to test the speed.

What I want is for this program to do is farm out Fisher.TwoTailed with each set of random variables (i.e., A, B, C, and D) to a different processor, rather than doing the exact same calculation on multiple processors. Thanks in advance for any insight--cheers!

Here is the code:

int 
main (int argc, char* argv[]) 
{ 

  int id;
  int p;
//
//  Initialize MPI.
//
  MPI::Init ( argc, argv );
//
//  Get the number of processors.
//
  p = MPI::COMM_WORLD.Get_size ( );
//
//  Get the rank of this processor.
//
 id = MPI::COMM_WORLD.Get_rank ( );

  FishersExactTest Fishers;

  int i = 0;
  while (i < 10) {
    int A = 0 + rand() % (100 - 0);
    int B = 0 + rand() % (100 - 0);
    int C = 0 + rand() % (100 - 0);
    int D = 0 + rand() % (100 - 0); 
    cout << Fishers.TwoTailed(A, B, C, D) << endl;
    i += 1;
  }

  MPI::Finalize ( );
  return 0;
}

Answer 1

You should look into some basic training about parallel computing and MPI. One good resource that taught me the basics was a free set of online courses put up by the National Center for Supercomputing Applications (NCSA).

Answer 2

You have to tell MPI how to parallelize the code - it won't do it automatically.

In other words, you can't initialize MPI on all the systems and then pass them the same loop. You want to use the id of each processor to determine which part of the loop it will work on. Then you need them to all pass their results back to ID 0.

Answer 3

All of the above answers are perfectly correct. Let me just add a little bit:

Here, since it looks like you're just doing random sampling, all you have to do to get the different processors to generate different random numbers to give to Fishers.TwoTailed is to ensure they all have different seeds to the PRNG:

int 
main (int argc, char* argv[]) 
{ 

  int id;
  int p;
//
//  Initialize MPI.
//
  MPI::Init ( argc, argv );
//
//  Get the number of processors.
//
  p = MPI::COMM_WORLD.Get_size ( );
//
//  Get the rank of this processor.
//
 id = MPI::COMM_WORLD.Get_rank ( );

  FishersExactTest Fishers;

  srand(id);  // <--- each rank gets a different seed

  int i = 0;
  while (i < 10) {
    int A = 0 + rand() % (100 - 0);
    int B = 0 + rand() % (100 - 0);
    int C = 0 + rand() % (100 - 0);
    int D = 0 + rand() % (100 - 0); 
    cout << Fishers.TwoTailed(A, B, C, D) << endl;
    i += 1;
  }

  MPI::Finalize ( );
  return 0;
}

Because the loop is from 1..10, you'll still get each process doing 10 samples. If you want them to do a total of 10, you can divide 10 by p and do something to distribute the remainder: eg

int niters = (10+id)/p;
int i=0;
while (i < niters) {
  ...
}

Answer 4

Well, what messages do you get when you run your MPI job? Just to reiterate what the others have said, you will have to explicitly define what the job of each processor is...for example..if you are rank 0 (default), do this...if you are rank 1 so on..(or some other syntax) that defines the role for each rank. Then you could, based on how you structure your code, have nodes Send/Recv, Gather, Scatter etc.