Thread-safe random number generator with openMP

Question

The following simple script produce random number in parallel

#include <random>
#include <iostream>
#include <omp.h>

int main(int argc, char *argv[])
{
  // Three integers are expected from the command line.
  // The first integer is a random seed
  // The second integer is the number of threads.
  // The third integer indicates the number of random numbers to produce

  // Read the seed and create the random number generator and the random distribution
  int seed = std::stoi(argv[1]);
  std::mt19937 mt(seed);
  std::uniform_real_distribution<float>  dist(0, 100);

  // Read the number of threads and set it.
  int nbThreads = std::stoi(argv[2]);
  omp_set_num_threads(nbThreads);

  // Number of random number for demonstration
  int n = std::stoi(argv[3]);

  // Will store the random number to print them conveniently
  std::vector<float> store(n);

  // produce 'n' random numbers
  #pragma omp parallel for
  for (int i = 0 ; i < n ; ++i)
  {
    store[i] = dist(mt);
  }

  // print the random numbers
  for ( auto& rnd : store )
  {
    std::cout << rnd << std::endl;
  }

  return 0;
}

The above script is deterministic when using a single thread

./test 3 1 2
55.0798
7.07249

./test 3 1 2
55.0798
7.07249

./test 7 1 2
7.63083
22.7339


./test 7 1 2
7.63083
22.7339

However, it is partially stochastic and contain correlation between threads (which can be a pretty big issue) when using more than one thread

./test 3 2 2
43.1925
43.1925

./test 3 2 2
55.0798
7.07249

 ./test 7 2 2
22.7339
7.63083

./test 7 2 2
7.63083
7.63083

I understand why my code is not thread-safe but I fail to understand how to make it thread-safe. Is it possible to have deterministic output regardless of the number of threads?

The goal is that ./test 87 1 200 yield the same output as ./test 87 3 200 (that is the number of threads won't affect the object store). If this is not possible, then the goal is that ./test 87 3 200 yield the same output as ./test 87 3 200.

Can you be more clear about what you want to do? Clearly, a PRNG that produces a sequence of outputs can only produce those outputs in sequence. What do you want? — David Schwartz
@DavidSchwartz I edited the post hoping to make it more clear. I am trying to make a deterministic process that takes a random seed and produce random number in a multithreaded fashion. Is it possible? I am not sure my comment just helped you to understand. Can you please be more specific about what is unclear? — Remi.b
What are the threads going to be doing exactly? For the result to be deterministic, there must be a first result that goes to some particular place. And then, and only then, there can be a second result. Right? It seems you are asking both to do things in a precise sequence (one, then the next, and so on) but also to do them in a multithreaded fashion. — David Schwartz
You can only get deterministic output per thread. This is because you cannot guarantee the sequence in which each thread generates random numbers. Obviously a shared PRNG will give you a deterministic sequence of numbers (provided you 'sabotage' your threading a bit to make it thread-safe); but since which thread gets which number is not deterministic, any derivations from the the random numbers are also not deterministic. Your only reasonable option is to use a separate PRNG instance per thread. — Disillusioned
@CraigYoung Ok, so it si not possible to ensure that ./test 87 1 200 give the same output as ./test 87 3 200 but it is possible to ensure that ./test 87 3 200 give systematically the same output if we define a PRNG seeded from a value of the main PRNG for each thread. Rest to investigate if this is possible with openMP. Thanks — Remi.b

IanM_Matrix1 IanM_Matrix1 · Accepted Answer · 2016-12-05T23:23:52

You are sharing the state in the std::mt19937 mt object with each thread, which is not thread-safe.

Either wrap the access of that object with locking of some sort, or provide a separate instance for each thread [EDIT] seeded differently (maybe from the first instance of mt19937 you create) so that each instance gives different results [/EDIT] (if any of that's possible using omp).

Thread-safe random number generator with openMP

2 Answers