Explicitly distribute MPI processes on cluster nodes

1
votes

My MPI program consists of a master process which sends messages to worker processes which then do the actual computation. The master process is single threaded whereas the worker processes are multi-threaded and use all CPU cores of a node.

In order that all CPU cores on all cluster nodes contribute to the computation I would like to specify that on exactly one of the cluster nodes there should be 2 processes i.e. the master process (proc_id = 0) and another worker process. On all other cluster nodes there should only be one worker process.

How can I achieve that?

Thanks.

1
cmpidistributed-computingopenmpimpich
Are you using a resource manager / batch system? If so, which one. Do you really need a solution that works for both openmpi and mpich? – Zulan
I run my program using mpiexec -n 100 ./my_executable if the cluster has 100 nodes. – Linoliumz
Ideally I would now like to execute mpiexec -n 101 ./my_executable if the cluster has 100 nodes and as described on one node there should be 2 processes i.e. the master process and one worker process. – Linoliumz

1 Answers

2
votes

If you run mpiexec with --map-by node in OpenMPI, it will do exectly what you describe:

https://www.open-mpi.org/doc/v1.8/man1/mpiexec.1.php

Launch processes one per node, cycling by node in a round-robin fashion. This spreads processes evenly among nodes and assigns MPI_COMM_WORLD ranks in a round-robin, "by node" manner.