mpiexec and python mpi4py gives rank 0 and size 1

Question

I have a problem with running a python Hello World mpi4py code on a virtual machine.

The hello.py code is:

#!/usr/bin/python
#hello.py
from mpi4py import MPI

comm = MPI.COMM_WORLD

size = comm.Get_size()
rank = comm.Get_rank()

print "hello world from process ", rank,"of", size

I've tried to run it using mpiexec and mpirun, but it is not running well. The output:

$ mpirun -c 4 python hello.py 
hello world from process  0 of 1
hello world from process  0 of 1
hello world from process  0 of 1
hello world from process  0 of 1

And from mpiexec:

$ mpiexec -n 4 python hello.py 
hello world from process  0 of 1
hello world from process  0 of 1
hello world from process  0 of 1
hello world from process  0 of 1

They seem not getting rank and size of comm. What can cause this? How to solve it?

mpiexec --version
mpiexec (OpenRTE) 1.6.5

mpirun --version
mpirun (Open MPI) 1.6.5

The system is Ubuntu 14.04 on the Virtal Machine.

Any ideas why? Thanks!

possible duplicate of MPI_Rank return same process number for all process — Jonathan Dursi
Most likely, your mpi4py is built against a different MPI implementation than the mpiexec/mpirun you're using to run the program. Does mpi4py.get_config() return compilers that are in the same path as your mpirun? — Jonathan Dursi
mpi4py.get_config() returns {'mpicxx': '/usr/bin/mpicxx', 'mpif77': '/usr/bin/mpif77', 'mpicc': '/usr/bin/mpicc', 'mpif90': '/usr/bin/mpif90'} But I tried the same on my localhost and get the same output (the same config), and at my localhost the code works. — ziuu
Ok - it looks like there's both MPICH and OpenMPI versions (or at least stubs of versions) installed for MPI. This is the canonical source of this problem - you have an mpi4py compiled with one implementation and are trying to run it with another. Try either explicitly running with both mpirun.mpich and mpirun.openmpi, or completely uninstall one or the other (including all libraries) — Jonathan Dursi
Yeah, the openmpi mpiexec version returned 0 and 1. The mpich mpiexec works great. I am really thankful for your help! — ziuu

user3148185 user3148185 · Accepted Answer · 2015-11-09T06:59:23

I had the same issue when running the python module emcee. It would give me an error:

"ValueError: Tried to create an MPI pool, but there was only one MPI process available. 
Need at least two."

The solution I found for my particular cluster was to use a different MPI. My code worked with intel-mpi and mpich2 but not openmpi. For this system, all I had to do was switch the MPI. In my PBS script I used module load mpich2 instead of module load openmpi. In this case mpiexec and mpirun worked correctly.

mpiexec and python mpi4py gives rank 0 and size 1

2 Answers