How to implement simple parallel computation on different cores using MPI in Python

Question

I want to implement a simple computation task in parallel. Lets say I have two arrays including 2 components in each and I want to sum the components of these array one by one and store them in a new array. There are 4 combinations of the components (2x2). A simple code could be written in serial that uses only 1 core and the summing operation is implemented 4 times on that core. Here is the code:

a = [1 , 5]
b = [10 , 20]

d = []

for i in range(2):
    for j in range(2):

        c = a[i] + b[j]
        d.append(c)

print (d)

Now I want to use MPI to run the above code in parallel to use 4 different cores on my PC to make it faster. With that being said I want each combination to be implemented on the assigned core (e.g. 4 summing operations on 4 different cores). Here is how I can import MPI:

from mpi4py import MPI
mpi_comm = MPI.COMM_WORLD
rank_process = mpi_comm.rank

I have never used parallel computations so it looks a bit confusing to me. I was wondering if someone could help me with this. Thanks in advance for your time.

Ed Smith Ed Smith · Accepted Answer · 2019-02-04T17:06:38

You can use a Create_cart to assign MPI processes to parts of a matrix so they can be given indices i and j as in your serial example. Here is the solution,

from mpi4py import MPI
mpi_comm = MPI.COMM_WORLD
rank = mpi_comm.rank
root = 0

#Define data
a = [1 , 5]
b = [10 , 20]
d = []

#Print serial solution
if rank == 0:
    for i in range(2):
        for j in range(2):
            c = a[i] + b[j]
            d.append(c) 

    print("Serial soln = ", d)

#Split domain into 2 by 2 comm and run an element on each process
cart_comm = mpi_comm.Create_cart([2, 2])
i, j = cart_comm.Get_coords(rank)
d = a[i] + b[j]

#Print solns on each process, note this will be jumbled
# as all print as soon as they get here
print("Parallel soln = ", d)

# Better to gather and print on root
ds = mpi_comm.gather(d, root=root)
if rank == root:
    print("Parallel soln gathered = ", ds)

where you get something like,

('Serial soln = ', [11, 21, 15, 25])
('Parallel ('Parallel soln = '('Parallel soln = 'soln = ', 11)
, 21)
('Parallel soln = ', 15)
, 25)
('Parallel soln gathered = ', [11, 21, 15, 25])

where the parallel output is jumbled. Note you need to run with mpiexec as follows,

mpiexec -n 4 python script_name.py

where script_name.py is your script name.

I'm not sure this is a very good example of how you'd use MPI. It is worth reading up on MPI in general and look at some of the canonical examples. In particular, note that as each processes is independent with its own data, you should be working on problems where you can split it up into parts. In your example, all of the lists a and b are defined individually on every process and each processes only uses a small part of them. The only difference between each processes is the rank (0 to 3) and later the 2D cartesian indixes from create_cart, which determines which part of the "global" array they use.

A better solution, closer to how you'd use this in practice may be to scatter parts of a large matrix to many processes, do some work using that part of the matrix and gather back the solution to get the complete matrix (again, see examples which cover this sort of thing).

How to implement simple parallel computation on different cores using MPI in Python

1 Answers