MPI Communication Pattern

0
votes

I was wondering if there was a smart way to do this. Let's say I have three nodes, 0, 1, 2. And let's say each node has an array, a0, a1, a2. If the contents of each node is something like

a0 = {0, 1, 2, 1}
a1 = {1, 2, 2, 0}
a2 = {0, 0, 1, 2}

Is there a clever communication pattern so to move each number to it's corresponding node, i.e.

a0 = {0, 0, 0, 0}
a1 = {1, 1, 1, 1}
a2 = {2, 2, 2, 2}

The approach I have in mind, would involve sorting and temporary buffers, but I was wondering if there was a smarter way?

1
mpi
I don't understand--can't you just fill a0 with 0, a1 with 1, etc.? Presumably there's more to your problem than is shown by your example. It's also not clear what you mean by "three nodes, 0, 1, 2" and how that relates to the MPI "nodes" (computers) or the 0,1,2 values in your arrays. – John Zwinck
Thanks John. I guess Zulan is on the right path. The three nodes are separate computers. Yes there is more to the problem. I only chose the values in the array in an attempt to make my intention clearer. I guess I failed at that xp – inJeans

1 Answers

2
votes

You can use MPI_Alltoallv for this in the following way:

  1. Sort the local_data (a) by corresponding node of each element in increasing order.
  2. Create a send_displacements array such that send_displacements[r] indicates the index of the first element in the local_data that refers to node r.
  3. Create a send_counts array such that send_counts[r] equals the number of elements in local_data that correspond to node r. This can be computed send_counts[r] = send_displacements[r+1] - send_displacements[r] except for the last rank.
  4. MPI_Alltoall(send_counts, 1, MPI_INT, recv_counts, 1, MPI_INT, comm)
  5. Compute recv_displacements such that recv_displacements[r] = sum(recv_counts[r'] for all r' < r).
  6. Prepare a recv_data with sum(recv_counts) elements.
  7. MPI_Alltoallv(local_data, send_counts, send_displacements, MPI_INT, recv_data, recv_counts, recv_displacements, MPI_INT, comm)