Read from standard input with all MPI processes

Question

So far I've been using OPEN(fid, FILE='IN', ...) and it seems that all MPI processes read the same file IN without interfering with each other.

Furthermore, in order to allow the input file being chosen among several, I simply made the IN file a symbolic link pointing to the desired input. This means that when I want to change the input file I have to run ln -sf desidered-input IN before running the program (mpirun -n $np ./program).

I'd really like to be able to run the progam as mpirun -n $np ./program < input-file. To do so I removed the OPEN statement, and the corresponding CLOSE statement, and changed all READ(fid,*) statements to READ(INPUT_UNIT,*) (I'm using ISO_FORTRAN_ENV module).

But, after all edits, I've realized that only one process (always 0, I noticed) reads from it, since all others reach EOF immediately. Here is a MWE, using OpenMPI 2.0.1.

! cat main.f90
program main
    use, intrinsic :: iso_fortran_env
    use mpi
    implicit none
    integer :: myid, x, ierr, stat
    x = 12
    call mpi_init(ierr)
    call mpi_comm_rank(mpi_comm_world, myid, ierr)
    read(input_unit,*, iostat=stat) x
    if (is_iostat_end(stat)) write(output_unit,*) myid, "I'm out"
    if (.not. is_iostat_end(stat)) write(output_unit,*) myid, "I'm in", myid, x
    call mpi_finalize(ierr)
end program main

that can be compiled with mpifort -o main main.f90, run with mpirun -np 4 ./main, and which results in this output

               1 I'm out
               2 I'm out
               3 I'm out
    17 this is my input from keyboard
               0 I'm in           0          17

I know that MPI has proper routines to perform parallel I/O, but I've found nothing about reading from standard input.

Which MPI implementation do you use? It is quite common for a single process only to get the standard input of mpirun (and sometimes controllable). — francescalus
using mpirun with a redirection is discouraged, and should only be used when there is no other alternative (for example if the input file is only available on the node invoking mpirun). The reason is it involves out of band traffic which is suboptimal. This use case is also lightly tested and some bugs have been reported. Also, note Open MPI 2.0.x series is no more supported, and you should upgrade to 3.0.x or at least 2.1.x series. — Gilles Gouaillardet

francescalus francescalus · Accepted Answer · 2018-02-27T20:14:07

You are seeing the expected behaviour with OpenMPI. By default, mpirun

directs UNIX standard input to /dev/null on all processes except the MPI_COMM_WORLD rank 0 process. The MPI_COMM_WORLD rank 0 process inherits standard input from mpirun.

The option --stdin can be used to direct standard input to another process, but not to direct to all.

One could also note that the behaviour of redirection of standard input isn't consistent across MPI implementations (the notion isn't specified by the MPI standard). For example, using Intel MPI there is the -s option to that mpirun. mpirun -np 4 -s all ./main does allow all processes access to mpirun's standard input. There's also no guarantee that processes without that redirection will fail, rather than wait, to read.

Read from standard input with all MPI processes

1 Answers