Write and read access=stream files in fortran

Question

I have a shell script from which I pass a binary file to a fortran program such that

Mth=$1
loop=1
it=1
while test $it -le 12
do 
    Mth=`expr $Mth + $loop`
    file="DataFile"$Mth".bin"
    ./fort_exe ${Yr} ${nt} ${it} 

# Increment loop
it=`expr $it + 1`
done

This script is used to pass 12 files within a do loop to the fortran program. In the fortran program, I read the binary file passed from the shell script and I am trying to write a 2nd file which would compile in a single file all the data that was read from the consecutive files e.g.

 !Open binary file passed from shell script
 open(1,file='Datafile'//TRIM{Mth)//.bin',action='read',form='unformatted',access='direct', &
 recl=4*x*y, status='old')

! Open write file for t 1. The status is different in t 1 and t > 1 so I open it twice: I guess there is a more elegant way to do this...
 open(2,file='Newfile.bin',action='write',form='unformatted', &
 access='stream', position='append', status='replace')
 irec = 0

 do t = 1, nt
 ! Read input file
  irec = irec + 1
  read(1,rec=irec) val(:,:)

 ! write output file
 irecW= irec + (imonth-1)*nt

 if ( t .eq. 1) write(2,pos=irecW) val(:,:)

 ! Close file after t = 1, update the status to old and reopen. 
 if ( t .eq. 2) then
    close (2)
    open(2,file='Newfile.bin',action='write',form='unformatted', &
         access='stream', position='append',status='old')
  endif

  if ( t .ge. 2) write(2,pos=irecW) val(:,:)

  enddo

I can read the binary data from the first file no problem but when I try and read from another program the binary data from the file that I wrote in the first program such that

 open(1,file='Newfile.bin',action='read',form='unformatted', &
 access='stream', status='old')

 irec=0
 do t = 1, nt
! Read input file
     irec = irec + 1
     read(1,pos=irec) val(:,:)
     write(*,*) val(:,:)
 enddo

val(:,:) is nothing but a list of zeros. This is the first time I use access=stream which I believe is the only way I can use position='append'. I have tried compiling with gfortran and ifort but I do not get any error messages.

Does anyone have any idea why this is happening?

sigma sigma · Accepted Answer · 2013-01-15T17:33:15

Firstly, I do not think you need to close and reopen your output file as you are doing. The status specifier is only relevant to the open statement in which it appears: replace will delete Newfile.bin if it exists at that time, before opening a new file with the same name. The status is implicitly changed to old, but this does not affect any operations done to the file.

However, since your Fortran code does not know you run it 12 times, you should have a way of making sure the file is only replaced the first time and opened as old afterwards; otherwise, Newfile.bin will only contain the information from the last file processed.

As for reading in the wrong values, this most likely occurs because of the difference between direct access (where you can choose a record length) and stream access (where you cannot). With stream access, data is stored as a sequence of "file storage units". Their size is in general compiler-dependent, but is available through the module iso_fortran_env as file_storage_size; it is usually 8 bits. This means that each entry will usually occupy multiple storage units, so you have to take care that a read or write with the pos = specifier does not access the wrong storage units.

Edit:
Some example code writing and reading with stream access:

program stream
  use, intrinsic :: iso_fortran_env
  implicit none

  integer :: i, offset
  real(real32), dimension(4,6) :: val, nval

  open(unit=2, file='Newfile.bin', action='readwrite', form='unformatted', &
     access='stream', status='replace')

  do i = 1,2
    call random_number(val)
    write(2) val
  enddo

  ! The file now contains two sequences of 24 reals, each element of which
  ! occupies the following number of storage units:
  offset = storage_size(val) / file_storage_size

  ! Retrieve the second sequence and compare:
  read(2, pos = 1 + offset*size(val)) nval
  print*, all(nval == val)

  close(2)
end program

The value true should be printed to the screen.

Note also that it's not strictly necessary to specify a pos while writing your data to the file, because the file will automatically be positioned beyond the last record read or written.

That said, direct or stream access is most beneficial if you need to access the data in a non-sequential manner. If you only need to combine input files into one, it could be easier to write the output file with sequential access, for which you can also specify recl and position = 'append'.

Write and read access=stream files in fortran

2 Answers