Fortran - writing to disk in columns by default?

Question

I am writing a simple code to output some large matrices to the disk that subsquently will be read in matlab.

I have written the following code, which exemplifies the writing for one such matrix. I am concerned with two things:

Efficiency in writing to disk (looking for something that is not too slow)
Easily being able to read it in matlab

a

PROGRAM WriteDisk

character(80)  :: filename = ' '    
INTEGER :: indt
INTEGER :: ind1, n1 = 161
INTEGER :: ind2, n2=20
INTEGER :: ind3, n3=2
INTEGER :: ind4, n4=2
INTEGER :: ind5, n5=21
INTEGER :: ind6, n6=20
INTEGER :: ind7, n7=2
INTEGER :: ind8, n8=2

INTEGER :: dummy    
REAL, ALLOCATABLE :: m1(:,:,:,:,:,:,:,:,:)
ALLOCATE(m1(2,n1,n2,n3,n4,n5,n6,n7,n8))   


dummy = 1

do ind8 = 1,n8
do ind7 = 1,n7
do ind6 = 1,n6    
do ind5 = 1,n5
do ind4 = 1,n4        
do ind3 = 1,n3        
do ind2 = 1,n2      
do ind1=1,n1    

    m1(2,ind1,ind2,ind3,ind4,ind5,ind6,ind7,ind8) = dummy
    dummy = dummy + 1

end do
end do
end do
end do
end do
end do
end do
end do

indt = 1
write(filename,'(a,i0,a)')'PF_m1_',indt,'.txt'    
OPEN(UNIT=25,FILE=filename,STATUS='replace',ACTION='write')
WRITE(25, *)  m1(2,:,:,:,:,:,:,:,:)
CLOSE(UNIT=25)



END PROGRAM

The program above writes the matrix m1 as a 4327680 x 5 . This makes it cumbersome to reshape it in matlab (although totally possible), as in Matlab I need to do the following:

Maybe I was not clear enough in my question. When Fortran writes that matrix it writes is with 4327680 rows and 5 columns. I.e. when I open it in matlab I have to do something like to get the matrix in the original format:

n1 = 161;
n2 = 20;
n3 = 2;
n4 = 2;
n5 = 21;
n6 = 20;
n7 = 2;
n8 = 2;

m1 = load('PF_m1_1.txt'); %This is a two dimensional matrix that needs to be transposed and reshaped TWICE to get the original matrix
m1 = m1';
m1 = m1(:);
m1 = reshape(m1, n1,n2,n3,n4,n5,n6,n7,n8)

Is there anyway to write it as a single vector with element with element m1(2,1,1,1,1,1,1,1,1) as first element, m1(2,2,1,1,1,1,1,1,1) as second element, ... , m1(2,end,end,end,end,end,end,end,end) as last element, etc?

Or anyway that I am not aware of, to quickly save it directly as .mat file?

If you care about efficiency, use unformatted I/O, otherwise you can't do much. Also, AFAIK, Matlab is column major as well as Fortran, can you explain why they are not compatible and why is the Fortran order cumbersome? Efficient I/O from Fortran was dealt with many times here, there is no point in repeating everything over and over and over. — Vladimir F
Notably, the order you ask for is EXACTLY the order Fortran uses by default. I don't understand your question. — Vladimir F
Fortran DOES write m1(2,1,1,1,1,1,1,1,1) as first element and m1(2,2,1,1,1,1,1,1,1) as second element!!! Really, what is the problem? — Vladimir F
Hi Vladimir. Maybe I was not clear enough. Please see the edits above. I have searched for unformatted I/O, but it seems that it is not possible to use filenames with that? Most examples I could find on stackoverflow or other forums about efficient I/O deal with 1 or 2 dimensional matrices and do not seem trivial to expand to higher dimensional spaces. — phdstudent
You CAN use filenames and it is TRIVIAL to expand it to higher dimensions. — Vladimir F

Vladimir F Vladimir F · Accepted Answer · 2017-08-10T14:17:36

"Is there anyway to write it as a single vector with element with element m1(2,1,1,1,1,1,1,1,1) as first element, m1(2,2,1,1,1,1,1,1,1) as second element, ... , m1(2,end,end,end,end,end,end,end,end) as last element, etc?"

Yes, this the default Fortran column major order. This is the order your file is already written. There is nothing you have to do.

"This makes it cumbersome to reshape it in Matlab (although totally possible), as in Matlab I need to do the following: m1 = reshape(m1, n1,n2,n3,n4,n5,n6,n7,n8)"

Reshape just updates the internal descriptor. It should be a very fast operation. Completely negligible. Even if it needed to shuffle the data, it would still be much quicker than reading from the hard-drive.

"I am concerned with two things: 1. Efficiency in writing to disk (looking for something that is not too slow)"

Use unformatted (also known as binary) I/O:

OPEN(UNIT=25,FILE=filename,ACCESS='stream',STATUS='replace',ACTION='write')
WRITE(25)  m1(2,:,:,:,:,:,:,:,:)
CLOSE(UNIT=25)

"2. Easily being able to read it in Matlab"

To read it from Matlab, learn how to read binary data from Read and write from/to a binary file in Matlab from the Matlab documentation https://www.mathworks.com/help/matlab/ref/fread.html and from loads of other resources.

Don't forget to tell Matlab the right dimensions. Or store the dimensions in the first bytes of the data file (a header).

Fortran - writing to disk in columns by default?

1 Answers