C++ why does the binary file read skips to the end

Question

I have a 178 MB FORTRAN unformatted binary file that I am reading with C++ and storing it in an Eigen Matrix. I have read the file with FORTRAN and MATLAB to confirm that I understand the values in the file. The first record in the file is the size of the matrix. After that each record begins with 3 integers. These integers contain the column and row where the data begins and the number of numbers to read. This is followed by the single precision numbers themselves.

Here is my code:

std::ifstream infile("MSFILE", std::ios::in | std::ios::binary);
if(!infile)
{
    std::cout << "Mode shape .f12 file (MSFILE) not found\n";
    exit(1);
}

int r_size1; // size of each record read
int r_size2; // size after reading for comparison

int cols, rows;
infile.read(reinterpret_cast<char *> (&r_size1), 4);
infile.read(reinterpret_cast<char *> (&cols), 4);
infile.read(reinterpret_cast<char *> (&rows), 4);
infile.read(reinterpret_cast<char *> (&r_size2), 4);

if (r_size1 != r_size2)
{
    std::cout << "Error reading MSFILE\n";
    exit(1);
}

MatrixXf data(rows, cols);
data.setZero();

int * vals = new int[3]; // vals holds i_col, i_row and i_word for each record
float * tempf; // pointer to array of floats that holds the data from the file

// Read in the record, and continue through the file
infile.read(reinterpret_cast<char *> (&r_size1), 4);

while (!infile.eof())
{
    infile.read(reinterpret_cast<char *> (vals), 12);
    tempf = new float[vals[2]];
    int buf_size = vals[2] * 4;
    // read the data from the file
    infile.read(reinterpret_cast<char *> (tempf), buf_size);

    // write the float array into the matrix
    data.col(vals[0] - 1) = Map<VectorXf>(tempf, rows);

    infile.read(reinterpret_cast<char *> (&r_size2), 4);

    if (r_size1 != r_size2)
    {
        std::cout << "Error reading MSFILE\n";
        exit(1);
    }

    delete tempf;

    // finish out by reading the next record size...this will also force EOF
    infile.read(reinterpret_cast<char *> (&r_size1), 4);
}

delete vals;

infile.close();

The problem is that when reading the float array the first time, the file goes to the end. I have used infile.tellg() after each infile.read to track what's happening. Everything moves the desired amount until the first instance of the float array. Following the first float array read the file goes to EOF. I expect the record to contain 26130 numbers. This is confirmed by vals[2]. buf_size is 104520 which is 4 * 26130 as expected. tempf is not fully populated either.

The constructor call should be std::ifstream infile("MSFILE", std::ios::in|std::ios::binary); (or you can skip the ios::in part and just use ios::binary). I have no idea how your call managed to compile. What's your compiler? — T.C.
@DaleWilson std::ifstream has no constructor taking three arguments. — T.C.
Try again: The Microsoft compiler has a three argument constructor for ifstream. The third argument controls file sharing: msdn.microsoft.com/en-us/library/zek0beca.aspx — Dale Wilson
@Matt, what do you get when you display the value of vals[2] after reading it from the file? — Dale Wilson
@T.C.: "17.6.5.5/2 An implementation may declare additional non-virtual member function signatures within a class ... by adding a member function signature for a member function name." The extension in question is conforming. — Igor Tandetnik

David K David K · Accepted Answer · 2014-09-10T23:09:14

If you open a binary file as a text file, that is, if you leave the | std::ios::binary out of the second parameter of the constructor of ifstream, then the read function may encounter a byte sequence that it interprets as an EOF, but which is actually just part of your binary data.

At least, that is a problem I have encountered when reading binary data on Microsoft platforms in the past, and the symptoms you describe (not as many bytes read as expected, program acts as if it reached EOF) are consistent with what I've observed when my program read a "false EOF" from a binary file while reading in text mode.

C++ why does the binary file read skips to the end

1 Answers