Using fscanf to read an input file fails mid-line

Question

Long time, first time. I've done a ton of digging on this to no avail, so I'll get right to it...

I'm working in C for a physics research project. I am trying to read in a file of input values... each line contains 12 values of type double that are space-delimited, and the input file is 1006 lines long. Since the file input is well-known, I have elected to use fscanf() to ingest the input, perform an operation to reorder it, and then output it into another file that will be more suitable for gnuplot use. Here is the line prior to the problem line and the problem line in question:

2.250000000000000 0.500000000000000 2.668878914693362 0.081121085306632 2.668879345525446 0.081120608880718 0.081120609109235 2.668879508723290 -1.139145600698256 -0.478208465494011 -0.476544273039587 -0.184392862164658
2.250000000000000 0.550000000000000 2.723599351123168 0.076400593002322 2.723599435547186 0.076400582995125 0.076400821024960 2.723599264264542 -0.996035795911154 -0.408011755823990 -0.409827430433329 -0.178196609653836

All I am trying to do at the moment is read the file and output only what I am concerned with before adding more logic, but fscanf() is being problematic. The relevant code in question follows (debug statements included):

int readEOF = 0;
double maxThreePhaseParticleCount = 0.0;
double particlesATotal = 0.0;
double particlesBTotal = 0.0;
double rhoA1 = 0;
double rhoA2 = 0;
double rhoA3 = 0;
double rhoB1 = 0;
double rhoB2 = 0;
double rhoB3 = 0;

FILE * two_phase_coords;
char two_phase_coords_name[255];
sprintf(two_phase_coords_name,"~/threePhaseDiagram-densities-twoPhases_tcA%f_tcB%f_aA%f_aAB%f_aB%f.dat", tcA, tcB, aA, aAB, aB);
two_phase_coords = fopen(two_phase_coords_name, "r");

readEOF = fscanf(two_phase_coords, "%lf %lf %lf %lf %lf %lf %lf %lf %*lf %*lf %*lf %*lf", &particlesATotal, &particlesBTotal, &rhoA1, &rhoB1, &rhoA2, &rhoB2, &rhoA3, &rhoB3);
while (readEOF != EOF) {
    readEOF = fscanf(two_phase_coords, "%lf %lf %lf %lf %lf %lf %lf %lf %*lf %*lf %*lf %*lf", &particlesATotal, &particlesBTotal, &rhoA1, &rhoB1, &rhoA2, &rhoB2, &rhoA3, &rhoB3);
    printf("just read %i...\n%.15f %.15f %.15f %.15f %.15f %.15f %.15f %.15f %.15f\n", readEOF, particlesATotal, particlesBTotal, maxThreePhaseParticleCount, rhoA1, rhoB1, rhoA2, rhoB2, rhoA3, rhoB3);
}

The 6th field of the problem line (line 988 out of 1006) is where things fall on its face... instead of reading out 0.076400582995125, the value 0.000000000000000 is read instead... readEOF returns 6 instead of the expected 8, and fscanf() fails/returns EOF on the next loop iteration.

I'm baffled. Things I have tried...

Lots of Googling.
Changing from fscanf() to fgets()/sscanf()... failure occurs at exactly the same location, and the remaining 18 lines are still NOT processed.
Creating dummy input files of 1000+ carbon copies of both the line before the problem line and the problem line... processing of each file is complete and without error.
Examining the input file with a hex editor... everything before and after the problem line looks like standard/expected ASCII to me.
Reading all 12 values on each input line into separate variables (i.e. not using the ignore character in %*lf in fscanf()).
Lots of Googling.

I would appreciate any and all help... it's been a very long time since I've been a C guru. And since this is my very first SO post, apologies in advance if I've accidentally stomped on any community expectations/etiquette.

Thank you for all of your help!!

sprintf(two_phase_coords_name,"~/threePhaseDiagra ... I dont think that fopen() understands tilde expansion. Also: you should test the result you get from fopen. and .. dont repeat yourself ... And: maybe there is a strange character at the particular position. — wildplasser
Suggest you provide a link to the exact input file. Normally the input should be provided inline in the question. But since it is so long in this case and the exact file is critical, an exception should be acceptable. Also please provide an minimal reproducible example so we can copy and run exactly as shown to reproduce the problem. — kaylum
The input file will have a glitch in it. So there's nothing you can do other than repair it, which might not be possible depending on where the data comes from and how it is broken. — Malcolm McLean
Note that the shell expands ~ in a file name, but fopen() does not. You should always test the return value from fopen() — things may have happened to the file so that the open fails. — Jonathan Leffler

hotquakeroats hotquakeroats · Accepted Answer · 2017-05-22T23:04:26

So I'm not sure how often this happens, but naturally right after I post this question I manage to track down what was happening... the function in question is part of a larger environment, and the call to this function was happening before the file pointer to the input file was closed (undefined behavior). I want to down vote myself.

Thank you for chiming in everyone!

Using fscanf to read an input file fails mid-line

1 Answers