Trying to parse a fairly complex text file

0
votes

I have multiple text file, each is a result of a simulation. The content of the file is as follow:

Parameter_1 = value
Parameter_2 = value
.....

Test 1
Min: value
Max: value
Average: value

Test 2
Min: value
Max: value
Average: value

Each file contains the same type of parameters with different values, and of course the tests values are different as well.

I need to be able to import this data to Matlab. What I want to do in Matlab is to be able to create charts of parameters (x-axis) and test results. For example, a chart of Test 1 Min values when Parameter_1 changes means selecting n files where only Parameter_1 differs and compare the Test 1 Min results.

Here's my question: how should I organize that data in my text file to make it easy to import to Matlab? I'm new to Matlab and so I'm don't know what's the best way.

Any ideas that can help me get started would be great. Thanks!

3
matlabtext-parsing
What is the variability between the text files? More specifically, do all files have the same number of Parameter_Ns? Do they all have the same number of Tests? Do all tests have just 3 values Min, Max, Average? - slayton
Different files has different parameter values and different tests values. I've edited my question. - Yotam
you could try an excel or a csv file - Gir

3 Answers

2
votes

"Best way to organize data" is a very contentious question. If you ask 10 people you will get 11 different answers. It often depends on the data and the functionality you have available to you for importing and exporting the data.

That being said, Matlab excels (ha, no pun intended) at importing purely numerical data. If you can organize your file to be composed of only numbers, then a quick 'load', 'dlmread', or 'csvread' command will import them. Including textual data makes things a fair bit more complex.

For example, if you files are very consistent and you could organize the files like this:

Param1Value,Param2Value,Param3Value
1,Test1min,test1max,test1average
2,Test2min,test2max,test2average

where all the text in the example are simply numerical values (integers or floats), it would be very easy to import into Matlab. You would know the first row contains your parameter values

data = csvread('input.csv');
params = data(1,:);

And you could pull out quickly the test numbers, min, max, and average values.

tests = data(2:end,1);
mins = data(2:end,2);
maxs = data(2:end,3);
avgs = data(2:end,4);

But this all hinges on how flexible you are on the output side.

3
votes

There's nothing fundamentally wrong with your file. You will need to write a parser. But fear not it's not too hard.

Regexp is very useful for this. I takes a bit reading to get the hang of it - but it's incredible powerful.

I would use code like this:

fid = fopen('myfile.txt');
result = {};
result_entry=[];
while 1
    tline = fgetl(fid);
    if ~ischar(tline), break, end

    r = regexp(tline,'^(?<key>\w+)\W*=\W*(?<value>.*?)\W*$','names');
    if ~isempty(r)
    parameter_list.(r.key) = r.value;
    continue
    end

    % does a new entry start?
    r = regexp(tline,'^Test\W+(?<num>\d+)\W*$','names');
    if ~isempty(r)
        result = [result,{result_entry}];
        result_entry = struct('TestNumber',r.num);
        continue
    end

    r = regexp(tline,'^(?<key>\w+)\W*:\W*(?<value>.*?)\W*$','names');
    if ~isempty(r)
        result_entry.(r.key) = r.value;
    end
end
fclose(fid);
0
votes

Just an idea - analogous to JSON you could make your saves as valid matlab m files. That way you can have all structure features matlab offers an still rather quick reading.