Read and reshape many matrices from many large .mat files efficiently

Question

I have nearly a thousand .mat files with weird names:

My first problem is running a foor loop to open files that are named like this:

exp_trial1_0001
exp_trial1_0002
...
exp_trial1_1000

The fact that it's 4-digits makes it difficult for me. This won't work:

load(['exp_trial_', num2str(%04i), '.mat'])

There are 1000 .mat files corresponding to 1000 trials. In each one of these .mat files there's a matrix A. Matrix A has 500 rows. I want to take out all the row 1s (and 2s, 3s, ... 500) for each .mat file ("trial") and put them in a separate matrix.

I cannot load all these .mat files simultaneously and then do this because I run out of memory. I'd like to know what the most efficient way to do this is.

Thank you very much!

Luis Mendo Luis Mendo · Accepted Answer · 2013-12-16T19:03:23

I suggest something along these lines:

N = 100; %// Number of columns of the matrix "A" in the files. Set as needed. 
F = 1000; %// Number of files
rows = [1 2 3]; %// rows you want to extract from the files
result = NaN(numel(rows),N,F); %// preallocate
for ii = 1:F %// iterate over all files
    load(['exp_trial1_' sprintf('%04d',ii) '.mat'], 'A'); %// load matrix A only
    result(:,:,ii) = A(rows,:); %// store desired rows from each file
end

Read and reshape many matrices from many large .mat files efficiently

1 Answers