MATLAB: vectors of different length

Question

I want to create a MATLAB function to import data from files in another directory and fit them to a given model, but because the data need to be filtered (there's "thrash" data in different places in the files, eg. measurements of nothing before the analyzed motion starts).

So the vectors that contain the data used to fit end up having different lengths and so I can't return them in a matrix (eg. x in my function below). How can I solve this?

I have a lot of datafiles so I don't want to use a "manual" method. My function is below. All and suggestions are welcome.

datafit.m

function [p, x, y_c, y_func] = datafit(pattern, xcol, ycol, xfilter, calib, p_calib,    func, p_0, nhl)

    datafiles = dir(pattern);
    path = fileparts(pattern);
    p = NaN(length(datafiles));
    y_func = [];
    for i = 1:length(datafiles)
        exist(strcat(path, '/', datafiles(i).name));
        filename = datafiles(i).name;
        data = importdata(strcat(path, '/', datafiles(i).name), '\t', nhl);
        filedata = data.data/1e3;
        xdata = filedata(:,xcol);
        ydata = filedata(:,ycol);
        filter = filedata(:,xcol) > xfilter(i);
        x(i,:) = xdata(filter);
        y(i,:) = ydata(filter);
        y_c(i,:) = calib(y(i,:), p_calib);
        error = @(par) sum(power(y_c(i,:) - func(x(i,:), par),2));
        p(i,:) = fminsearch(error, p_0);
        y_func = [y_func; func(x(i,:), p(i,:))];
    end
end

sample data: http://hastebin.com/mokocixeda.md

Answer 1

There are two strategies I can think of:

• I would return the data in a vector of cells instead, where the individual cells store vectors of different lengths. You can access data the same way as arrays, but use curly braces: Say c{1}=[1 2 3], c{2}=[1 2 10 8 5] c{3} = [ ].
• You can also filter the trash data upon reading a line, if that makes your vectors have the same length.

Answer 2

If memory is not an major issue, try filling up the vectors with distinct values, such as NaN or Inf - anything, that is not found in your measurements based on their physical context. You might need to identify the longest data-set before you allocate memory for your matrices (*). This way, you can use equally sized matrices and easily ignore the "empty data" later on.

(*) Idea ... allocate memory based on the size of the largest file first. Fill it up with e.g. NaN's

matrix = zeros(length(datafiles), longest_file_line_number) .* NaN;

Then run your function. Determine the length of the longest consecutive set of data.

new_max = length(xdata(filter));
if new_max > old_max
    old_max = new_max;
end
matrix(i, length(xdata(filter))) = xdata(filter);

Crop your matrix accordingly, before the function returns it ...

matrix = matrix(:, 1:old_max);