How to determine the mean of the last e.g. 100 non-zero numbers in each column of a matrix in matlab

Question

I would like to calculate the mean of the last e.g. 3 non-zero numbers in each column of a matrix in matlab. The columns were filled with zeros at the end to create vectors of the same length.

example matrix:

A = [5 6 3 5 6 8 9;
     1 2 3 5 4 7 6;
     0 1 2 3 4 5 6; 
     0 0 1 2 3 4 5; 
     0 0 0 1 2 3 4; 
     0 0 0 0 2 3 4;
     0 0 0 0 2 3 4; 
     0 0 0 0 0 0 3]

Answer 1

There may be a more efficient solution but one way is to use sum to find the number of non-zero rows in a given column. Then grab average the values of A by looping through all columns with arrayfun and averaging the N rows before the zero in the column.

%// Number of elements to average
N = 3;

%// Last non-zero row in each column
lastrow = sum(A ~= 0, 1);

%// Ensure that we don't have any indices less than 1
startrow = max(lastrow - N + 1, 1);

%// Compute the mean for each column using the specified rows
means = arrayfun(@(k)mean(A(startrow(k):lastrow(k),k)), 1:size(A, 2));

Example

For your example data this would yield:

3.0000    3.0000    2.0000    2.0000    2.0000    3.0000    3.6667

UPDATE: An Alternative

An alternate approach would be to use convolution to actually solve this for you. You can compute a mean using a convolution kernel. If you want the mean of all 3-row combinations of an matrix, your kernel would be:

kernel = [1; 1; 1] ./ 3;

When convolved with the matrix of interest, this will compute the average of all 3-row combinations within the input matrix.

B = [1 2 3;
     4 5 6;
     7 8 9];

conv2(B, kernel)

    0.3333    0.6667    1.0000
    1.6667    2.3333    3.0000
    4.0000    5.0000    6.0000
    3.6667    4.3333    5.0000
    2.3333    2.6667    3.0000

In the example below, I do this and then only return the values at the regions we care about (where the average is only composed of the last N non-zeros in each column)

%// Find the last non-zero entry in each column
lastrow = sum(A ~= 0, 1);

%// Use convolution to compute the mean for every N rows
%// This will be applied to ALL of A
convmean = conv2(A, ones(N, 1)./N);

%// Select only the means that we care about
%// Because of the padding of CONV2, these will live at the rows
%// stored in LASTROW
means = convmean(sub2ind(size(convmean), lastrow, 1:size(A, 2)));

%// Now correct for cases where fewer than N samples were averaged
means = (means * N) ./ min(lastrow, N);

And the output, again, is the same

3.0000    3.0000    2.0000    2.0000    2.0000    3.0000    3.6667

Comparison

I ran a quick test script to compare the performance between these two methods. It is clear that the convolution-based approach is much faster.

Here is the full test script.

function benchmark()
    dims = round(linspace(1, 1000, 100));

    times1 = zeros(size(dims));
    times2 = zeros(size(dims));

    N = 3;

    for k = 1:numel(dims)
        A = triu(rand(dims(k)));
        times1(k) = timeit(@()test_arrayfun(N, A));
        A = triu(rand(dims(k)));
        times2(k) = timeit(@()test_convolution(N, A));
    end

    figure;
    plot(dims, times1);
    hold on
    plot(dims, times2);

    legend({'arrayfun', 'convolution'})
    xlabel('Dimension of A')
    ylabel('Execution Time (seconds)')
end

function test_arrayfun(N, A)
    %// Last non-zero row in each column
    lastrow = sum(A ~= 0, 1);

    %// Ensure that we don't have any indices less than 1
    startrow = max(lastrow - N + 1, 1);

    %// Compute the mean for each column using the specified rows
    means = arrayfun(@(k)mean(A(startrow(k):lastrow(k),k)), 1:size(A, 2));
end

function test_convolution(N, A)
    %// Find the last non-zero entry in each column
    lastrow = sum(A ~= 0, 1);

    %// Use convolution to compute the mean for every N rows
    %// This will be applied to ALL of A
    convmean = conv2(A, ones(N, 1)./N);

    %// Select only the means that we care about
    %// Because of the padding of CONV2, these will live at the rows
    %// stored in LASTROW
    means = convmean(sub2ind(size(convmean), lastrow, 1:size(A, 2)));

    %// Now correct for cases where fewer than N samples were averaged
    means = (means * N) ./ min(lastrow, N);
end