Working with Big Matrices in Matlab

Question

Preamble:

My framework is Matlab. I have a very large data matrix M (size(M) = 30 20 30 20 51 300 ) and I need to manipulate this matrix (to calculate some correlations, mean, shift it circularly, interpolate it and so on).

!Important! : most of the elements of this matrix are zeros or ones !!

My question: Since it is very time consuming to work with such a huge matrix, is it possible to perform the same manipulations, but on the sparse form of this matrix? Of course, one should not loose any information about zeros or ones (for example, for calculations of averages or correlations between different elements).

Is there any other way to handle such matrices? (huge and mostly 0's and 1's)

Thanks in advance!

Answer 1

You can use sparse matrices.

The only issue with sparse matrices is, that they only come in two dimensions, so the straight-forward way to represent your matrix would be to wrap it into a sparse matrix, of size [N 1] where N = prod([ 30 20 30 20 51 300]) in your case. I've done this for N-dimension histograms (which sounds similar to your application) and it works fine.

You'll lose the possibility to use all the smart indexing though. So using mean/sum etc. on single dimensions will become somewhat more complicated since you'll have to convert subscript-indices to linear indices and vice versa.

For that, you should have a look at sub2ind and ind2sub. (Sounds like a fun project on wrapping the builtin sparse matrix into a n-dimensional sparse matrix...)