Eigen: different row-wise memory reservation for row major sparse matrix

0
votes

Is it at all possible in the Eigen library to reserve space for a row-major sparse matrix row by row (different for each row)?

I am trying to optimize the memory consumtion of filling a pretty big sparse matrix (~70mio x 70mio with ~2 billion nnz is the biggest I could reach but I'd like go even further). To clearify the way I went:

First I was using the recommended setFromTriplets which is probably the fastest way filling the matrix, but checking on the memory consumption I found a peak about double the average memory at the point where I used this function, which makes sense since I - at some point - store the elments in both, the matrix and the vector of Triplets until the vector goes out of scope.

Using insert() improved the maximum memory consumption a lot obviously. Though, I still got a peak due to reallocation. I then also used reserve() so that there is no (or less) reallocation. It also lowers the peak quite a bit, but it is not completely gone, again due to some reallocation (if valgrind is correct). Since most of my rows have a lower NNZ than the max I get quite a few empty allocated entries in the storage which increases the average memory consumption. Using makeCompressed lowers the average again but obviously makes the peak higher again as well because more reallocation has to be done when calling it.

Why I asked the question above now is: I can calculate the NNZ for each row up front and also sort them so that I actually should be able to completely optimize this having a compressed matrix without any empty allocation and with no reallocation peak if I could reserve a different number of NNZ for each row.

I would be greatful if anybody would let me know if that is possible or not in Eigen and if not: do you know any library that supports it?

Thanks a bunch!

1
c++matrixsparse-matrixeigen

1 Answers

1
votes

Yes, it is possible, and you perhaps could have found that function in the docu:

template<class SizesType> void SparseMatrix::reserve(const SizesType & reserveSizes);

Note that SizesType can be, e.g., an std::vector (or std::deque) or an Eigen::VectorXi.

Also, if you are able to insert the elements to your matrix in order, you may also have a look at the (internal) function insertBack.