OOM with Eigen Large Sparse Matrix?

Question

I test SparseMatrix with eigen, it seem that when i new a 10*5billion SparseMatrix which include only a few nonzeros elements, it takes 50gb memory!

The demo code:

#include <Eigen/Core>
#include <Eigen/SparseCore>
#include <iostream>
using namespace std;

int main()
{
    typedef Eigen::SparseMatrix<double, 0, long int> SMatrixXd;
    cout << "tag1"<< endl;
    SMatrixXd sfeature(10, 5000000000);
    cout << "tag1 done" << endl;

    // load data
    typedef Eigen::Triplet<double, long int> T;
    std::vector<T> tripletList;
    tripletList.push_back(T(0, 1, 1.0));
    tripletList.push_back(T(0, 2, 2.0));
    tripletList.push_back(T(1, 3, 2.0));
    tripletList.push_back(T(2, 4, 2.0));
    tripletList.push_back(T(3, 5, 2.0));
    tripletList.push_back(T(4, 6, 2.0));
    tripletList.push_back(T(5, 7, 2.0));
    cout << "tag2 " << endl;
    sfeature.setFromTriplets(tripletList.begin(), tripletList.end());
    cout << "tag2 done" << endl;
    return 0;
}

ggael ggael · Accepted Answer · 2017-12-15T07:24:40

Look at how sparse matrices are stored to understand that in your case it needs to allocate an array of 5000000000 long int. In your case, simply use a RowMajor layout:

typedef Eigen::SparseMatrix<double, RowMajor, long int> SMatrixXd;

and the previous huge array will boil down to an array of 10 long int.

OOM with Eigen Large Sparse Matrix?

2 Answers