Set multiple rows and columns of large scipy sparse matrix to 0s

Question

I'm trying to "translate" some Matlab code into Python and there is one line in Matlab that sets multiple rows in a sparse matrix to 0s:

Ks(idx,:)=0; %no affinity for W inside fs

where Ks is the sparse matrix (which is symmetrical and with pretty big size), and idx is an 1D array denoting the row indices to do the changes and it's also quite big in size. And in the next line it also changes those columns to 0s, so Ks is still symmetric:

Ks(:,idx)=0;

Doing the similar thing in Python (Ks[idx,:]=0) seems to only work for small matrices, when it gets big I got MemoryError. Currently my Ks is a csr matrix, converting it to lil and do that is super slow.

I'm not quite familiar with sparse matrices, I know that in Python there are more than 1 type (e.g. csr, csc, lil etc.), but in the Matlab code there aren't such distinctions, I only found a function call of sparse(). So what's my best bet in this situation?

Thanks in advance.

Siva-Sg Siva-Sg · Accepted Answer · 2018-05-08T05:48:14

One way to speed up is , instead of setting the sparse matrix elements to zero , first set the elements of numpy nd array to zero , and then convert to sparse matrix. I got a speed boost of more than 10 times in the below example.

import numpy as np
import scipy.sparse as sps
np.random.seed(20)
mat = np.random.randint(-2000,2000,size=(1000,1000))
sym_mat = (mat + mat.T)/2

zero_rows =  np.random.randint(0,999,(900,))


%%timeit
sparse = sps.csr_matrix(sym_mat)
sparse[zero_rows,:] = 0
sparse[:,zero_rows] = 0

/usr/local/lib/python3.6/dist-packages/scipy/sparse/compressed.py:774: SparseEfficiencyWarning: Changing the sparsity structure of a csr_matrix is expensive. lil_matrix is more efficient.
  SparseEfficiencyWarning)
1 loop, best of 3: 206 ms per loop

%%timeit
sym_mat[zero_rows,:] = 0
sym_mat[:,zero_rows] = 0
sparse1 = sps.csr_matrix(sym_mat)

100 loops, best of 3: 18.9 ms per loop

Set multiple rows and columns of large scipy sparse matrix to 0s

1 Answers