Dot product between scipy sparse matrix and numpy array give ValueError

Question

I'm trying to calculate the dot product between a scipy parse matrix and a numpy array.

First I was using a numpy matrix, which you can see in the following code:

def power_iteration(matrix, n):
    b_k = np.random.rand(matrix.shape[1])
    for _ in range(n):
        b_k = np.dot(matrix, b_k)

    return b_k

Here the matrix is a numpy matrix and no error occurs.

If you pass a scipy sparse matrix as a parameter, the following error occurs: ValueError: shapes (6762,6762) and (1,6762) not aligned: 6762 (dim 1) != 1 (dim 0)

I have changed

b_k = np.random.rand(matrix.shape[1])

into

b_k = np.random.rand(matrix.shape[1], 1)

which makes the dot product work, but doesn't return the right b_k shape. The shape I need is: (6762,)

Edit: so far I've tried to reshape like this:

b_k = np.reshape(b_k, (matrix.shape[1],))

but this transforms the shape (6762, 1) into (1, 6762), instead of (6762,)

Any tips? Thanks!

Practice this on small arrays and matrix, and one loop at a time. Pay attention to b_k is - is it a 1d ndarray, or a 2d sparse matrix? A sparse matrix is always 2d. What does a dot with such a matrix produce? — hpaulj
@hpaulj The b_k is a 2d matrix with a shape of (6762, 1) in this case. After each dot product b_k still has a shape of (6762, 1), which is logical. I'm trying to convert the (6762,1) shape into (6762,) using reshape. I'm basing myself on this answer: stackoverflow.com/questions/17869840/… But it doesn't seem to work — Lucas Belpaire
What is the type and dtype of b_k after one dot product? Is matrix is sparse, then matrix.dot(b_k), and np.dot(matrix.b_k) will produce different objects. Mixing sparse and dense array has to be done with care. And if the intermediate b_k is np.matrix or sparse matrix, it can't be reshaped to 1d. — hpaulj
I've spotted a problem, the type of matrix is already <class 'numpy.matrix'>. I had passed this as a parameter into the power iteration function: (d*matrix_s + (1 - d)*matrix_t), here d is a scalar and matrix_t is also a numpy.matrix. This means that before the first dot product, b_k is a numpy array and after the first it is a numpy matrix. — Lucas Belpaire

Anna Slastnikova Anna Slastnikova · Accepted Answer · 2019-04-20T19:01:16

Seems like in order to use np.dot on sparse matrix you'd need to convert it to dense matrix first with matrix.toarray(). Also see https://docs.scipy.org/doc/scipy/reference/sparse.html#matrix-vector-product

Dot product between scipy sparse matrix and numpy array give ValueError

1 Answers