tensorflow: how to perform element-wise multiplication between two sparse matrix

Question

I have two sparse matrices declared using the tf.sparse_placeholder. I need to perform the element-wise multiplication between the two matrices. But I cannot find such an implementation in tensorflow. The most related function is tf.sparse_tensor_dense_matmul, but this is a function performing matrix multiplication between one sparse matrix and one dense matrix.

What I hope to find is to performing element-wise multiplication between two sparse matrices. Is there any implementation of this in tensorflow?

I show the following example of performing multiplication between dense matrices. I'm looking forward to seeing a solution.

import tensorflow as tf
import numpy as np

# Element-wise multiplication, two dense matrices
A = tf.placeholder(tf.float32, shape=(100, 100))
B = tf.placeholder(tf.float32, shape=(100, 100))
C = tf.multiply(A, B)
sess = tf.InteractiveSession()
RandA = np.random.rand(100, 100)
RandB = np.random.rand(100, 100)
print sess.run(C, feed_dict={A: RandA, B: RandB})

# matrix multiplication, A is sparse and B is dense
A = tf.sparse_placeholder(tf.float32)
B = tf.placeholder(tf.float32, shape=(5,5))
C = tf.sparse_tensor_dense_matmul(A, B)
sess = tf.InteractiveSession()
indices = np.array([[3, 2], [1, 2]], dtype=np.int64)
values = np.array([1.0, 2.0], dtype=np.float32)
shape = np.array([5,5], dtype=np.int64)
Sparse_A = tf.SparseTensorValue(indices, values, shape)
RandB = np.ones((5, 5))
print sess.run(C, feed_dict={A: Sparse_A, B: RandB})

Thank you very much!!!

Answer 1

you can use tf.matmul or tf.sparse_matmulfor sparse matrices also; setting a_is_sparse and b_is_sparse as True.

matmul(
    a,
    b,
    transpose_a=False,
    transpose_b=False,
    adjoint_a=False,
    adjoint_b=False,
    a_is_sparse=False,
    b_is_sparse=False,
    name=None
)

For element-wise multiplication, one workaround is to use tf.sparse_to_dense for converting sparse tensor to dense representation and using tf.multiply for element-wise multiplication

Answer 2

TensorFlow currently has no sparse-sparse element-wise multiplication operation.

We don't plan to add support for this currently, but contributions are definitely welcome! Feel free to create a github issue here: https://github.com/tensorflow/tensorflow/issues/new and perhaps you or someone in the community can pick it up :)

Thanks!

Answer 3

Solution from another post works.

https://stackoverflow.com/a/45103767/2415428

Use the __mul__ to perform the element-wise multiplication.

TF2.1 ref: https://www.tensorflow.org/api_docs/python/tf/sparse/SparseTensor#mul

Answer 4

I'm using Tensorflow 2.4.1.

Here's my workaround to multiply two sparse tensor element-wise:

def sparse_element_wise_mul(a: tf.SparseTensor, b: tf.SparseTensor):
    a_plus_b = tf.sparse.add(a, b)
    a_plus_b_square = tf.square(a_plus_b)
    minus_a_square = tf.negative(tf.square(a))
    minus_b_square = tf.negative(tf.square(b))
    _2ab = tf.sparse.add(
        tf.sparse.add(
            a_plus_b_square,
            minus_a_square
        ),
        minus_b_square
    )
    ab = tf.sparse.map_values(tf.multiply, _2ab, 0.5)
    return ab

Here's some simple explanation:

Given that

(a+b)^2 = a^2 + 2a*b + b^2

we can calculate a*b by

a*b = ((a+b)^2 - a^2 - b^2) / 2

It seems the gradient can be calculated correctly with such a workaround.