tensorflow matrix multiplication

Question

So, i want to multiply a matrix with a matrix. When I try an array with a matrix, it works:

import tensorflow as tf

x = tf.placeholder(tf.float32, [None, 3])
W = tf.Variable(tf.ones([3, 3]))
y = tf.matmul(x, W)

with tf.Session() as sess:
    sess.run(tf.global_variables_initializer())
    curr_y = sess.run(y, feed_dict={x: [[1,2,3],[0,4,5]]})
    print curr_y

So the array has the batch size 2 and shape 3x1. So I can multiply the matrix with shape 3x3 with the array 3x1. But when I have again a matrix with the shape 3x3, but this time a matrix and not an array with the shape 3x2, with batch size 2, its not working.

But if I try to multiply a matrix with a matrix. It doesn't work.

import tensorflow as tf
x = tf.placeholder(tf.float32, [None, 3,3])
W = tf.Variable(tf.ones([3, 3]))
y = tf.matmul(x, W)

with tf.Session() as sess:
    sess.run(tf.global_variables_initializer())
    curr_y = sess.run(y, feed_dict={x: [[[1,2,3],[1,2,3]],[[1,1,4],[0,4,5]]]})
    print curr_y

ValueError: Shape must be rank 2 but is rank 3 for 'MatMul' (op: 'MatMul') with input shapes: [?,3,3], [3,3].

########EDIT

Sorry, what I want to do, is, to matmul a matrix to a batch of matrix or arrays. So I dont want to do

y = tf.matmul(x, W)

actually, I want to do

y = tf.matmul(W, x)

Answer 1

Your input to tensor 'x' has a shape (2, 2, 3). You're trying to do matrix multiplication of (2, 2, 3) and (3, 3). they don't have the same rank, and that's the reason for the error.

from Tensorflow official site: https://www.tensorflow.org/api_docs/python/tf/matmul

Args:

a: Tensor of type float16, float32, float64, int32, complex64, complex128 and rank > 1.
b: Tensor with same type and rank as a.

Answer 2

When you do matrices multiplication, the shape of the matrices need to follow the rule (a, b) * (b, c) = (a, c)

Keep in mind the shape of W as you defined is (3, 3).

This feed_dict={x: [[1,2,3],[0,4,5]]} is a 2D array, the shape of it is (2, 3)

In [67]: x = [[1, 2, 3], [0, 4, 5]]

In [68]: x = np.array(x)

In [69]: x.shape
Out[69]: (2, 3)

It follows the rule (2, 3) * (3, 3) => (2, 3)

But your second example, the shape doesn't follow the rule of multiplication. The shape of your input is (2, 2, 3) which is not even in the same dimension as your defined W, so it won't work.

In [70]: foo = [[[1,2,3],[1,2,3]],[[1,1,4],[0,4,5]]]

In [71]: foo = np.array(foo)

In [72]: foo.shape
Out[72]: (2, 2, 3)