Tensorflow matrix multiplication is slower than numpy

2
votes
import tensorflow as tf
import numpy as np
from time import time

def print_timer(func):
  start_time = time()
  func()
  end_time = time()
  print(end_time - start_time)

N = 4
A = np.random.randn(N, 1000, 16000)
B = np.random.randn(N, 16000, 10)
sess = tf.Session()


A_ = tf.constant(A)
B_ = tf.constant(B)

def np_test():
  r = np.empty([N, 1000, 10])
  for i in range(N):
    r[i] = np.matmul(A[i], B[i])

print_timer(lambda: np.matmul(A, B))
print_timer(lambda: sess.run(tf.matmul(A,B)))

When I ran this code, I got the result as below:

1.3403866291046143
4.291470527648926

which are the running time.

I don't know why tensorflow.matmul is slower than numpy.matmul. I am running this code on the P40 NVIDIA GPU and the tensorflow version I am using is 1.4.

When I tried running this code on tensorflow 1.8, I got the same result.

If tensorflow runs matrix multiplication in parallel, shouldn't the run times for the matrix multiplication on the GPU be much faster than those that are run on numpy, which are run on the CPU?

2
numpytensorflowgpu
Isn't it possible that copying data to the CPU adds a big overhead? I would try with a longer operation, for instance 100000 matrix multiplications. I might be completely wrong. - nicoco
Do the same with A, B = tf.get_variable('v0', initializer=A), ... and be prepared for a surprise (no cudaMemcpy). - Patwie

2 Answers

1
votes

You have tf.matmul inside the evaluation, meaning that you are mearuring the time of the creation of the operation together with its computation time.

Try this instead:

import tensorflow as tf
import numpy as np

sess = tf.InteractiveSession()

N = 4
A = np.random.randn(N, 1000, 16000)
B = np.random.randn(N, 16000, 10)
A_ = tf.constant(A)
B_ = tf.constant(B)

AB_ = A_ @ B_

%timeit np.matmul(A, B)
%timeit sess.run(AB_)

Some remarks:

  • Don't reinvent the wheel, use timeit to measure computatio times.
  • You can use @ as a shortcut for matrix multiplication since python 3.5.
0
votes

You're not using the constant tensors that you've created. Change this:

print_timer(lambda: sess.run(tf.matmul(A,B)))

to this:

print_timer(lambda: sess.run(tf.matmul(A_,B_)))