Tensorflow tf.squared_difference outputs unexpected shape

Question

Newbie here, I am sorry if this question is silly but I couldn't find anything about it online. I am getting an unexpected shape for the output of tf.squared_difference. I would expect the obtain of a Tensor with shape=(100, ?) shape as the loss from the following snippet

[print("Logits",logits,"#Labels",labels,"LOSS",tf.squared_difference(labels,logits)) for logits, labels in zip(logits_series,labels_series)]

However it produces (100,100) Loss

Logits Tensor("add_185:0", shape=(100, 1), dtype=float32) #Labels Tensor("unstack_29:0", shape=(100,), dtype=float32) LOSS Tensor("SquaredDifference_94:0", shape=(100, 100), dtype=float32) Logits Tensor("add_186:0", shape=(100, 1), dtype=float32) #Labels Tensor("unstack_29:1", shape=(100,), dtype=float32) LOSS Tensor("SquaredDifference_95:0", shape=(100, 100), dtype=float32)

I have tested another example with following code and gives the expected output shape.

myTESTX = tf.placeholder(tf.float32, [100, None])
myTESTY = tf.placeholder(tf.float32, [100, 1])

print("Test diff X-Y",tf.squared_difference(myTESTX,myTESTY) )
print("Test diff Y-X",tf.squared_difference(myTESTY,myTESTX) )

Test diff X-Y Tensor("SquaredDifference_92:0", shape=(100, ?), dtype=float32) Test diff Y-X Tensor("SquaredDifference_93:0", shape=(100, ?), dtype=float32)

I am having issue why these two snippets produce different output shape

Brenden Petersen Brenden Petersen · Accepted Answer · 2017-10-10T19:16:56

There's a slight difference between your first example (with logits and labels) and second example (with myTESTX and myTESTY). logits has the same shape as myTESTY: (100, 1). However, labels has shape (100,) (which is NOT a dynamic shape), but myTESTX has shape (100, ?).

In the first case (logits and labels), input shapes are (100,) and (100,1), and tensorflow uses broadcasting. Neither input shape is dynamic, so your output shape is static: (100, 100) due to broadcasting.

In the second case (myTESTX and myTESTY), input shapes are (100, ?) and (100, 1). The first input shape is dynamic, so your output shape is dynamic: (100, ?).

As a simpler, illustrative example in numpy (which uses the same broadcasting), consider this:

import numpy as np
x = np.arange(10)                # Shape: (10,)
y = np.arange(10).reshape(10,1)  # Shape: (10, 1)
difference = x-y                 # Shape: (10, 10)

Tensorflow tf.squared_difference outputs unexpected shape

1 Answers