Tensorflow difficulty feeding batch size

Question

As a preface, I understand how to make None sized tensors and their primary usage.

My hidden layers depend upon the batch_size of the data I am passing in. As such I'm passing in the batch size to a placeholder. Unfortunately, this has caused me a bunch of stress and bugs, as many functions don't work well with None sized shapes. I am looking for a way to use the dynamic shape of a tensor to calculate the number of nodes in my hidden layers.

Currently, I'm receiving the error

InvalidArgumentError (see above for traceback): You must feed a value for placeholder tensor 'Placeholder_2' with dtype int32 [[Node: Placeholder_2 = Placeholderdtype=DT_INT32, shape=[], _device="/job:localhost/replica:0/task:0/cpu:0"]]

It doesn't like initializing the fill constant with a shape that is unknown at run. I would greatly appreciate help

Below is a snippet of code where the error is isolated.

import tensorflow as tf

x = tf.placeholder(tf.float32, shape=(None, 784))

nodes = tf.div(tf.shape(x)[0],2)

bias = tf.Variable(tf.constant(.1 ,shape = [nodes]), validate_shape =  False )

init = tf.initialize_all_variables()
sess = tf.InteractiveSession()
sess.run(init)

I also receive the error:

shape = [int(dim) for dim in shape] TypeError: int() argument must be a string, a bytes-like object or a number, not 'Tensor'

When replacing the fill line with

bias = tf.Variable(tf.constant(.1 ,shape = [nodes]), validate_shape = False )

Thank you in advance.

Quick question: The shape of the tensor x is (?,784), right? How is it that you can access 784 with tf.shape(x)[0], since I would expect [0] to be None. — jbuchel

Peter Hawkins Peter Hawkins · Accepted Answer · 2016-10-25T14:09:56

The shape of a Tensor cannot depend on another Tensor. Shapes must contain only integer values, or None, indicating the shape is unknown. If you want to perform calculations on shapes, you need to do those calculations using Python primitives that operate on integers, not Tensorflow primitives that operate on Tensors.

(You also should not turn off shape validation --- that's a sign you're doing something wrong.)

That said, it's unusual for different layers in a neural network to use different values for the batch size --- are you sure that's what you want to do?

If you're sure, try something like this:

import tensorflow as tf

batch_size = 32

x = tf.placeholder(tf.float32, shape=(batch_size, 784))
bias = tf.Variable(tf.constant(.1, shape = [batch_size / 2]))

init = tf.initialize_all_variables()
sess = tf.InteractiveSession()
sess.run(init)

I hope that helps!

Tensorflow difficulty feeding batch size

1 Answers