Tensorflow cannot initialize tf.Variable for dynamic batch size

7
votes

I tried creating a tf.Variable with a dynamic shape. The following outlines the problem.

Doing this works. 

init_bias = tf.random_uniform(shape=[self.config.hidden_layer_size, tf.shape(self.question_inputs)[0]])

However, when i try to do this:

init_bias = tf.Variable(init_bias)

It throws the error ValueError: initial_value must have a shape specified: Tensor("random_uniform:0", shape=(?, ?), dtype=float32)

Just come context (question input is a placeholder which dynamic batch ):

self.question_inputs = tf.placeholder(tf.int32, shape=[None, self.config.qmax])

It seems like putting a dynamic value into random uniform gives shape=(?,?) which gives an error with tf.Variable.

Thanks and appreciate any help!

1
tensorflow

1 Answers

11
votes

This should work:

init_bias = tf.Variable(init_bias,validate_shape=False)

If validate_shape is False, tensorflow allows the variable to be initialized with a value of unknown shape.

However, what you're doing seems a little strange to me. In tensorflow, Variables are generally used to store weights of a neural net, whose shape remains fixed irrespective of the batch size. Variable batch size is handled by passing a variable length tensor into the graph (and multiplying/adding it with a fixed shape bias Variable).