Shape ValueError in LSTM network using Tensorflow

Question

I want to train a LSTM model with Tensorflow. I have a text data as input and I get doc2vec of each paragraph of the text and pass it to the lstm layers but I get ValueError because of inconsistency of shape rank. I've searched through Stackoverflow for similar questions and some tutorials, but I couldn't solve this error. Do you have any idea what should I do? Here is the error:

Traceback (most recent call last): File "writeRNN.py", line 97, in outputs, final_state = tf.nn.dynamic_rnn(cell, embed, initial_state=initial_state) File "myven/lib/python3.5/site-packages/tensorflow/python/ops/rnn.py", line 627, in dynamic_rnn dtype=dtype) File "myven/lib/python3.5/site-packages/tensorflow/python/ops/rnn.py", line 690, in _dynamic_rnn_loop for input_ in flat_input) File "myven/lib/python3.5/site-packages/tensorflow/python/ops/rnn.py", line 690, in for input_ in flat_input) File "myven/lib/python3.5/site-packages/tensorflow/python/framework/tensor_shape.py", line 761, in with_rank_at_least raise ValueError("Shape %s must have rank at least %d" % (self, rank)) ValueError: Shape (?, ?) must have rank at least 3

And below is the code:

lstm_size = 128
lstm_layers = 1
batch_size = 50
learning_rate = 0.001

# Create the graph object
graph = tf.Graph()
# Add nodes to the graph
with graph.as_default():
    inputs_ = tf.placeholder(tf.int32, [None, None], name='inputs')
    labels_ = tf.placeholder(tf.int32, [None, None], name='labels')
    keep_prob = tf.placeholder(tf.float32, name='keep_prob')
with graph.as_default():
    lstm = tf.contrib.rnn.BasicLSTMCell(lstm_size)
    drop = tf.contrib.rnn.DropoutWrapper(lstm, output_keep_prob=keep_prob)
    cell = tf.contrib.rnn.MultiRNNCell([drop] * lstm_layers)
    initial_state = cell.zero_state(batch_size, tf.float32)

with graph.as_default():
    outputs, final_state = tf.nn.dynamic_rnn(cell, inputs_, initial_state=initial_state)


with graph.as_default():
    predictions = tf.contrib.layers.fully_connected(outputs[:, -1], 1, activation_fn=tf.sigmoid)
    cost = tf.losses.mean_squared_error(labels_, predictions)
    optimizer = tf.train.AdamOptimizer(learning_rate).minimize(cost)


with graph.as_default():
    correct_pred = tf.equal(tf.cast(tf.round(predictions), tf.int32), labels_)
    accuracy = tf.reduce_mean(tf.cast(correct_pred, tf.float32))
with tf.Session(graph=graph) as sess:
    sess.run(tf.global_variables_initializer())
    for e in range(epochs):
        state = sess.run(initial_state)
        for ii, (x, y) in enumerate(get_batches(train_x, train_y, batch_size), 1):
            feed = {inputs_: x, labels_: y[:, None], keep_prob: 0.5, initial_state: state}
            loss, state, _ = sess.run([cost, final_state, optimizer], feed_dict=feed)

I got error on outputs, final_state = tf.nn.dynamic_rnn(cell, inputs_, initial_state=initial_state) Line as the error I described. doc2vec model is trained on gensim and converts each sentence into a vector with 100 value. I tried to change inputs_ shape and labels_ shape but also I get same error! I really don't know what should I do?!

I really thank if you could answer my question.

Aaron Aaron · Accepted Answer · 2018-05-15T20:01:46

If you look at the documentation for the dynamic_rnn function it says that the inputs should have three dimensions. The first one is for the batch size, the second is for the sequence length, and the third is the data embedding size. Your input only has two dimensions, which is why there is an error.

Shape ValueError in LSTM network using Tensorflow

1 Answers