I want to train a LSTM model with Tensorflow. I have a text data as input and I get doc2vec of each paragraph of the text and pass it to the lstm layers but I get ValueError because of inconsistency of shape rank. I've searched through Stackoverflow for similar questions and some tutorials, but I couldn't solve this error. Do you have any idea what should I do? Here is the error:
Traceback (most recent call last): File "writeRNN.py", line 97, in outputs, final_state = tf.nn.dynamic_rnn(cell, embed, initial_state=initial_state) File "myven/lib/python3.5/site-packages/tensorflow/python/ops/rnn.py", line 627, in dynamic_rnn dtype=dtype) File "myven/lib/python3.5/site-packages/tensorflow/python/ops/rnn.py", line 690, in _dynamic_rnn_loop for input_ in flat_input) File "myven/lib/python3.5/site-packages/tensorflow/python/ops/rnn.py", line 690, in for input_ in flat_input) File "myven/lib/python3.5/site-packages/tensorflow/python/framework/tensor_shape.py", line 761, in with_rank_at_least raise ValueError("Shape %s must have rank at least %d" % (self, rank)) ValueError: Shape (?, ?) must have rank at least 3
And below is the code:
lstm_size = 128
lstm_layers = 1
batch_size = 50
learning_rate = 0.001
# Create the graph object
graph = tf.Graph()
# Add nodes to the graph
with graph.as_default():
inputs_ = tf.placeholder(tf.int32, [None, None], name='inputs')
labels_ = tf.placeholder(tf.int32, [None, None], name='labels')
keep_prob = tf.placeholder(tf.float32, name='keep_prob')
with graph.as_default():
lstm = tf.contrib.rnn.BasicLSTMCell(lstm_size)
drop = tf.contrib.rnn.DropoutWrapper(lstm, output_keep_prob=keep_prob)
cell = tf.contrib.rnn.MultiRNNCell([drop] * lstm_layers)
initial_state = cell.zero_state(batch_size, tf.float32)
with graph.as_default():
outputs, final_state = tf.nn.dynamic_rnn(cell, inputs_, initial_state=initial_state)
with graph.as_default():
predictions = tf.contrib.layers.fully_connected(outputs[:, -1], 1, activation_fn=tf.sigmoid)
cost = tf.losses.mean_squared_error(labels_, predictions)
optimizer = tf.train.AdamOptimizer(learning_rate).minimize(cost)
with graph.as_default():
correct_pred = tf.equal(tf.cast(tf.round(predictions), tf.int32), labels_)
accuracy = tf.reduce_mean(tf.cast(correct_pred, tf.float32))
with tf.Session(graph=graph) as sess:
sess.run(tf.global_variables_initializer())
for e in range(epochs):
state = sess.run(initial_state)
for ii, (x, y) in enumerate(get_batches(train_x, train_y, batch_size), 1):
feed = {inputs_: x, labels_: y[:, None], keep_prob: 0.5, initial_state: state}
loss, state, _ = sess.run([cost, final_state, optimizer], feed_dict=feed)
I got error on outputs, final_state = tf.nn.dynamic_rnn(cell, inputs_, initial_state=initial_state) Line as the error I described. doc2vec model is trained on gensim and converts each sentence into a vector with 100 value. I tried to change inputs_ shape and labels_ shape but also I get same error! I really don't know what should I do?!
I really thank if you could answer my question.