In TensorFlow, Is there any difference in saving and restore the LSTM models?

Question

I have tried to save and restore the LSTM models according to the tutorial. Actually, it works in the normal models saving and restoring like CNN models. However when i tried to restore the LSTM models it throw the error

FailedPreconditionError (see above for traceback): Attempting to use uninitialized value RNN_model/RNN/multi_rnn_cell/cell_0/basic_lstm_cell/kernel

the key codes shows follow:

   with tf.Session() as sess:
    saver.restore(sess, saver_path)
    with tf.variable_scope('RNN_model', reuse=None):
        train_rnn = RNNmodel.LSTMmodel(True, RNNmodel.TRAIN_BATCH_SIZE, RNNmodel.NUM_STEP)

    with tf.variable_scope('RNN_model', reuse=True):
        test_rnn = RNNmodel.LSTMmodel(False, RNNmodel.EVAL_BATCH_SIZE, RNNmodel.NUM_STEP)

I wonder if there any difference between the normal models and LSTM models in the saving and restoring. Please help

EDIT: i try to move the restore and it works, but when i run my models, it still throws the same error, my run_epoch code like:

def run_epoch(session, model, datas, train_op, is_log, epoch=3000):
    state = session.run(model.initiate_state)
    total_cost = 0
    for i in range(epoch):
        data, label = random_get_data(datas, model.batch_size, num_step=RNNmodel.NUM_STEP)
        feed_dict = {
            model.input_data: data,
            model.target: label,
            model.initiate_state: state
        }
        cost, state, argmax_logit, target, _ = session.run([model.loss, model.final_state, model.argmax_target, model.target, train_op], feed_dict)

the log located the error at the :

cost, state, argmax_logit, target, _ = session.run([model.loss, model.final_state, model.argmax_target, model.target, train_op], feed_dict)

and the log shows follow:

tensorflow.python.framework.errors_impl.FailedPreconditionError: Attempting to use uninitialized value RNN_model/RNN/multi_rnn_cell/cell_0/basic_lstm_cell/kernel

it seems that the restore does not restore the lstm kernel operation.should i do anything to initiate the lstm operation specially?

EDIT2: I check out the checkpoint file at last and i am sure that save operation dose not save the variables about LSTM cells and i dont know why.It seems that i have to name the variables explicitly otherwise i cannot save it, and BasicLSTMCell class init() dose not have the name parameters.

Pavel Christof Pavel Christof · Accepted Answer · 2017-10-01T08:00:17

There is no difference, RNNs use ordinary variables.

I think you have to move

saver.restore(sess, saver_path)

after you create the LSTMmodel. Otherwise it's variables are not in the graph when you call restore - so they won't be restored.

In TensorFlow, Is there any difference in saving and restore the LSTM models?

2 Answers