Can't restore tensorflow model with Saver

1
votes

I'm following this guide to using the Saver class in Tensorflow version 1.

I'm first saving the model:

with tf.Session() as sess:
  init = tf.global_variables_initializer()
  sess.run(init)
  saver.save(sess, "./saved_model/tf/model", global_step=0)

which gives me these files:

$ ls saved_model/tf
checkpoint  model-0.data-00000-of-00001  model-0.index  model-0.meta

But when I try to restore the session, I get an error:

with tf.Session() as sess:
  saver.restore(sess, "./saved_model/tf/model")

---------------------------------------------------------------------------

ValueError                                Traceback (most recent call last)

<ipython-input-37-01cbbefb52af> in <module>()
      1 with tf.Session() as sess:
----> 2   saver.restore(sess, "./saved_model/tf/model")

/tensorflow-1.15.2/python3.6/tensorflow_core/python/training/saver.py in restore(self, sess, save_path)
   1280     if not checkpoint_management.checkpoint_exists_internal(checkpoint_prefix):
   1281       raise ValueError("The passed save_path is not a valid checkpoint: " +
-> 1282                        checkpoint_prefix)
   1283 
   1284     logging.info("Restoring parameters from %s", checkpoint_prefix)

ValueError: The passed save_path is not a valid checkpoint: ./saved_model/tf/model

What am I doing wrong? Unfortunately, the TF documentation on this feature does not help much.

1
pythontensorflow

1 Answers

0
votes

ValueError: The passed save_path is not a valid checkpoint: ./saved_model/tf/model

Here the error conveys that the checkpoint file is not present and therefore it is not a valid checkpoint.

I was able to recreate your problem, it caused because of global_step=0 in model save block. For better understanding printed model save path at the end of the program, which guides you where it saved and how it created files with this option.

Model Save:

%tensorflow_version 1.x
import tensorflow as tf

# Create some variables.
v1 = tf.get_variable("v1", shape=[3], initializer = tf.zeros_initializer)
v2 = tf.get_variable("v2", shape=[5], initializer = tf.zeros_initializer)

inc_v1 = v1.assign(v1+1)
dec_v2 = v2.assign(v2-1)

# Add an op to initialize the variables.
init_op = tf.global_variables_initializer()

# Add ops to save and restore all the variables.
saver = tf.train.Saver()

# Later, launch the model, initialize the variables, do some work, and save the
# variables to disk.
with tf.Session() as sess:
  sess.run(init_op)
  # Do some work with the model.
  inc_v1.op.run()
  dec_v2.op.run()
  # Save the variables to disk.
  save_path = saver.save(sess, "/content/gdrive/My Drive/checkpoint/test", global_step=0)
  print("Model saved in path: %s" % save_path)

Output:

TensorFlow 1.x selected.
Model saved in path: /content/gdrive/My Drive/checkpoint/test-0

Listing the contents of a directory:

!ls "/content/gdrive/My Drive/checkpoint/"
checkpoint  test-0.data-00000-of-00001  test-0.index  test-0.meta

Model Restore: 

%tensorflow_version 1.x
import tensorflow as tf

with tf.Session() as sess:
  saver.restore(sess, "/content/gdrive/My Drive/checkpoint/test")

Ouput:

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-3-313790e7866b> in <module>()
      4 
      5 with tf.Session() as sess:
----> 6   saver.restore(sess, "/content/gdrive/My Drive/checkpoint/test")

/tensorflow-1.15.2/python3.6/tensorflow_core/python/training/saver.py in restore(self, sess, save_path)
   1280     if not checkpoint_management.checkpoint_exists_internal(checkpoint_prefix):
   1281       raise ValueError("The passed save_path is not a valid checkpoint: " +
-> 1282                        checkpoint_prefix)
   1283 
   1284     logging.info("Restoring parameters from %s", checkpoint_prefix)

ValueError: The passed save_path is not a valid checkpoint: /content/gdrive/My Drive/checkpoint/test

Solution:

Please remove global_step=0 in model save block and observe where and how files are creating, thus resolves the problem.

%tensorflow_version 1.x
import tensorflow as tf

# Create some variables.
v1 = tf.get_variable("v1", shape=[3], initializer = tf.zeros_initializer)
v2 = tf.get_variable("v2", shape=[5], initializer = tf.zeros_initializer)

inc_v1 = v1.assign(v1+1)
dec_v2 = v2.assign(v2-1)

# Add an op to initialize the variables.
init_op = tf.global_variables_initializer()

# Add ops to save and restore all the variables.
saver = tf.train.Saver()

# Later, launch the model, initialize the variables, do some work, and save the
# variables to disk.
with tf.Session() as sess:
  sess.run(init_op)
  # Do some work with the model.
  inc_v1.op.run()
  dec_v2.op.run()
  # Save the variables to disk.
  save_path = saver.save(sess, "/content/gdrive/My Drive/checkpoint/test")
  print("Model saved in path: %s" % save_path)

Output:

TensorFlow 1.x selected.
Model saved in path: /content/gdrive/My Drive/checkpoint/test

Listing the contents of a directory:

!ls "/content/gdrive/My Drive/checkpoint/"
checkpoint  test.data-00000-of-00001  test.index  test.meta

Model restore : Ideal way of restore model is as below.

%tensorflow_version 1.x

import tensorflow as tf

tf.reset_default_graph()

# Create some variables.
v1 = tf.get_variable("v1", shape=[3])
v2 = tf.get_variable("v2", shape=[5])

# Add ops to save and restore all the variables.
saver = tf.train.Saver()

# Later, launch the model, use the saver to restore variables from disk, and
# do some work with the model.
with tf.Session() as sess:
  # Restore variables from disk.
  saver.restore(sess, "/content/gdrive/My Drive/checkpoint/test")
  print("Model restored.")
  # Check the values of the variables
  print("v1 : %s" % v1.eval())
  print("v2 : %s" % v2.eval())

Output:

TensorFlow 1.x selected.
INFO:tensorflow:Restoring parameters from /content/gdrive/My Drive/checkpoint/test
Model restored.
v1 : [1. 1. 1.]
v2 : [-1. -1. -1. -1. -1.]

Please refer Save and Restore explanation and Code for Tensorflow Version 1.x here