TensorFlow Variable Names - Assignment Error in Save/Restore

Question

I am saving and restoring a TensorFlow model using tf.train.Saver save and restore. In the restore process I am loading new input data. The restore method throws this error:

InvalidArgumentError (see above for traceback): Assign requires shapes of both tensors to match. lhs shape= [1334,3] rhs shape= [1246,3] [[Node: save/Assign_6 = Assign[T=DT_FLOAT, _class=["loc:@Variable_2"], use_locking=true, validate_shape=true, _device="/job:localhost/replica:0/task:0/cpu:0"](Variable_2, save/RestoreV2_6)]]

This seems to say that the problem is in Variable_2 but how does one determine which variable in the code corresponds to Variable_2?

Jai Jai · Accepted Answer · 2017-12-28T23:59:36

If you are restoring the model and doing a feed forward, then the model shape and model architecture should be the same when you saved it
So the above error is saying when you are restoring your model, one of the tensors which was saved has shape [1246,3] but you are assigning it to a tensor whose shape is [1334,3]
to explicitly know which variables are of which name you can assign unique names to the tensors, for example a = tf.placeholder("float", [3, 3], name="tensor_a")
So now when you restore the model you know your model has a tensor in the graph with name="tensor_a" which is of shape 3x3

Quick Tutorial in Code :

# Create some variables.
v1 = tf.get_variable("v1", shape=[3], initializer=tf.zeros_initializer)
v2 = tf.get_variable("v2", shape=[5], initializer=tf.zeros_initializer)

inc_v1 = v1.assign(v1+1)
dec_v2 = v2.assign(v2-1)

# Add an op to initialize the variables.
init_op = tf.global_variables_initializer()

# Add ops to save and restore all the variables.
saver = tf.train.Saver()

# Later, launch the model, initialize the variables, do some work, and save the
# variables to disk.
with tf.Session() as sess:
    sess.run(init_op)
    # Do some work with the model.
    inc_v1.op.run()
    dec_v2.op.run()
    # Save the variables to disk.
    save_path = saver.save(sess, "/tmp/model.ckpt")
    print("Model saved in file: %s" % save_path)

tf.reset_default_graph()

# Create some variables.
d1 = tf.get_variable("v1", shape=[3])
d2 = tf.get_variable("v2", shape=[5])

# Add ops to save and restore all the variables.
saver = tf.train.Saver()

# Later, launch the model, use the saver to restore variables from disk, and
# do some work with the model.
with tf.Session() as sess:
    # Restore variables from disk.
    saver.restore(sess, "/tmp/model.ckpt")
    print("Model restored.")
    # Check the values of the variables
    print("v1 : %s" % d1.eval())
    print("v2 : %s" % d2.eval())

If you noticed in above code d1 and v1 have the same shape now if you change shape of any ofthe variable it will throw you a error which is similar to the error what you are getting

TensorFlow Variable Names - Assignment Error in Save/Restore

2 Answers