AccessTensorFlow optimizer by name

Question

hope somebody can help me. I did not find an answer to my problem using google or this site.

I am trying to access a TesorFlow optimizer by name. Here is some minimal example code: import sys import numpy as np import tensorflow as tf

print( "Python version: {0}".format(sys.version) )
print( "Tensorflow version: {0}".format(tf.__version__) )
print('')

l_input = tf.placeholder(tf.float32, shape=(None, 2), name='input')
l_dense = tf.layers.dense(l_input, units=1, activation=None)
l_output = tf.identity(l_dense, name='output')
l_true = tf.placeholder(tf.float32, shape=(None, 2), name='true')
cost = tf.reduce_sum(tf.square(l_output - l_true),name='cost')
train_step = tf.train.AdamOptimizer(0.001,name='train_step').minimize(cost)

with tf.Session() as sess:
    sess.run(tf.global_variables_initializer())

    print( "cost" )
    print( cost )
    print( "cost:0" )
    print( tf.get_default_graph().get_tensor_by_name("cost:0") )
    print('')
    print( "train_step" )
    print( train_step )
    print( "type(train_step)" )
    print( type(train_step) )
    print( "collection train_step:0" )
    print( tf.get_default_graph().get_collection("train_step:0") )
    print( "operation train_step:0" )
    print( tf.get_default_graph().get_operation_by_name("train_step:0") )
    print( "tensor train_step:0" )
    print( tf.get_default_graph().get_tensor_by_name("train_step:0") )

with the following output:

Python version: 3.6.4 (v3.6.4:d48eceb, Dec 19 2017, 06:54:40) [MSC v.1900 64 bit (AMD64)]
Tensorflow version: 1.8.0

cost
Tensor("cost:0", shape=(), dtype=float32)
cost:0
Tensor("cost:0", shape=(), dtype=float32)

train_step
name: "train_step"
op: "NoOp"
....
type(train_step)
<class 'tensorflow.python.framework.ops.Operation'>
collection train_step:0
[]
operation train_step:0

So, it works for the cost tensor. However, I cannot get it to work with the train_step operation. Why did I try get_collection? That is the only function I found that will not raise an exception.

Since the type is operation, I tried get_operation_by_name and end up with the exception

....
ValueError: Name 'train_step:0' appears to refer to a Tensor, not a Operation.

Commenting that out and using get_tensor_by_name I get the following exception

....
KeyError: "The name 'train_step:0' refers to a Tensor which does not exist. The operation, 'train_step', exists but only has 0 outputs."

I would like to load the saved graph and continue training by calling sess.run() on the train_step. For that, however, I do need to access the train_step operation somehow. I found older examples that use get_tensor_by_name, but those have stopped working with the same exception.

Any help is greatly appreciated.

Peter Szoldan Peter Szoldan · Accepted Answer · 2018-05-02T12:29:08

train_step is an operation not a tensor, and it has no outputs. Therefore train_step:0 refers to nothing, as it is trying to point to the first output of the operation. Try

print( tf.get_default_graph().get_operation_by_name( "train_step" ) )

AccessTensorFlow optimizer by name

1 Answers