How to use the trained tensorflow network to run inference?

Question

I am new to tensorflow and I hope you can help me.

I have built a tensorflow CNN network and trained it successfully. The training datasets are matlab arrays. Now I would like to use the trained network to run inference. I am not sure how to write the python code for inference.

During training, I saved the mode. I am not sure how to load the model in inference.
My inference data is also a matlab array, same as training data. How can I use it? During training, I used miniPatch from Tensorlayer, should I use miniPatch in inference two?

Below is my inference code: it gave a lot of errors:

print("\n\nPreparing testing data........................")
test_data = sio.loadmat('MyTest.mat')
Z0 = test_data['Real_testing1']
img_num_test = Z0.shape[0]
X_test = np.empty([img_num_test, 128, 128, 1], dtype=float)
X_test[:,:,:,0] = Z0
Y_test = np.column_stack((np.ones([img_num_test, 1], dtype=int),np.zeros([img_num_test, 1], dtype=int)))
print("\tTesting X shape: {0}".format(X_test.shape))
print("\tTesting Y shape: {0}".format(Y_test.shape))


print("\n\Restore the network ...")
save_dir = "checkpoints/";
epoch = 1000
model_name = save_dir + str(epoch) + '_model'
if not os.path.exists(save_dir):
    os.makedirs(save_dir)
saver = tf.train.Saver().restore(sess, save_path=model_name)

start_time_begin = time.time()

print("\n\Running network...")

start_time = time.time()

y = model.Scribenet(X_test[0, :, :, :], False, 1.0)
y = sess.run([y], feed_dict=feed_dict)
print(y[0:9])

sess.close()

Below is my training code:

x = tf.placeholder(tf.float32, shape=[None, 128, 128, 1], name='x')
y_ = tf.placeholder(tf.int64, shape=[None, 2], name='y_')
keep_prob = tf.placeholder(tf.float32, name='keep_prob')
is_training = tf.placeholder(tf.bool, name='is_traininng')

net_in = x
net_out = model.MyCNN(net_in, is_training, keep_prob)

y = net_out
cost = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(logits=y, labels=y_, name='cost'))
correct_prediction = tf.equal(tf.argmax(y,1), tf.argmax(y_,1))
acc = tf.reduce_mean(tf.cast(correct_prediction, tf.float32))
y_op = tf.argmax(tf.nn.softmax(y),1)

train_op = tf.train.AdamOptimizer(learning_rate, beta1=0.9, beta2=0.999,
                                      epsilon=1e-08, use_locking=False).minimize(cost)

sess.run(tf.global_variables_initializer())

save_dir = "checkpoints/";
if not os.path.exists(save_dir):
    os.makedirs(save_dir)
saver = tf.train.Saver()

print("\n\nStart training the network ...")
start_time_begin = time.time()
for epoch in range(n_epoch):
    start_time = time.time()
    loss_ep = 0; n_step = 0
    for X_train_a, y_train_a in tl.iterate.minibatches(X_train, Y_train,
                                                batch_size, shuffle=True):
        feed_dict = {x: X_train_a, y_: y_train_a, is_training: True, keep_prob: train_keep_prob}
        loss, _ = sess.run([cost, train_op], feed_dict=feed_dict)
        loss_ep += loss
        n_step += 1
    loss_ep = loss_ep/ n_step

    if (epoch+1) % save_freq == 0:
        model_name = save_dir + str(epoch+1) + '_model'
        saver.save(sess, save_path=model_name)

Allen Lavoie Allen Lavoie · Accepted Answer · 2018-01-02T22:48:31

The main issue seems to be that there's no graph building in your inference code. You either need to save the whole graph (in SavedModel format), or build a graph in your inference code and load your variables via a training checkpoint (probably the easiest to start). As long as the variable names are the same, you can load variables saved from the training graph into the inference graph.

So inference will be your training code but without the y_ placeholder and without the loss/optimizer logic. You can feed a single image (batch size 1) to start, so no need for batching logic either.

How to use the trained tensorflow network to run inference?

1 Answers