tensorflow lite inference give different results then regular inference

Question

i have a model that extract 512 features from an image (numbers between -1,1). i converted this model to tflite float format using the instruction here https://github.com/tensorflow/tensorflow/tree/master/tensorflow/contrib/lite

i run an inference on the same image with the original model and the tflite model.

i am getting different results for the vector, i was expecting to get very similar results as i didn't use quantized format. and from what i understand tf-lite should only improve the inference performance time and not effect the features calculation.

my question is this normal ? anyone else encountered this ? i didn't find any topics regarding this at any place.

Updated with code.

i have this network i trained (removed many items as i can't share full network) placeholder = tf.placeholder(name='input', dtype=tf.float32,shape=[None, 128,128, 1])

with slim.arg_scope([slim.conv2d, slim.separable_conv2d],
                      activation_fn=tf.nn.relu, normalizer_fn=slim.batch_norm):
    net = tf.identity(placeholder)
    net = slim.conv2d(net, 32, [3, 3], scope='conv11')
    net = slim.separable_conv2d(net, 64, [3, 3], scope='conv12')
    net = slim.max_pool2d(net, [2, 2], scope='pool1')  # 64x64

    net = slim.separable_conv2d(net, 128, [3, 3], scope='conv21')
    net = slim.max_pool2d(net, [2, 2], scope='pool2')  # 32x32
    net = slim.separable_conv2d(net, 256, [3, 3], scope='conv31')

    net = slim.max_pool2d(net, [2, 2], scope='pool3')  # 16x16
    net = slim.separable_conv2d(net, 512, [3, 3], scope='conv41')
    net = slim.max_pool2d(net, [2, 2], scope='pool4')  # 8x8
    net = slim.separable_conv2d(net, 1024, [3, 3], scope='conv51')
    net = slim.avg_pool2d(net, [8, 8], scope='pool5')  # 1x1
    net = slim.dropout(net)
    net = slim.conv2d(net, feature_vector_size, [1, 1], activation_fn=None, normalizer_fn=None, scope='features')
    embeddings = tf.nn.l2_normalize(net, 3, 1e-10, name='embeddings')

bazel-bin/tensorflow/contrib/lite/toco/toco --input_file=/tmp/network_512.pb --input_format=TENSORFLOW_GRAPHDEF --output_format=TFLITE --output_file=/tmp/tffiles/network_512.tflite --inference_type=FLOAT --input_type=FLOAT --input_arrays=input --output_arrays=embeddings --input_shapes=1,128,128,1

i run network_512.pb using tensorflow in python and network_512.tflite using the code from https://github.com/tensorflow/tensorflow/tree/master/tensorflow/contrib/lite/java/demo where i modified the code to load my network with and run it.

@ma3oun updated the question. but i am not sure how this helps, as the question is more a general question. — Arkady Godlin

Arkady Godlin Arkady Godlin · Accepted Answer · 2018-04-02T14:42:43

Update that i have found. the test i did was using the Demo app tensorflow provide and change it to use my costume model and extracting features, and there i noticed the difference in the features values.

once i compiled the tf-lite c++ lib manually on latest android, and run the flow with the same flow i use (which is TF-C API until now) i got almost same results for the features.

didn't have time to investigate from where come the difference. but i am happy now.

tensorflow lite inference give different results then regular inference

1 Answers