tflite uint8 quantization model input and output float conversion

0
votes

I have successfully converted a quantized 8bit tflite model for object detection. My model was originally trained on images that are normalized by dividing 255 so the original input range is [0, 1]. Since my quantized tflite model requires input to be uint8, how can I convert my image (originally [0, 255]) to be correct for my network? Also how can I convert output to float to compare the results with floating point model?

The following code does not give me the right result.

'''python
im = cv2.imread(image_path)
im = im.astype(np.float32, copy=False)
input_image = im
input_image = np.array(input_image, dtype=np.uint8)
input_image = np.expand_dims(input_image, axis=0)

interpreter.set_tensor(input_details[0]['index'], input_image)
interpreter.invoke()
output_data = interpreter.get_tensor(output_details[0]['index'])
output_data2 = interpreter.get_tensor(output_details[1]['index'])
output_data3 = interpreter.get_tensor(output_details[2]['index'])

min_1 =  -8.198164939880371
max_1 = 8.798029899597168
scale = (max_1 - min_1)/ 255.0

min_2 = -9.77856159210205
max_2 = 10.169703483581543
scale_2 = (max_2 - min_2) / 255.0

min_3 = -14.382895469665527 
max_3 = 11.445544242858887
scale_3 = (max_3 - min_3) / 255.0

output_data = (output_data ) * scale + min_1
output_data2  = (output_data2) * scale_2 + min_2
output_data3 = (output_data3) * scale_3 + min_3

'''

2
tensorflowtensorflow-litequantization
For input, it should be input_image = im / 256 But for output, where do you take these min/max values from? - Alex Cohn
it's from the fakequantization node added by tf.lite.create_train_graph() that keeps track of each layer's min max activation for quantization - Jasmine Liu
The values that you happen to get for output in certain case may not correct for scaling the output to unit8. - Alex Cohn
I'm not sure if I understand you correctly. But tflite quantization keeps track of min and max value and perform a uniform quantization over the range, so the floating output should be correctly represented in uint8. I test the floating point model with the fake quantization nodes and the output is correct. - Jasmine Liu
maybe if you provide correctly scaled inputs, the outputs will become better - Alex Cohn

2 Answers

0
votes

i met the same problem but in pose estimation.

have you solved the problem yet?

you use quantized aware training?

i think you can get a q and z value(because you have to give mean and std-err when you use tflite api or toco commonad to get a quantized 8bit tflite model) about your input image.

try these codes:

  image = q_input* (image - z_input)

  output_data  = q_output(image - z_output)

etc.

(for different layers you can access different q and z)

Let me know if you tried this way

0
votes

I've converted the image via OpenCV to "CV_8UC3" and this worked for me:

  // Convert to RGB color space
  if (image.channels() == 1) {
    cv::cvtColor(image, image, cv::COLOR_GRAY2RGB);
  } else {
    cv::cvtColor(image, image, cv::COLOR_BGR2RGB);
  }

  image.convertTo(image, CV_8UC3);