Tensorflow SSD-Mobilenet model accuracy drop after quantization using transform_graph

Question

I am working on the recently released "SSD-Mobilenet" model by google for object detection. Model downloaded from following location: https://github.com/tensorflow/models/blob/master/object_detection/g3doc/detection_model_zoo.md

The frozen graph file downloaded from the site is working as expected, however after quantization the accuracy drops significantly (mostly random predictions).

I built tensorflow r1.2 from source, and used following method to quantize:

bazel-bin/tensorflow/tools/graph_transforms/transform_graph --in_graph=frozen_inference_graph.pb --out_graph=optimized_graph.pb --inputs='image_tensor' --outputs='detection_boxes','detection_scores','detection_classes','num_detections' --transforms='add_default_attributes strip_unused_nodes(type=float, shape="1,224,224,3") fold_constants(ignore_errors=true) fold_batch_norms fold_old_batch_norms quantize_weights strip_unused_nodes sort_by_execution_order'

I tried various combinations in the "transforms" part, and the transforms mentioned above gave sometimes correct predictions, however no where close to the original model.

Is there any other way to improve performance of the quantized model?

Answer 1

In this case SSD uses mobilenet as it's feature extractor . In-order to increase the speed. If you read the mobilenet paper , it's a lightweight convolutional neural nets specially using separable convolution inroder to reduce parameters .

As I understood separable convolution can loose information because of the channel wise convolution.

So when quantifying a graph according to TF implementation it makes 16 bits ops and weights to 8bits . If you read the tutorial in TF for quantization they clearly have mentioned how this operation is more like adding some noise in to already trained net hoping our model has well generalized .

So this will work really well and almost lossless interms of accuracy for a heavy model like inception , resnet etc. But with the lightness and simplicity of ssd with mobilenet it really can make a accuracy loss .

MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications

How to Quantize Neural Networks with TensorFlow