Tensorflow object detection api test time (Google object detection running time)

Question

Google Object Detection API:

https://github.com/tensorflow/models/tree/master/research/object_detection

Test code:

https://github.com/tensorflow/models/blob/master/research/object_detection/object_detection_tutorial.ipynb

I executed the test code of Google Object Detection API as follows:

with detection_graph.as_default():
  with tf.Session(graph=detection_graph) as sess:
    start = time.time()
    image_tensor = 
    detection_graph.get_tensor_by_name('image_tensor:0')
    # Each box represents a part of the image where a particular 
    #object was detected.
    detection_boxes = 
    detection_graph.get_tensor_by_name('detection_boxes:0')        
    detection_scores = 
    detection_graph.get_tensor_by_name('detection_scores:0')
    detection_classes = 
    detection_graph.get_tensor_by_name('detection_classes:0')
    num_detections = 
    detection_graph.get_tensor_by_name('num_detections:0')

    for image_path in TEST_IMAGE_PATHS:
      image = Image.open(image_path)
      image_np = load_image_into_numpy_array(image)
      image_np_expanded = np.expand_dims(image_np, axis=0)
      # Actual detection.
      (boxes, scores, classes, num) = sess.run(
          [detection_boxes, detection_scores, detection_classes, 
         num_detections], feed_dict={image_tensor: image_np_expanded})

      # Visualization of the results of a detection.
      vis_util.visualize_boxes_and_labels_on_image_array(
          image_np,
          np.squeeze(boxes),
          np.squeeze(classes).astype(np.int32),
          np.squeeze(scores),
          category_index,
          use_normalized_coordinates=True,
          line_thickness=2)

    print("--- %s seconds ---" % (time.time() - start))

According to Google research paper, all models supported by Google Object Detection API have real time performance! However, the above test code show that it took about 3 second to detect one image (Actually, 200 frames->130 second, 400 frames->250 second). I think that this result is wrong because this model has real time performance.

Possible cause I expected...

GPU doesn't work properly.
Wrong measurement test runtime method

Please, let me know how to exactly measure detection time.

For more details, please refer to the link below https://github.com/tensorflow/models/issues/3531

Thommy257 Thommy257 · Accepted Answer · 2018-08-16T12:04:17

Actually, the "object_detection_tutorial notebook" runs very slowly because it's much more than just the actual inference. It loads the image, puts it into a numpy array, loads the graph (very computationally expensive), runs the actual inference with batch size 1 and outputs the image with the boxes on it.

This script is far away from being optimized and also not meant for time-critical purposes. It's just meant for quick visual validation of a model.

If you want to deploy a model for production (where time normally counts), Tensorflow Serve is what you looking for. With Tensorflow serve you can easily build a GPU Server that runs your model. It has several features that ease your life. So you'll just have to pass an image to your server and it will return the output of your model. Post-processing should be done with another server, as GPU-Servers are normally quite expensive. There are several good tutorials out there, which describe how to set up an object detection server with Tensorflow Serve. For example here. It requires some experience with docker but you'll get along with it!

Hope this helps!

Tensorflow object detection api test time (Google object detection running time)

1 Answers