Tensorflow object detection api validation data size

Question

I am running tutorial from object detection API and I am using Oxford dataset with ResNet Faster-RCNN.

When I evaluate my trained model by running (eval.py), Tensorboard returns about 0.95 smoothed precision value.

My question is how many image set does it evaluate? Because from Tensorboard and their tutorial link (https://github.com/tensorflow/models/blob/master/object_detection/g3doc/running_pets.md), Tensorboard only shows 10 images.

Does it mean that they check precision only with 10 images?

My Oxford dataset validation jpg counts should be about 2,200.

In my configuration, I specified input path correctly like this:

eval_input_reader: {
  tf_record_input_reader {
    input_path: "my_path/pet_val.record"
  }
  label_map_path: "my_path/pet_label_map.pbtxt"
  shuffle: false
  num_readers: 1
}

And does eval.py prints mAP at the end?

I run eval.py about three days ago on my 1 GPU local machine, but it does not print anything.

Finally, does this API provide F-value and fps (frame per second)? Anyone has experience with this?

edit: it seems that we can setup eval size limit from configuration such as /object_detection/samples/configs/faster_rcnn_resnet101_pets.config#L131. When I print len(result_lists) from https://github.com/tensorflow/models/blob/master/object_detection/eval_util.py#L404, it prints 2000, which was my eval num_examples.

I was also able to generate fps by comparing timestamp manually.

Derek Chow Derek Chow · Accepted Answer · 2017-08-07T17:04:19

By default, we only visualize 10 images on Tensorboard (to avoid overwhelming it with images) but this is configurable from the eval_config. You can also change the number of images evaluated (defaults to 5000) in the config too.

Tensorflow object detection api validation data size

1 Answers