How to evaluate Tensorflow Object Detection API on only one particular class?

1
votes

I have trained an object detection model from Tensorflow Object Detection API faster_rcnn_inception_resnet_v2_atrous_coco on about 10 classes. When I run the model_main.py file to evaluate the model, it seems to only give the mean Average Precision (AP) and Average Recall (AR) of all the 10 classes, like below: 

Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.331
 Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=100 ] = 0.479
 Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=100 ] = 0.395
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.600
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.407
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.333
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=  1 ] = 0.358
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets= 10 ] = 0.544
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.548
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.600
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.545
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.551

However, if I want to just evaluate the performance on 1 particular class, instead of all 30 detected classes, what should I do?

2
tensorflowcomputer-visionartificial-intelligenceobject-detection-apifaster-rcnn

2 Answers

0
votes

Add the below code to filter objects of interest, after the detection -

objectOfInterest = 1  # Interested object class number as per label file 
box = np.asarray(boxes)
cls = np.asarray(classes).astype(np.int32)
scr = np.asarray(scores)
bl = (cls == objectOfInterest) 
classes = np.extract(boolar,cls)
scores = np.extract(boolar,scr)
boxes = np.extract(boolar,box)
0
votes

Better late than never - From this post

1. Use a different evaluation configuration

Simply change the metrics_set value in the *.config file for your model to "pascal_voc_detection_metrics".

The TensorFlow Object Detection API supports a variety of evaluation metrics, detailed in the documentation here. The PASCAL VOC 2010 detection metric gives AP scores for each class.

2. Edit the cocoeval.py file in your pycocotools package

This method involves pasting 8 lines of code into the cocoeval.py file. It is well explained and documented in this post.