python - COCO api evaluation for subset of classes

7
votes

I'm using the python coco api to run evaluation for object detection. I have two files, a ground truth json, and a results json. The coco notebook demo only shows running eval for all classes. How can I run it for only one specific class or a subset of classes? Currently I'm doing this:

from pycocotools.coco import COCO
from pycocotools.cocoeval import COCOeval

annType = 'bbox'

cocoGt=COCO(gt_json)
cocoDt=cocoGt.loadRes(results_json)

imgIds=sorted(cocoGt.getImgIds())

# running evaluation
cocoEval = COCOeval(cocoGt,cocoDt,annType)
cocoEval.params.imgIds = imgIds
cocoEval.evaluate()
cocoEval.accumulate()
cocoEval.summarize()
2
pythonsubsetmscocopycocotools

2 Answers

11
votes

I refer to this page (http://www.programmersought.com/article/3065285708/) 

cocoEval = COCOeval(cocoGt,cocoDt,annType)
coco_eval.params.catIds = [1] #person id : 1
cocoEval.params.imgIds = imgIds   
cocoEval.evaluate()  
cocoEval.accumulate()  
cocoEval.summarize()

Additionaly, I modified cocoapi/PythonAPI/pycocotools/cocoeval.py to calculate AP for each category.
line 458-464 in https://github.com/kimyoon-young/centerNet-deep-sort/blob/master/tools/cocoeval.py

The result is like below.

category : 0 : 0.410733757610904 #person AP
category : 1 : 0.20226150054237374 #bird AP
....
category : 79 : 0.04993736566987926
(all categories) mAP : 0.27999824034118914 # my results
Average Precision (AP) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.280 #original

0
votes

In lines 458-464 of https://github.com/kimyoon-young/centerNet-deep-sort/blob/master/tools/cocoeval.py, to support a flexible number of categories, you can replace line 458 (num_classes = 80) with

num_classes = len(p.catIds)