3
votes

I am training a single object detector with mask rcnn and I have tried several methods for reducing false positives. I started with a few thousand examples of images of the object with bounding boxes and trained that, got decent results, but when running on images that don't contain that object, would often get false matches with high confidence (sometimes .99).

The first thing I tried was adding the hard example miner in the config file. I believe I did this correctly because I added a print statement to ensure the object gets created. However none of the configs for faster rcnn have hard example mining in them. So I am suspicious that the miner only works correctly for ssd. I would expect a noticeable improvement with a hard example miner but I did not see it

The second thing I tried was to add "background" images. I set the minimum number of negatives to a non-zero value in the hard example miner config and added tons of background images that previously got false detections as part of the training. I even added these images into the tfrecords file so that it would be balanced evenly with images that do have the object. This approach actually made things worse - and gave me more false detections

The last thing I tried was creating another category, called "object-background" and took all the false matches and assigned them to this new category. This approach worked pretty well, but I view it as a hack.

I guess to summarize my main question is - what is the best method for reducing false positives within the current tensorflow object detection framework? Would SSD be a better approach since that seems to have a hard example miner built into it by default in the configs?

thanks

1
This is a research problem, not a programming question. - Dr. Snoopy
in the tensorflow github, they advise to ask questions on stackoverflow. There are well known methods for reducing false positives (such as the mentioned hard example mining - ohem) but there doesn't seem to be a lot of documentation on using it. - Nadav Ben-Haim
Sure, but Stack Overflow is only about programming, it is too common that people ask off-topic questions, specially when related to Machine Learning, but that does not make it okay. Your question is better suited to stats.stackexchange.com - Dr. Snoopy
Also note that if you rewrite your question to ask "how can I use TF Object Detection API's hard example mining" that would be more programming specific and probably on-topic here. - Dr. Snoopy
did not know about stats, i will ask it there thanks - Nadav Ben-Haim

1 Answers

2
votes

After some more investigation I actually was able to get the hard example miner with faster rcnn working. I had a bug where I wasn't actually inserting background images into the tf records file.

I think when training a single object detector (category with one model) it's most crucial to add background images if you want to have good precision/recall. If you just have a few thousand examples of the object, that won't be nearly enough images for the model to learn all the various background noise you will be sending when actually using the model for your application