How to plot miss rate vs false positive rate?

2
votes

i have a problem to plot a curve for miss rate vs false positive rate to analyze the performance of my proposed system (as sampled on picture below). I have two samples dataset for positive and negative sample. I want to plot the performance of my system whether it can classify people or non people with this curve.

As far as I know, I need to get True Positive and False Positive values after the classification, but I am not sure yet how to plot the curve. Any one can help please??

Sample of curve

3
matlabimage-processingclassificationfalse-positive
What are you plotting against? Miss rate against false positive (one on each axis), both against some other variable (some setting of your system), etc... – nkjt
I want to plot the curve as illustrated on picture above, miss rate against false positive per image to analyze the performance of my system. But i have no idea to scratch it. I want to compare my system with two other methods. @nkjt – Indrasyach
If you've got those two bits of data, just semilogx to get the log on the x like in the image. Is plotting or calculating the data your issue? – nkjt
my problem concerns on both issues. I dont know how to obtain the computation. Could you explain me how to do the calculation of data? I have been looking through some literatures but found nothing. @nkjt – Indrasyach
You need some "ground truth" (e.g. testing image set, such as INRIA ). Miss rate = percentage of people (or whatever you're looking for) which aren't found. "False positives" = number of 'people' found who aren't people, per image. As you increase the sensitivity/reduce thresholds for a "hit", you usually get less misses but more false positives. – nkjt

3 Answers

1
votes

Since MATLAB R2017a, you can use evaluateDetectionMissRate function.

[logAverageMissRate,fppi,missRate] = evaluateDetectionMissRate(detectionResults,groundTruthData)

This function returns data points for plotting the log MR-FPPI curve. (MR: Miss-Rate, FPPI: False Positive Per Image).

For an example of its usage, type doc evaluateDetectionMissRate command in MATLAB or go to here.

0
votes

There are two type of bounding boxes in the object detection, the boxes that data-set labeled theme as object and second the boxes that your algorithm detects.

If your bbox have huge intersection with data-set bbox, it is okey.

If your bbox have NOT intersection with data-set bbox it is False Posative.

And we call All data-set bbox without intersection with your bbox in an image MISS Rate. and after calculating these numbers, plotting these values is straight forward.

-1
votes

You can use the following GitHub repo for plotting MR vs FPPI. It may seem like the code only calculates the mAP but it does much more than that. It also calculates the miss-rate, false positives per image and the log-average miss rate. All these are computed in the main.py file present in the repo(line 81) but are not plotted. All you have to do is just plot the MR vs FPPI using matplotlib (or any other module). Just follow the ReadMe file to get started. Hope this helps!