Why does opencv FREAK extractor remove so many keypoints, specifically using ORB detector

Question

I am using OpenCV 2.4.3 c++ interface to find matching points between two images. The first attempt was using SURF. The only problem is the consuming time, so I tried the new FREAK extractor. Using SURF for detection and FREAK for description, I realized that FREAK reduces the number of keypoints to almost half the detected, and the resulting matches were not enough. That is the reason, I tried FAST to obtain more keypoints. The results:

1. SURF detector, SURF extractor, BFMatcher crosscheck true, RANSAC: 70keypoints first image, 50 keypoints second image, 200ms. 250ms. 15ms. 15ms.
2. SURF detector, FREAK extractor, BFMatcher crosscheck true, RANSAC: 39 keypoints first image, 30 keypoints second image (after FREAK), 200ms., 50 ms. , 0ms., 0ms. It results that there too few good matchings.
3. FAST detector, FREAK extractor, BFMatcher crosscheck true, RANSAC: 120 keypoints, 90 keypoints, (69 and 48 keypoints after FREAK), 10ms., 450 ms., 15 ms., 10 ms.

After that, I used ORBFeatureDetector, and it is obtaining the same number of keypoints than FAST, but after FREAK extractor, the resulting keypoints are 0 for each image. Am I doing something wrong? Are ORB keypoints different from those obtained from FAST? Maybe I could open another question for this, but I have last one. What could it be the best combination of detector/extractor to obtain the same results than my first experiments using SURF, but reducing the processing time? Because as I obtain more keypoints the extractor part is also more time consuming, although I use FREAK.

Answer 1

FREAK removes points if it can not generate a descriptor for it, many times this occurs in the border of the image as it cannot generate a descriptor if it falls out of the boundary image. I avoid this issue by applying a ROI before extraction.

I also use FAST combined with FREAK and I get the best results, but I still have the problem of reducing extractor time, which is too high for me.

Answer 2

Actually, you used parameter cross check = true. this is also a reason why plenty of your points are eliminated. This parameter , when true, is expensive from a computationnal point of view. It is used to avoid pairs of descriptors that do not perfectly match during the matching process.

If you have two sets of descriptors D1 and D2, then, this parameters allows only pairs that are commonly match in the D1-> D2 and D2->D1 matching directions.

then, it all depends on your application, maybe you do not need that much matching precision...

Best regards.

Alex

Answer 3

Apart from the removal of border-points, as Jav_Rock suggests, the huge (inconsistent?!) decrease of points really depends on the size-parameter you stored in keyPoint. Even if you set scaleNormalized to false, with a float value for the size-parameter close to zero FREAK will just discard this keyPoint. (but I don't seem to be able to figure out why, as the size-parameter of the keyPoint is only used if scaleNormalized is true: source)

So be sure to set the size parameter to something bigger than zero (e.g. one) if you aren't using scaleNormalization.
And to interpert it as a value with the unit 'pixelsize' (when using scaleNormalization).
Btw. the minimum keypoint size is 7 by default.

Hope this helps...

Answer 4

FAST is only a keypoint detector (no descriptor). If you combine FAST and use for description (multi-scale) BRIEF you will get ORB.