Why does linear svms work well with HoG descriptors?

10
votes

Ok, almost all applications I have seen that use HoG features use linear svm as classifier. Can someone explain for me why linear svm are chosen and why they give good performance?

Are linear svm chosen because it more simple and easier to train than svms that use polynomial or gaussian kernel and using these kernels is not giving significantly better performance?

4
machine-learningcomputer-visionclassificationobject-detection
Training with non-linear kernel does not come always with good performance. Non-linear kernel some times may lead in over training and thus in bad testing performance. This has to do sometimes with how complicated your data is. Usually, not complex data in combination with a RBF kernel may harm the test accuracy (overtraining, not trainning model generalization in out of sample objects). So kernel decision and good calibration of them can affect the performance significantly. I hope that helps! – Darkmoor
@endif Interesting question. Try this: "Why do linear SVMs trained on HOG features perform so well?" arxiv.org/abs/1406.2419 – Bull

4 Answers

1
votes

For more information read journal paper named, "Why do linear SVMs trained on HOG features perform so well? " Hilton Bristow, Simon Lucey (2014)

0
votes

Linear or Non-linear is not a question of HOG or any other feature. It is simply related to number of instances + number of clusters + number of feature dimensions. In general Linear models are preferential for datasets including more feature dimensions than the instances. If the case is reversal, than you should go for non-linear like kernel SVM since it implicitly project your data into another space where again your instances are represented with more dimensions.

In most cases, you get very good number of feature dimensions by applying HOG to images. Therefore you can simply use linear models. However if you have 100000... classes and 10000000... images then HOG + Linear model will insuffice. Therfore no one for instance in ImageNet challenge uses HOG with linear SVM.

0
votes

Personally I've never worked with Histogram of Gradients but in your case I would evaluate if your HoG data is linearly discriminated. I wouldn't just assume that if everybody uses a linear classifier for HoG is because it you should. Evaluate that assertion, critically.

Try this: project a HoG dataset using LDA and then do a scatter plot of the transformed feature space. Check if it is possible to use a maximum margin hyperplane(s) to discriminate between the classes.

0
votes

It is the speed that really matters. Kernel SVM can get better performance in detection no matter what feature you use. But kernel SVM is time-consuming, especially for sliding window detectors in which the classifier is evaluated many times. So linear SVM is often chosen in object detection. HOG is a good descriptor for object detection, and good performance can be achieved with linear SVM. One can expect even better performance with kernel SVM, if the computational complexity is not considered.