OpenCV Linear SVM not training

Question

I've been stuck on this for some time now. OpenCV's SVM implementation doesn't seem to work for a linear kernel. I'm fairly sure there's no bug in the code: when I change the kernel_type to RBF or POLY, keeping everything else as is, it works.

The reason I say it doesn't work is, I save the generated model and check it out. It shows support vector count as 1. Which is not the case in RBF or POLYnomial kernels.

There's nothing special about the code in itself, I've used OpenCV's SVM implementation before, but never a linear kernel. I tried setting the degree to 1 in a POLY kernel and it results in the same model. Which makes me believe something is buggy here.

The code structure, if required:

Mat trainingdata;    //acquire from files. done and correct.
Mat testingdata;     //acquire from files. done and correct again.
Mat labels;          //corresponding labels. checked and correct.

SVM my_svm;
SVMParams my_params;
my_params.svm_type = SVM::C_SVC;
my_params.kernel_type = SVM::LINEAR;    //or poly, with my_params.degree = 1.
my_param.C = 0.02;    //doesn't matter if I set it to 20000, makes no difference.

my_svm.train( trainingdata, labels, Mat(), Mat(), my_params );
//train_auto(..) function with 10-fold cross-validation takes the same time as above (~2sec)!
Mat responses;
my_svm.predict( testingdata, responses );
//responses matrix is all wrong.

I have 500 samples from one class and 600 from the other class to test, and the correct classifications I get are: 1/500 and 597/600.

Craziest part: I have done the same experiment with the same data on libSVM's MATLAB wrapper, and it works. Was just trying to do an OpenCV version of it.

Are you sure the data's linearly separable? The fact that it works with fancier kernels strongly suggests non-linearly separable data... Are you training libSVM using linear kernels? — Nicu Stiurca
Do you know the formulation of SVM? There should be difference when you set C to 0.02 or 20000. Be sure that you set it correctly. It can't be same for all parameters. Also you should set epsilon parameter to 0.001, the default is something else AFAIK. One last thing, what happens when you run train_auto() instead of train()? — guneykayim
Try adapt standard example examples/cpp/points_classifier.cpp there is also svm classifier. I've checked it, and it worked. — Andrey Smorodov
@SchighSchagh I duplicated the experiment from what I did in libSVM. It works like a charm there. — a-Jays
@guneykayim I'm aware of the formulation of SVM and I understand what C does to the model. Which is why I find it surprising that the value of C is not making any difference at all. train_auto() does nothing at all. Gives me C=1. I tried. — a-Jays

Temak Temak · Accepted Answer · 2015-04-28T14:16:58

It is not a bug that you always get only one support vector with linear CvSVM.

OpenCV optimizes a linear SVM down to one support vector.
The idea here is that the support vectors define the classification margin, and to do the actual classification only the separating hyperplane is needed and it can be defined by only one vector.

Parameter C doesn't matter if your training data is linearly separable. Maybe it is your case.

OpenCV Linear SVM not training

1 Answers