using libsvm to understand SVM questions

Question

Here I post two figures as well as some questions with them, can anyone help me to solve them, cause I really get stuck in it?

1. Training a simple linear SVM:
% svm-train -t 0 -c 100 data0 data0.model
% python drawBoundary.py data0
where: 
-t 0    -- use a linear kernel
-c 100  -- set "C" = 100, which means "overfit a lot"

Then here comes the resulting plot: enter image description here

This is an easily separable dataset, which is reflected by the small number of support vectors. In the plot, the SVs are drawn big (and are on the margin, the dashed line one unit away from the decision boundary, the solid line).

Then here comes the first question:

Q1: You should have found that it takes 3 support vectors. Could you have fewer (eg., 2) support vectors here? And why?

My answer is NO, 3 is the smallest number. But that's only my intuition. I don't know why. Could somebody explain the reason to me?

2.
% svm-train -t 2 -c 100 -g 100 data0 data0.model
% python drawBoundary.py data0
where:
-t 2 means RBF and -g100 means gamma=100
A gamma of 100 means that you have to be really close to a point to have a kernel value that's non-zero.

Here is the resulting plot: enter image description here

Then here is the second question:

Q2: Why do you get these little blobs? How high do you have to turn gamma up in order to get a little decision boundary around each example (i.e. each decision boundary surrounds exactly one example)?

For this question, I completely get lost.

Nuageux Nuageux · Accepted Answer · 2017-05-08T09:03:25

First question should be 3 points because you need two points on the same side to draw a line, then one point on the other side that will serve to plot the second line (which is parallel to the first one). These 3 points will be chosen to have the maximum margin (i.e. so the parallel line have the maximum separation/distance). If you don't use 3 points, then there is always a way to increase this distance between the two parallel lines so it is not the solution we are looking for.

For the second question, I guess to have to try several gamma values to answer. Gamma is usually taken in a set of values that are power of 10, (not sure that's proper English so:) e.g. {1 10 100} = {10^0 10^1 10^2}. And that we choose by cross-validation to fit our data. Doing so enable us to have an efficient SVM that do not overfit nor underfit.

Hope this helps.

using libsvm to understand SVM questions

1 Answers