Plot SVM margins using MATLAB and libsvm

Question

I am using svmlib to classify linearly two dimensional non-separable data. I am able to train the svm and obtain w and b using svmlib. Using this information I can plot the decision boundary, along with the support vectors, but I am not sure about how to plot the margins, using the information that svmlib gives me.

Below is my code:

model = svmtrain(Y,X, '-s 0 -t 0 -c 100');

w = model.SVs' * model.sv_coef;
b = -model.rho;
if (model.Label(1) == -1)
    w = -w; b = -b;
end
y_hat = sign(w'*X' + b);

sv = full(model.SVs);

% plot support vectors
plot(sv(:,1),sv(:,2),'ko', 'MarkerSize', 10);

% plot decision boundary
plot_x = linspace(min(X(:,1)), max(X(:,1)), 30);
plot_y = (-1/w(2))*(w(1)*plot_x + b);
plot(plot_x, plot_y, 'k-', 'LineWidth', 1)

The margin is the distance between the decision boundary and the support vectors. How would you like to plot it? — hbaderts

Itamar Katz Itamar Katz · Accepted Answer · 2015-02-17T08:14:07

It depends on what you mean by "the margins". It also depends on what SVM version you are talking about (separable on non-separable), but since you mentioned libsvm I'll assume you mean the more general, non-separable version.

The term "margin" can refer to the Euclidean distance from the separating hyperplane to the hyperplane defined by wx+b=1 (or wx+b=-1). This distance is given by 1/norm(w).

"Margin" can also refer to the margin of a specific sample x, which is the Euclidean distance of x from the separating hyperplane. It is given by

(wx+b)/norm(w)

note that this is a signed distance, that is it is negative/positive, depending on which side of the hyperplane the point x resides. You can draw it as a line from the point, perpendicular to the hyperplane.

Another interesting value is the slack variable xi, which is the "algebraic" distance (not Euclidean) of a support vector from the "hard" margin defined by wx+b=+1 (or -1). It is positive only for support vectors, and if a point is not a support vector, its xi equals 0. More compactly:

xi = max(0, 1 - y*(w'*x+b))

where y is the label.

Plot SVM margins using MATLAB and libsvm

1 Answers