SVM as a type of instance-based learning?

Question

I have been reading a lot about support vector machines, and in all the books and online articles I've seen, an SVM is categorized as a linear classifier that uses a hyperplane. If data cannot be linearly separable, then the data can be mapped to a higher dimension to enable a linear boundary.

Now, I've come across some articles and slides by Professor Pedro Domingos from U. of Washington, a well-known expert in machine learning. He specifically categorizes SVM as an instance-based machine learning algorithm, similar to kNN. Can anyone explain that to me?

For example, in an article in Communications of the ACM (October 2012), he specifically puts SVM under "instances"-based representation, when most machine learning folks would put it under "hyperplanes" with logistic regression.

Furthermore, in his lecture slides, he gives this reasoning:

Can someone explain this line of reasoning? Why would SVM be an instance-based learner (like KNN) instead of a linear classifier (like logistic regression)?

Answer 1

You can see SVM as an instance-based learning algorithm because you need to memorize the support vectors if you cannot represent the feature space and hence the discriminating hyperplane in this space explicitly.

If you use an RBF kernel your decision boundary will be made up of Gaussian bumps around each support vector, which is close to what you would get with a kNN classifier using the support vectors weighted with the alpha_i.

Answer 2

I think the best would be to ask Prof Domingos directly.

SVMs indeed employ a hyperplane - both are binary after all. However comparing SVM with formulation of LR -- unlike LR, SVM is not probabilistic. HTH, although surely one could argue that all ML is instance-based.