Support Vector Machine in Torch7

Question

I have based my model upon the following tutorial:

https://github.com/torch/tutorials/tree/master/2_supervised

For the last stage a neural network is used upon the features extracted from the CNN. I want to use a SVM in the final layer. How can I add that to my existing model ?

It has been shown in some papers that SVM seem to function better than neural network as the final layer in the CNN and therefore I wanted to try them out to increase the accuracy of the model. Also SVM's can be use for one class classification which neural networks lack.I need a one class classifier in the end and so the need for adding an SVM to th CNN.

Kindly help

This isn't an official answer, but have you considered rolling your own SVM module? All you'd need to do is write your own forwards and backwards message functions... The hardest part, conceptually, will be taking derivatives of the response function... — StevieP
Also, I'm sure you saw this, but there's a google group question dedicated to this problem (kinda of): groups.google.com/forum/#!topic/torch7/bEDxJgaDlj8 — StevieP
please review my edit -- I may have just disqualified it fro being "acceptable", sorry! — StevieP

StevieP StevieP · Accepted Answer · 2015-06-24T22:30:58

Edit: My old answer was complete rubbish since you cannot code a (linear) SVM as a complete module. Instead, you can think of an SVM as

a 1-layer NN with linear activation on the output node and trained via hinge loss

(see accepted answer's comments.)

This means that in Torch, you can mock up a (linear) SVM with something like

linearSVM = nn.Sequential()
linearSVM:add(nn.Linear(ninputs, 1))
criterion = nn.MarginCriterion()

see the following question in the Torch7 google code mailing list...

Support Vector Machine in Torch7

1 Answers