python - SciKit One-class SVM classifier training time increases exponentially with size of training data

Question

I am using the Python SciKit OneClass SVM classifier to detect outliers in lines of text. The text is converted to numerical features first using bag of words and TF-IDF.

When I train (fit) the classifier running on my computer, the time seems to increase exponentially with the number of items in the training set:

Number of items in training data and training time taken: 10K: 1 sec, 15K: 2 sec, 20K: 8 sec, 25k: 12 sec, 30K: 16 sec, 45K: 44 sec.

Is there anything I can do to reduce the time taken for training, and avoid that this will become too long when training data size increases to a couple of hundred thousand items ?

Kernel-SVM is between O(n^2) and O(n^3) (depends on time/memory trade-off). There is not much you can do. Nonlinear SVMs are not great for large data! Linear SVMs can be, but it seems the underlying implementation is libsvm only (and not able to use liblinear or SGD). — sascha

Flair Flair · Accepted Answer · 2017-08-29T00:10:21

Well scikit's SVM is a high-level implementation so there is only so much you can do, and in terms of speed, from their website, "SVMs do not directly provide probability estimates, these are calculated using an expensive five-fold cross-validation."

You can increase your kernel size parameter based on your available RAM, but this increase does not help much.

You can try changing your kernel, though your model might be incorrect.

Here is some advice from http://scikit-learn.org/stable/modules/svm.html#tips-on-practical-use: Scale your data.

Otherwise, don't use scikit and implement it yourself using neural nets.

python - SciKit One-class SVM classifier training time increases exponentially with size of training data

2 Answers