Different kernels for different features - scikit-learn SVM

Question

I am trying to build a classifier using sklearn.svm.SVC but I would like to train the kernel separately on different subsets of features to better represent the feature space (as described here).

I have read the User Guide page and I understand that I can create kernels that are sums of individual kernels or feed into the SVC a precomputed kernel (kernel = 'precomputed'), but I do not understand how I apply different kernels to different features? Is there a way to implement this in sklearn?

I have found a way to calculate kernels in sklearn (https://scikit-learn.org/stable/modules/gaussian_process.html#gp-kernels), and so I could calculate the kernel on each set separately. However, once I output the distance matrix, I am not sure how I would use it to train the SVM.

Do I have to create a custom kernel like:

if feature == condition1:
   use kernel X
else:
   use kernel Y

and add it to the SVM?

Or is there any other python libraries I could be using for this?

alift alift · Accepted Answer · 2020-05-29T19:52:51

You are referring to the problem of Multiple Kernel Learning (MKL). Where you can train different kernels for different groups of features. I have used this in a multi-modal case, where I wanted different kernels for image and text.

I am not sure if you actually can do it via scikit-learn.

There are some libraries provided on GitHub, for example, this one: https://github.com/IvanoLauriola/MKLpy 1

Hopefully, it can help you to achieve your goal.

Different kernels for different features - scikit-learn SVM

2 Answers