Select optimized parameters for libsvm-linear kernel

Question

I use RBF kernel for ML with libsvm. I am looking into exploring other kernels for my dataset.

There are multiple parameters to optimize each for specific kernel. C and g parameters are used for grid search for selecting optimal combination of Cost and gamma.

-d degree : set degree in kernel function (default 3)
-g gamma : set gamma in kernel function (default 1/num_features)
-r coef0 : set coef0 in kernel function (default 0)
-c cost : set the parameter C of C-SVC, epsilon-SVR, and nu-SVR (default 1)
-n nu : set the parameter nu of nu-SVC, one-class SVM, and nu-SVR (default 0.5)
-p epsilon : set the epsilon in loss function of epsilon-SVR (default 0.1)
-m cachesize : set cache memory size in MB (default 100)
-e epsilon : set tolerance of termination criterion (default 0.001)
-h shrinking : whether to use the shrinking heuristics, 0 or 1 (default 1)
-b probability_estimates : whether to train a SVC or SVR model for probability estimates, 0 or 1 (default 0)
-wi weight : set the parameter C of class i to weight*C, for C-SVC (default 1)

I want to know relevant parameters for each individual kernel. Since there are multiple parameters for selection. Ex: C and g for RBF kernel. Also provide grid size and range of parameters Ex: 10^-3 to 10^11 for C and 10^3 to 10^-13 for g

My perl grid generator for RBF kernel:

    for ( $i = -3; $i <= 11; $i += 1 ) {

    for ( $j = 3; $j >= -13; $j += -1 ) {

        my $a = 2**$i;
        my $b = 2**$j;

        $output = "svm-train -c $a -g $b -v 5 $ARGV[0]";

        print  "$output >& ${ARGV[0]}_${a}_${b}.out \n";

    }
}

lejlot lejlot · Accepted Answer · 2016-02-06T16:20:44

libsvm supports four kernels: linear, poly, rbf and sigmoid (which is actually not a valid kernel).

linear: no parameters
poly: gamma (>0, float), coef0 (float), degree (>1, int)
rbf: gamma (>0, float)
sigmoid: gamma (>0, float), coef0 (float)

you cannot really provide general parameters grids as they are data dependent.

C is an SVM parameter, thus it needs to be fitted always. Remaining parameters are not kernel specific and you should not worry about them.

Select optimized parameters for libsvm-linear kernel

1 Answers