Interpreting libsvm epsilon-SVR result

Question

I tried to train & cross validate a set of data with 8616 samples using epsilon SVR. Among the datasets, I take 4368 for test, 4248 for CV. Kernel type = RBF kernel. Libsvm provides a result as shown below.

optimization finished, #iter = 502363
nu = 0.689607
obj = -6383530527604706.000000, rho = 2884789.960212
nSV = 3023, nBSV = 3004

This is a result gotten by setting

-s 3 -t 2 -c 2^28 -g 2^-13 -p 2^12

(a) What does "nu" means? Sometimes I got nu = 0.99xx for different parameter.

(b) It seems that "obj" is surprisingly large. Does it sounds correct? Libsvm FAQ said this is "optimal objective value of the dual SVM problem". Does it means that this is the min value of f(alpha)?

(c) "rho" is large too. This is the bias term, b. The dataset labels (y) consist of value between 82672 to 286026. So I guess this is reasonable, am I right?

For training set,

Mean squared error = 1.26991e+008 (regression)
Squared correlation coefficient = 0.881112 (regression)

For cross-validation set,

Mean squared error = 1.38909e+008 (regression)
Squared correlation coefficient = 0.883144 (regression)

Using the selected param, I have produced the below result

kernel_type=2 (best c:2^28=2.68435e+008, g:2^-13=0.00012207, e:2^12=4096)
NRMS: 0.345139, best_gap:0.00199433
Mean Absolute Percent Error (MAPE): 5.39% 
Mean Absolute Error (MAE): 8956.12 MWh
Daily Peak MAPE: 5.30%

The CV set MAPE is low (5.39%). Using Bias-Variance test, the difference between train set MAPE and CV set MAPE is only 0.00199433, which mean the param seems to be set correctly. But I wonder if the extremely large "obj", "rho" value is correct....

I am very new to SVR, do correct me if my interpretation or validation method is incorrect/insufficient.

Method to calculate MAPE

train_model = svmtrain(train_label, train_data, cmd); 
[result_label, train_accuracy, train_dec_values] = svmpredict(train_label, train_data, train_model); 
train_err = train_label-result_label; 
train_errpct = abs(train_err)./train_label*100; 
train_MAPE = mean(train_errpct(~isinf(train_errpct)));

how did you get MAPE value? I mean I calculate it by hand however reversing scaling points is hard and do not give exact values. — ARAT
I calculate it as the percentage of deviation of forecast value from the actual value, see above last part of edited portion. — twfx

marcopolo marcopolo · Accepted Answer · 2016-02-12T11:15:25

The objective and rho values are high because (most probably) the data were not scaled. Scaling is highly recommended to avoid overflow; the overflow risk also depends on the type of kernel. Btw, when scaling the training data, do not forget to also scale the test data, which is most easily accomplished by scaling all data first, and then splitting them into a training and test set.

Interpreting libsvm epsilon-SVR result

1 Answers