Pecuiliar output with Liblinear

Question

I am facing a very peculiar problem with lib-linear package. I have two levels (+1, -1). Say I have only one feature which takes values $x_1$, $x_2$,..., $x_n$ for n points. It classifies well giving some positive weight $w*$ and cost C say for example. Now if I stack $1$ to the previous feature to make a new feature vectors [1 x_i] i=1, 2, ...,n; Now with this new problem lib-linear gives the following: a weight vector [w_1 -w_2]; w_i>0 i.e. weights to 1 is w_1 and to x is w_2. Cost C1 much greater than previous cost C.

I understand that new feature (1) has no variation throughout and hence the weight to it should automatically go zero.

It is a minimization problem so it should give w_1~0 so that now the cost C1 is at most equal to C.

Can anyone help?

Answer 1

Since you have a constant input dimension, its contribution in the decision function will also be constant. LIBLINEAR's decision function is

f(x)=sign(w^T*x-rho)

My guess is that your new model corrects for the extra term (due to non-zero w_1) through rho. I can't say I have a good idea as to why w_1 was not minimized to zero, though. Are the predictions of both models equal?