I am solving for a classification problem using Python's sklearn + xgboost module. I have a highly imbalanced data with ~92% of class 0 and only 8% class 1. The train data set can be download here. http://www.filedropper.com/kangarootrain
I cant use numclaims and claimcst0 variables in this dataset. The variables in this dataset are: id,claimcst0,veh_value,exposure,veh_body,veh_age,gender,area,agecat,clm,numclaims
gender, area, and agecat are categorical variables and rest are continuous variables. Id is the id for that record.
Top 10 records are
id,claimcst0,veh_value,exposure,veh_body,veh_age,gender,area,agecat,clm,numclaims
1,0,6.43,0.241897754,STNWG,1,M,A,3,0,0
2,0,4.46,0.856522757,STNWG,1,M,A,3,0,0
3,0,1.7,0.417516596,HBACK,1,M,A,4,0,0
4,0,0.48,0.626974524,SEDAN,4,F,A,6,0,0
5,0,1.96,0.089770031,HBACK,1,F,A,2,0,0
6,0,1.78,0.25654335,HBACK,2,M,A,3,0,0
7,0,2.7,0.688128611,UTE,2,M,A,1,0,0
8,0,0.94,0.912765859,STNWG,4,M,A,2,0,0
9,0,1.98,0.157753423,SEDAN,2,M,A,4,0,0
I tried several methods to predict the 'clm' which is my target variables. I tried knn, RF, svm, nb. I even tried to subsample data. But whatever I do does not make the predictions better. With trees/boosting I am getting ~93% accuracy but only because I am predicting all the 0's correctly.
The model is incorrectly predicting all the 1s as 0s too.
Any help would be really helpful. This is the basic code I tried for NB.
from sklearn.naive_bayes import GaussianNB
clfnb = GaussianNB()
clfnb.fit(x_train, y_train)
pred = clfnb.predict(x_test)
#print set(pred)
from sklearn.metrics import accuracy_score, confusion_matrix
print accuracy_score(y_test, pred)
print confusion_matrix(y_test, pred)
0.92816091954
[[8398 0]
[ 650 0]]