XGBoost features with more feature importance giving less accuracy

Question

I have six features for my model f1,f2,f3,f4,f5 and f6. And feature importance scores are in order

f1>f2>f3>f4>f5>f6

but rmse of model with features f1,f4 and f5 is less than rmse of model with features f1,f2,f3,f4,f5 and f6 or model with features f1,f2,f3. Any possible reason for this?

your question is little bit harder to understand than necessary as you are switching between accuracy and rmse — CAFEBABE
@CAFEBABE sorry that was a typo, I dont know how I missed it — d.c

CAFEBABE CAFEBABE · Accepted Answer · 2020-06-27T09:15:16

It is hard to guess without the data. However, typically this results from correlated features. So if f2==f1 it would be the case that adding f2 to a model which contains already f1 does not provide any value. However, adding an uncorrelated feature, e.g. f4 can add a lot even so f2 > f1

XGBoost features with more feature importance giving less accuracy

1 Answers