I developed a pipeline using XGBoost which returned me a best estimator. However, trying to use this best estimator to predict my test set the following error is raised: "ValueError: Specifying the columns using strings is only supported for pandas DataFrames".
Below is my code for the pipeline that I have used: Note: ct is just ColumnTransformer using SimpleImputer and OneHotEncoder for categorical columns and SimpleImputer and StandardScaler for numerical columns
ml_step_1 = ('transform', ct)
ml_step_2 = ('pca', PCA())
xgb = ('xgb', XGBRegressor())
xgb_pipe = Pipeline([ml_step_1, ml_step_2, xgb])
xgb = RandomizedSearchCV(xgb_pipe, xgb_param_grid, cv=kf, scoring='neg_mean_absolute_error');
xgb.fit(train_full_features, train_full_target);
Running the following pipeline, here is the best estimator that I got:
Best XGBoost parameters: {'xgb__silent': True, 'xgb__n_estimators': 1000, 'xgb__max_depth': 4, 'xgb__learning_rate': 0.09999999999999999, 'transform__num__imputer__strategy': 'median', 'transform__cat__imputer__strategy': 'most_frequent', 'pca__n_components': 68}
Now, I called this best estimator and did the following:
test_full_imp = pd.DataFrame(xgb.best_estimator_.named_steps['transform'].transform(test_full))
test_final = xgb.best_estimator_.named_steps['pca'].transform(test_full_imp)
predictions = xgb.best_estimator_.predict(test_final)
