If no scoring is specified, the estimator passed should have a 'score' method

Question

I am doing hyperparameter tuning and I wrote this code(from this tutorial -https://machinelearningmastery.com/grid-search-hyperparameters-deep-learning-models-python-keras/?unapproved=524264&moderation-hash=83f45bd57dd6c1c5e37699b257905830#comment-524264- )

from sklearn.model_selection import GridSearchCV
# fix random seed for reproducibility
seed = 7
np.random.seed(seed)

# define the grid search parameters
batch_size = [10, 20, 40, 60, 80, 100]
epochs = [10, 50, 100]
param_grid = dict(batch_size=batch_size, epochs=epochs)
grid = GridSearchCV(estimator=model, param_grid=param_grid, n_jobs=-1, cv=3)
grid_result = grid.fit(scaled_X, y)
# summarize results
print("Best: %f using %s" % (grid_result.best_score_, grid_result.best_params_))
means = grid_result.cv_results_['mean_test_score']
stds = grid_result.cv_results_['std_test_score']
params = grid_result.cv_results_['params']
for mean, stdev, param in zip(means, stds, params):
    print("%f (%f) with: %r" % (mean, stdev, param))

And this is the error I got-

TypeError                                 Traceback (most recent call last)
<ipython-input-39-3821841029c0> in <module>
     11 param_grid = dict(batch_size=batch_size, epochs=epochs)
     12 grid = GridSearchCV(estimator=model, param_grid=param_grid, n_jobs=-1, cv=3)
---> 13 grid_result = grid.fit(scaled_X, y)
     14 # summarize results
     15 print("Best: %f using %s" % (grid_result.best_score_, grid_result.best_params_))

~\AppData\Local\Continuum\anaconda3\lib\site-packages\sklearn\model_selection\_search.py in fit(self, X, y, groups, **fit_params)
    607 
    608         scorers, self.multimetric_ = _check_multimetric_scoring(
--> 609             self.estimator, scoring=self.scoring)
    610 
    611         if self.multimetric_:

~\AppData\Local\Continuum\anaconda3\lib\site-packages\sklearn\metrics\scorer.py in _check_multimetric_scoring(estimator, scoring)
    340     if callable(scoring) or scoring is None or isinstance(scoring,
    341                                                           str):
--> 342         scorers = {"score": check_scoring(estimator, scoring=scoring)}
    343         return scorers, False
    344     else:

~\AppData\Local\Continuum\anaconda3\lib\site-packages\sklearn\metrics\scorer.py in check_scoring(estimator, scoring, allow_none)
    293                 "If no scoring is specified, the estimator passed should "
    294                 "have a 'score' method. The estimator %r does not."
--> 295                 % estimator)
    296     elif isinstance(scoring, Iterable):
    297         raise ValueError("For evaluating multiple scores, use "

TypeError: If no scoring is specified, the estimator passed should have a 'score' method. The estimator <keras.engine.sequential.Sequential object at 0x0000025F8892C248> does not.

I have tried the answer from-

Scikit-learn TypeError: If no scoring is specified, the estimator passed should have a 'score' method

Rajith Thennakoon Rajith Thennakoon · Accepted Answer · 2020-03-12T00:24:53

I think you need to specify type of score to use in GridSearch.Because GridSearch maximizes a score over the grid of parameters.For example classiication problem you can use f1 score,precision,recall scores. If scoring is None GridSearch will use estimator’s score method.

from sklearn.metrics import make_scorer
from sklearn.metrics import accuracy_score, precision_score, recall_score

from sklearn.model_selection import GridSearchCV
scorers = {
        'precision_score': make_scorer(precision_score),
        'recall_score': make_scorer(recall_score),
        'accuracy_score': make_scorer(accuracy_score)
        }
grid_search_cv=GridSearchCV(model,param_grid,verbose=1,cv=3,scoring=scorers,refit="precision_score")

check about scoring parameter documentation in GridSearch

you can use built in scoring parameters or define your own function.

If no scoring is specified, the estimator passed should have a 'score' method

2 Answers