Hyperparameter tuning using hyperopt sklearn with RandomForestClassifier

Question

I'm currently trying to optimize hyperparameters using either RandomizedSearchCV or GridSearchCV. For comparison I want to try hyperopt which is also available as hyperopt-sklearn (https://github.com/hyperopt/hyperopt-sklearn). Unfortunately there is not much documentation, so I'm not sure how to use it.

Is there a similar way to use hyperopt like I'm doing now with random and grid search?

skf = StratifiedKFold(n_splits=5, random_state=42)
params_randomSearch = {"min_samples_leaf": np.arange(1,30,1),
              "min_samples_split": np.arange(2,20,1),
              "max_depth": np.arange(2, 20, 1),
              "min_weight_fraction_leaf": np.arange(0. ,0.4, 0.1),
              "max_features" : ['auto', 'sqrt', 'log2', None],
              "criterion" : ['entropy', 'gini']}
scoring = {'Accuracy' : make_scorer(accuracy_score), 'Recall' : 'recall_weighted', 'Kappa' : make_scorer(cohen_kappa_score)}

rs = RandomizedSearchCV(DecisionTreeClassifier(random_state=42), param_distributions=params_randomSearch, scoring = scoring, cv = skf, refit = 'Accuracy', n_iter=150, n_jobs=-1, random_state=42)
        rs.fit(x_train, y_train)
        y_predict = rs.best_estimator_.predict(x_test)
        acc = accuracy_score(y_test, y_predict)

According to the documentation/example on github, it should be something like this:

estim = HyperoptEstimator(classifier=random_forest('RF1'))
estim.fit(x_train, y_train)

This results in the following error:

TypeError: 'generator' object is not subscriptable

Another question I have is if there is any integrated cross validation option like with RandomizedSearchCV or GridSearchCV?

Christian Christian · Accepted Answer · 2018-04-17T09:55:37

I could solve it by myself. It seems that there is some conflict with networkx v2. A downgrade to 1.11 solves the problem (https://github.com/hyperopt/hyperopt/issues/333)

Hyperparameter tuning using hyperopt sklearn with RandomForestClassifier

1 Answers