Scikit-learn Custom Scoring Function

Question

I'm currently using the development branch of scikit-learn: 0.15-git.

Trying to initialize a RidgeClassifierCV object with a custom scoring function is currently failing with error message TypeError: __init__() got an unexpected keyword argument 'scoring'.

model = RidgeClassifierCV(scoring=make_scorer(score_func))

According to the docs, RidgeClassifier takes a scoring parameter. But according to the function signature, the argument is score_func. However, passing the output of sklearn.metrics.maker_scorer as score_func also fails. Any ideas?

The end goal is to get RidgeClassifierCV working with multiclass (one vs. all) roc auc scores used for the scoring function.

In the parameter doc they say "scoring", but the function definition only has a "score_func" optional parameter... ? — Gregor
Okay, sorry, You mention this already. I clicked on the link at thought I had maybe solved your issue — Gregor
1) This is the way the project advances fastest - users discovering issues and reporting them. Please continue like that. 2) There are some extremely efficient people contributing to this project, one of which you just saw in action. — eickenberg

eickenberg eickenberg · Accepted Answer · 2014-06-18T21:47:27

score_func as opposed to the now standard scikit-learn scoring objects take as arguments y_true, y_pred, instead of estimator, X, y_true. So if you have written your own scoring function that can work with predictions coming out of the Ridge classifier, then this is the signature you need.

Although score_func is deprecated, it seems to be the only option in the current state of the master branch. This will almost surely change with the 0.15 release and scoring objects will be available. The fact that this is documented wrongly is a discrepancy that should also be fixed then.

Scikit-learn Custom Scoring Function

1 Answers