I'm cross validating a sklearn classifier model and want to quickly obtain average values of precision, recall and f-score. How can I obtain those values?
I don't want to code the cross validation by myself, instead I'm using the function cross_validation.cross_val_score. Is it possible to use this function to obtain the intended averaged values for each label, by supplying an adequate scoring function?
You can consider using all the method in sklearn.metrics package.
I think this method could do the work you expect. It gives you a 2D array with one row for each target unique value and columns for precision, recall, fscore and support.
For fast logging you can use classification_report too.
For precision and recall, in the metrics package there is a function called precision_recall_curve that should do what you're looking for.
Assuming you have a trained classifier clf, test exemplars X and test targets Y, then you'll need to pass the targets and predicted class probabilities. The following example will find precision and recall for a two class problem.
probs = clf.predict_proba(X)[:,1]
precision, recall, thresholds = precision_recall_curve(Y, probs)
The F-score can be found using a different function in the metrics package, f1_score. This is used in a similar way, but requires the predicted class membership as an argument, rather than the probability of membership.
