Making ROC curves with results from cross_validate?

Question

I am running 5 fold cross validation with a random forest as such:

from sklearn.ensemble import RandomForestClassifier

from sklearn.model_selection import cross_validate

forest = RandomForestClassifier(n_estimators=100, max_depth=8, max_features=6)

cv_results = cross_validate(forest, X, y, cv=5, scoring=scoring)

However, I want to plot the ROC curves for the 5 outputs on one graph. The documentation only provides an example to plot the roc curve with cross validation when specifically using StratifiedKFold cross validation (see documentation here: https://scikit-learn.org/stable/auto_examples/model_selection/plot_roc_crossval.html#sphx-glr-auto-examples-model-selection-plot-roc-crossval-py)

I tried tweeking the code to make it work for cross_validate but to no avail.

How do I make a ROC curve with the 5 results from the cross_validate output being plotted on a single graph?

Thanks in advance

Answer 1

cross_validate is a Model validation tool rather than a splitter class. You need to choose the splitter class which is right for you. You are probably after KFold. Something like this:

from sklearn.model_selection import KFold
cv = KFold(n_splits=5)