Training error rate in `caret`

Question

I don't know how to extract the training error (i.e., the error obtained on the training set) using the caret package in R. For example, I have the following model:

data(iris);
library(caret);
model<-train(Species~., data=iris, method='knn', trControl=trainControl(method='cv', number=10), tuneGrid=data.frame(k=20))

What I want to do is to see how well the model performed on the training data.

I know I can get the performance on the test set for each fold using model$results, but that's not what I want. I want to show how the training error is way over-optimistic, but I can't do that. The documentation here:

http://www.inside-r.org/packages/cran/caret/docs/train

states that

results: a data frame the training error rate and values of the tuning parameters.

which is not true, because in my case, model$results$Accuracy is always exactly equal to the value of mean(model$resample$Accuracy). This is the value of the testing error rate. I want the training eror rate. Is there a way to get it?

@phiver I don't really see how that post is helpful. It tells me nothing new and doesn't answer the question. — 5xum

George Dontas George Dontas · Accepted Answer · 2016-03-16T11:29:37

I think that you are asking for something like this:

model3<-train(Species~., data=iris, method='knn', trControl=trainControl(method='none'), tuneGrid=data.frame(k=20))
testPred <- predict(model3, iris)
postResample(testPred, iris$Species)

Accuracy    Kappa 
    0.98     0.97

Training error rate in `caret`

1 Answers