How do i set different threshold to get multiple values for ROC plot

Question

Below is the code I have written to build a SVM model. I am using ROCR package for plotting the ROC plot.

library(e1071)
library(caret)
library(gplots)
library(ROCR)

inTraining <- createDataPartition(data$Class, p = .70, list = FALSE)
training <- data[ inTraining,]
testing  <- data[-inTraining,]

svm.model <- svm(Class ~ ., data = training,cross=10, metric="ROC",type="C-classification",kernel="linear",na.action=na.omit,probability = TRUE)

#prediction and ROC
svm.model$index
svm.pred <- predict(svm.model, testing, probability = TRUE)
c <- as.numeric(svm.pred)
c = c - 1
pred <- prediction(c, testing$Class)
perf <- performance(pred,"tpr","fpr")
plot(perf,fpr.stop=0.1)

I tried following this solution Obtaining threshold values from a ROC curve But, I get a single threshold cutoff of (0.173913 0.673913)

 > head(cutoffs)
  cut      fpr      tpr
1 Inf 0.000000 0.000000
2   1 0.173913 0.673913
3   0 1.000000 1.000000

How do I get multiple thresholds to get different Tpr and fpr rates for plotting a ROC curve?

Calimo Calimo · Accepted Answer · 2018-10-09T06:17:27

This is because you are predicting class labels directly. Your predictions probably look like:

table(svm.pred)
pred
    class1     class2
        28         37

Therefore there is no thresholds to choose from to build the ROC curve.

Try to do a regression instead (in e1071 you'll need to make sure your class labels are numeric):

svm.model <- svm(as.numeric(Class) ~ ., data = training, type="eps-regression", [...])

How do i set different threshold to get multiple values for ROC plot

1 Answers