Variable Importance for caret loclda method

Question

I am trying to compute the variable importance out of a caret 'loclda' model.

My training dataset contains 20 numeric predictors and 4 discrete outcome classes (2500 samples).

I built the model and computed the variable importance with the following commands:

# fit the model
control <- trainControl(
                    method="repeatedcv", 
                    number=10, 
                    repeats=3,
                    savePredictions='final',
                    summaryFunction=multiClassSummary,
                    returnData=F,
                    trim=T,
                    allowParallel=F
                )

my_loclda_model <- train(MyClass ~ ., data=trainData, method='loclda', trControl=control, importance=T)

# compute variable importance
varImp(my_loclda_model)
Error : is.list(x) is not TRUE

traceback()
7: stop(msg, call. = FALSE, domain = NA)
6: stopifnot(is.list(x), is.list(val))
5: modifyList(data, lapply(data[, fc], as.numeric))
4: asNumeric(x)
3: filterVarImp(x_dat, y_dat, nonpara = nonpara, ...)
2: varImp.train(model_name)
1: varImp(model_name)

From caret user manual we can read the following:

For models that do not have corresponding varImp methods, see filterVarImp

Assuming loclda model does not have a varImp method, I tried to use the filterVarImp function directly from the final model:

str(my_loclda_model$finalModel)
List of 12
 $ learn           : num [1:2500, 1:20] -0.404 -0.336 -0.655 -0.618 -0.151 ...
  ..- attr(*, "dimnames")=List of 2
  .. ..$ : chr [1:2500] "X2" "X3" "X5" "X6" ...
  .. ..$ : chr [1:20] "indexA1" "indexF1" "index4" "index5" ...
 $ grouping        : Factor w/ 4 levels "group1","group2","group3",..: 1 5 2 5 1 1 3 4 3 1 ...
  ..- attr(*, "names")= chr [1:2500] "2" "3" "5" "6" ...
 $ lev             : chr [1:4] "group1" "group2" "group3" ...
 $ weight.func     :function (x)
 $ k               : num 3401
 $ weighted.apriori: logi TRUE
 $ call            : language loclda(x = x, grouping = y, k = floor(param$k), importance = ..1)
 $ xNames          : chr [1:20] "indexA1" "indexF1" "index4" "index5" ...
 $ problemType     : chr "Classification"
 $ tuneValue       :'data.frame':   1 obs. of  1 variable:
  ..$ k: num 3401
 $ obsLevels       : atomic [1:4] group1 group2 group3 ...
  ..- attr(*, "ordered")= logi FALSE
 $ param           :List of 1
  ..$ importance: logi TRUE
 - attr(*, "class")= chr "loclda"


imp_out <- filterVarImp(x=my_loclda_model$finalModel$learn, y=my_loclda_model$finalModel$grouping)

Error in data[, fc] : (subscript) logical subscript too long
traceback()
5: lapply(data[, fc], as.numeric)
4: stopifnot(is.list(x), is.list(val))
3: modifyList(data, lapply(data[, fc], as.numeric))
2: asNumeric(x)
1: filterVarImp(x = model_name$finalModel$learn, y = model_name$finalModel$grouping)

I don't understand. What x and y arguments should I pass to filterVarImp?

Thanks !

Note: the same problem also happened when using other methods (e.g. svmRadial, svmPoly, LogitBoost, regLogistic, ...).

user31888 user31888 · Accepted Answer · 2018-12-30T12:21:59

Specifying x=as.data.frame(my_loclda_model$finalModel$learn) instead of a matrix seems to solve the problem, although the manual mentions that both matrix and data frame can be used.

Variable Importance for caret loclda method

1 Answers