Using parallelMap Package with Custom Filter in mlr

Question

I working with mlr to do a text classification task. I have written a custom filter as described here

The filter works as intended, however when I try and and ustilise parallelization I receive the following error:

Exporting objects to slaves for mode socket: .mlr.slave.options
Mapping in parallel: mode = socket; cpus = 4; elements = 2.
Error in stopWithJobErrorMessages(inds, vcapply(result.list[inds], as.character)) : 
  Errors occurred in 2 slave jobs, displaying at most 10 of them:

00001: Error in parallel:::.slaveRSOCK() : 
  Assertion on 'method' failed: Must be element of set {'anova.test','carscore','cforest.importance','chi.squared','gain.ratio','information.gain','kruskal.test','linear.correlation','mrmr','oneR','permutation.importance','randomForest.importance','randomForestSRC.rfsrc','randomForestSRC.var.select','rank.correlation','relief','rf.importance','rf.min.depth','symmetrical.uncertainty','univariate','univariate.model.score','variance'}.

I'm assuming from the error that my custom filter needs to be an element in the set to stand a chance of working in parallel, but haven't managed to work out if (a) this is possible, and (b) if it is, how do I go about it.

Thanks in advance for any help, Azam

Added: Test Script I can't let you see the actual script/data I'm working with due to sensitivity, but this example reproduces the error I see. Apart from the custom feature selection and data-set, the steps to set up the learner and evaluate it are as I have in my 'real' script. As in my real case, if you remove the parallelStartSocket() command then the script runs as expected.

I should also add that I have sucessfully used (or at least I received no errors) parallel processing when tuning the hyper-parameters of an SVM with RBF kernel: the script being identical apart from the makeParamSet() definition.

library(parallelMap)
library(mlr)
library(kernlab)

makeFilter(
  name = "nonsense.filter",
  desc = "Calculates scores according to alphabetical order of features",
  pkg = "mlr",
  supported.tasks = c("classif", "regr", "surv"),
  supported.features = c("numerics", "factors", "ordered"),
  fun = function(task, nselect, decreasing = TRUE, ...) {
    feats = getTaskFeatureNames(task)
    imp = order(feats, decreasing = decreasing)
    names(imp) = feats
    imp
  }
)

# set up svm with rbf kernal
svm.lrn <- makeLearner("classif.ksvm",predict.type = "response")  

# wrap learner with filter
svm.lrn <- makeFilterWrapper(svm.lrn, fw.method = "nonsense.filter")

# define feature selection parameters 

ps.svm = makeParamSet(
  makeDiscreteParam("fw.abs", values = seq(2, 3, 1)) 

)

# define inner search and evaluation strategy
ctrl.svm = makeTuneControlGrid()
inner.svm = makeResampleDesc("CV", iters = 5, stratify = TRUE)

svm.lrn <- makeTuneWrapper(svm.lrn, resampling = inner.svm, par.set = ps.svm, 
                           control = ctrl.svm)

# set up outer resampling
outer.svm <-  makeResampleDesc("CV", iters = 10, stratify = TRUE)

# run it...

parallelStartSocket(2)

run.svm <- resample(svm.lrn, iris.task, 
                    resampling = outer.svm, extract = getTuneResult)

parallelStop()

Could you provide a complete example that allows to reproduce the problem please? — Lars Kotthoff
@LarsKotthoff, example script added to orginal post. Thanks, Azam — Azam Mullick

Lars Kotthoff Lars Kotthoff · Accepted Answer · 2017-02-11T21:06:27

The problem is that makeFilter registers S3 methods, which are not available in separate R processes. You have two options to make this work: either simply use parallelStartMulticore(2) so that everything runs in the same R process, or tell parallelMap about the pieces that need to be present in the other R processes.

There are two parts to the latter. First, use parallelLibrary("mlr") to load mlr everywhere and pull out the definition of the filter into a separate file that can be loaded using parallelSource(). For example:

filter.R:

makeFilter(
  name = "nonsense.filter",
  desc = "Calculates scores according to alphabetical order of features",
  pkg = "mlr",
  supported.tasks = c("classif", "regr", "surv"),
  supported.features = c("numerics", "factors", "ordered"),
  fun = function(task, nselect, decreasing = TRUE, ...) {
    feats = getTaskFeatureNames(task)
    imp = order(feats, decreasing = decreasing)
    names(imp) = feats
    imp
  }
)

main.R:

library(parallelMap)
library(mlr)
library(kernlab)

parallelStartSocket(2)

parallelLibrary("mlr")
parallelSource("filter.R")

# set up svm with rbf kernal
svm.lrn = makeLearner("classif.ksvm",predict.type = "response")  

# wrap learner with filter
svm.lrn = makeFilterWrapper(svm.lrn, fw.method = "nonsense.filter")

# define feature selection parameters 

ps.svm = makeParamSet(
  makeDiscreteParam("fw.abs", values = seq(2, 3, 1)) 

)

# define inner search and evaluation strategy
ctrl.svm = makeTuneControlGrid()
inner.svm = makeResampleDesc("CV", iters = 5, stratify = TRUE)

svm.lrn = makeTuneWrapper(svm.lrn, resampling = inner.svm, par.set = ps.svm, 
                           control = ctrl.svm)

# set up outer resampling
outer.svm =  makeResampleDesc("CV", iters = 10, stratify = TRUE)

# run it...
run.svm = resample(svm.lrn, iris.task, resampling = outer.svm, extract = getTuneResult)

parallelStop()

Using parallelMap Package with Custom Filter in mlr

1 Answers