Having trouble running NetLogo models using parLapply

Question

I'm trying to run a NetLogo simulation (using RNetLogo package) in R using parallel processing on my laptop. I'm trying to assess "t-feeding of females" using 3 (i.e., 0, 25, and 50) different "minimum-separation" values. For each "minimum-separation" value, I'd like to replicate the simulation 10 times. I can run everything correctly just using lapply but I'm having trouble with parLapply. I've just started using the package "parallel" so I'm sure it is something in the syntax.

#Set up clusters for parallel
processors <- detectCores()
cl <- makeCluster(processors)

#Simulation
sim3 <- function(min_sep) {
 NLCommand("set minimum-separation ", min_sep, "setup")
 ret <- NLDoReport(720, "go", "[t-feeding] of females", as.data.frame=TRUE)
 tot <- sum(ret[,1])
 return(tot)
}  

#Replicate simulations 10 times using lapply and create boxplots.  This one works.
rep.sim3 <- function(min_sep, rep) {
 return(
 lapply(min_sep, function(min_sep) {
 replicate(rep, sim3(min_sep))
 })
 )
}
d <- seq(0,50,25)
res <- rep.sim3(d,10)
boxplot(res,names=d, xlab="Minimum Separation", ylab="Time spent feeding")

#Replicate simulations 10 times using parLapply.  This one does not work.
rep.sim3 <- function(min_sep, rep) {
 return(
 parLapply(cl, min_sep, function(min_sep) {
 replicate(rep, sim3(min_sep))
 })
 )
}
d <- seq(0,50,25)
res <- rep.sim3(d,10)

# Error in checkForRemoteErrors(val) : 3 nodes produced errors; first error: could not find function "sim3"

#Replicate simulations 10 times using parLapply.  This one does work but creates a list of the wrong length and therefore the boxplot cannot be plotted correctly.
rep.sim3 <- function(min_sep, rep) {
 return(
 parLapply(cl, replicate(rep, d), sim3))
}
d <- seq(0,50,25)
res <- rep.sim3(d,10)

Ideally I'd like to make the first parLapply work. Alternatively, I guess I could modify res from the parLapply that works so that the list has a length of max_sep instead of 30. However, I can't seem to do that. Any help would be much appreciated!

Thanks in advance.

Steve Weston Steve Weston · Accepted Answer · 2013-07-05T13:26:27

You need to initialize the cluster workers before executing rep.sim3. The error message indicates that your workers can't execute the sim3 function because you haven't exported it to them. Also, I noticed that you haven't loaded the RNetlogo package on the workers, either.

The easiest way to initialize the workers is with the clusterEvalQ and clusterExport functions:

clusterEvalQ(cl, library(RNetLogo))
clusterExport(cl, 'sim3')

Note that you shouldn't do this in your rep.sim3 function, since that would be inefficient and unnecessary. Do it just once after creating the cluster object and sim3 has been defined.

This initialization is necessary because the workers started via makeCluster don't know anything about your variables or functions, or anything else about your R session. And parLapply doesn't analyze the function that you pass to it any more than lapply does. The difference is that lapply executes in your local R session where sim3 is defined and the RNetLogo package is loaded. parLapply executes the specified function in remote R sessions that have not been initialized by executing your R script.

Having trouble running NetLogo models using parLapply

1 Answers