parallel using plyr

1
votes

I am reading in an XML file and want to try and improve the performance using foreach and doSNOW, however when using plyr I encounter an error. I think it might be because I have missed a way to send the functions and packages to the workers.

library(foreach)  
library(doSNOW)  
cl <- makeCluster(4)  
registerDoSNOW(makeCluster(4, type = "SOCK"))

> getDoParWorkers()  
[1] 4  
> getDoParName()  
[1] "doSNOW"  
> getDoParVersion()  
[1] "1.0.5"

namelist = list()  
out1 <- foreach(i = 1:xmlSize(root[[3]])) %dopar% {  
     namelist[[i]] <- llply(xmlToList(root[[3]][[i]][[2]]), 
                        data.frame, .parallel = TRUE)
}

out2 <- t(as.vector(as.data.frame(namelist)))

Generates the error:

Error in { : task 1 failed - "could not find function "llply""
1
rforeachparallel-processingplyr
It would be nice if you could repost it as a self-contained example once it works. What is root, and it certainly requires a library(XML). – Dieter Menne

1 Answers

4
votes

?foreach then look at the .packages argument