Possible reasons for " Error in checkForRemoteErrors(lapply(cl, recvResult)) : ... object '.doSnowGlobals' not found" Error?

Question

I have been executing a function script repeatedly in R for many years. Within the function definition, I set up a parallel cluster using on my multi-core Windows workstation using:

  # cores0 <- 20 (cores set to 20 outside of function definition)
  cl.spec <- rep("localhost", cores0)
  cl <- makeCluster(cl.spec, type="SOCK", outfile="") 
  registerDoParallel(cl, cores=cores0)

As of yesterday, my function execution is no longer working, and was getting hung up for hours. (Additionally, using the Resource Monitor, I could see that none of my CPUs were active despite my script specifying 20 cores). When I went back into the function and tested line, by line, I discovered that the following line is not executing (i.e., is getting hung up when it would usually execute in a few seconds):

cl.spec <- rep("localhost", cores0)
cl <- makeCluster(cl.spec, type="SOCK", outfile="")

I tried looking up the problem and found several references to using "PSOCK" type, but could not determine when to use PSOCK versus SOCK. Nonetheless, I attempted the same script using "PSOCK" instead of "SOCK":

  cl <- makeCluster(cl.spec, type="PSOCK", outfile="") 
  registerDoParallel(cl, cores=cores0)

With the PSOCK modification, it no longer got hung up and it appeared to execute this as well as the registerDoParallel() call.

However, when I then executed the complete function containing the above two lines and then called the function, as below, I got an error I had never seen:

Error in checkForRemoteErrors(lapply(cl, recvResult)) : 
  20 nodes produced errors; first error: object '.doSnowGlobals' not found

I also tried not specifying the type or outfile, but this produced the identical error as using type="PSOCK"

  cl <- makeCluster(cl.spec) 
  registerDoParallel(cl, cores=cores0)

My questions: 1. Why might the makeCluster() line be getting hung up when it never has before? cl <- makeCluster(cl.spec, type="SOCK", outfile="")

The problem happens whether I have only the parallel and doParallel packages loaded AND if I also have the snow and doSNOW packages loaded. Are all 4 packages required to execute foreach() commands?

Here is the function definition and function call containing the makeCluster() and registerDoParallel() calls, as above:

# FUNCTION DEFINITION
FX_RFprocessingSNPruns <- function(path, CurrentRoundSNPlist, colSAMP, Nruns, ntreeIN, coresIN,CurrentRoundGTframeRDA){

   ...do a bunch of steps ...

  #&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&
  # SET UP INTERNAL FUNCTION
  #&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&
  ImpOOBerr<-function(x,y,d) { 

    create function
  }


  #################################################################
  # SET UP THE CLUSTER
  #################################################################
  #Setup clusters via parallel/DoParallel
  cl.spec <- rep("localhost", cores0)
  cl <- makeCluster(cl.spec, type="PSOCK", outfile="") 
  registerDoParallel(cl, cores=cores0)

  #################################################################
  # *** EMPLOY foreach TO CARRY OUT randomForest IN PARALLEL
  #################################################################
  system.time(RFoutput_runs <- foreach(i=1:Nruns0, .combine='cbind', .packages= 'randomForest', .inorder=FALSE, .multicombine=TRUE, .errorhandling="remove") 

              %dopar% {    

                ...do a bunch of steps ...
              ImpOOBerr(x,y,d)
              })

  #################################################################
  # STOP THE CLUSTER
  #################################################################
  stopCluster(cl)

  return(RFoutput_runs)

}


# CALL FUNCTION
path0="C:/USERS/KDA/WORKING/"
system.time(GTtest_5runs <- FX_RFprocessingSNPruns(
  path=path0,
  CurrentRoundSNPlist="SNPlist.rda",
  colSAMP=20, 
  Nruns=5, 
  ntreeIN=150, 
  coresIN=5,
  CurrentRoundGTframeRDA="GT.rda"))


#Error in checkForRemoteErrors(lapply(cl, recvResult)) : 
#  20 nodes produced errors; first error: object '.doSnowGlobals' not found.

I found these posts that reference the error, but the solutions are not working for me: error: object '.doSnowGlobals' not found? http://grokbase.com/t/r/r-sig-hpc/148880dpsm/error-object-dosnowglobals-not-found

I'm working on Windows 8 machine, 64-bit with 40 cores.

R.Version()
$platform
[1] "x86_64-w64-mingw32"

$arch
[1] "x86_64"

$os
[1] "mingw32"

$system
[1] "x86_64, mingw32"

$status
[1] ""

$major
[1] "3"

$minor
[1] "3.0"

$year
[1] "2016"

$month
[1] "05"

$day
[1] "03"

$`svn rev`
[1] "70573"

$language
[1] "R"

$version.string
[1] "R version 3.3.0 (2016-05-03)"

$nickname
[1] "Supposedly Educational"

KDA KDA · Accepted Answer · 2016-06-03T20:19:57

1

votes

It was institutional anti-virus software preventing access to the cores.

Possible reasons for " Error in checkForRemoteErrors(lapply(cl, recvResult)) : ... object '.doSnowGlobals' not found" Error?

1 Answers