Filter data within function over do.call

Question

Data:

 data <- structure(list(index = c(1, 2, 3, 4, 5, 6), 
hour = c(13, 13, 13, 1, 1, 1), minutes = c(31, 
     31, 32, 36, 36, 36)), class = "data.frame", row.names = c(1067L, 
    1069L, 1070L, 1072L, 1073L, 1074L))

use do.call:

func <- function(hourFilt, minutesFilt){
  filt <- data[data$hour == hourFilt & data$minutes == minutesFilt, ]$idx
  sum(filt)
}

do.call(func, list(hourFilt = 1:5, minutesFilt = 31:35))

I am aware what the warning means:

Warning messages: 1: In data$hour == hourFilt : longer object length is not a multiple of shorter object length

but I would expect, that the function is called five times and not only once, resulting in that warning and wrong result.

Expected result:

mapply(func, hourFilt = 1:5, minutesFilt = 31:35)
1 2 0 0 0

However, I would prefer using do.call() as it is faster.

Edit: Adding intention I want to filter a dataset of 25k-200k rows for a specific time. The filtering on the original data set will result in three rows. And for that result i want to sum up a given variable - here: index. That process will be repeated for a few hundred times. Therefore, i looked into mapply(), do.call(). Performance does matter, thats why i prefer do.call().

If one had not seen your code, how would you describe your intention? Please edit to reflect the expected output more explicitly. — NelsonGon
If I run your function and then use mapply like mapply(func, hourFilt = 1:5, minutesFilt = 31:35) I get all 0's on your data which makes sense since for the first case data$index[data$hour == 1 & data$minutes == 31] returns numeric(0). How are you getting 1 ? — Ronak Shah
I'm not sure that it's fair to say that do.call is "faster" as they do completely different things. do.call() will only ever call a function once with one set of parameters. You seem to want to call a function multiple times with different parameters. Those are very different operations. I'm not sure do.call does what you think it does. — MrFlick

Athe Athe · Accepted Answer · 2019-04-26T15:37:54

The purpose of do.call is not to run a function multiple times, such as mapply would do, but to run a function once over a list of elements. This is very practical when working with large lists and trying to apply a single function to all arguments of the list. A typical example would be do.call(cbind,list), if you want to create a table in which each column is an element of the list.

So, when you run do.call(func, list(hourFilt = 1:5, minutesFilt = 31:35)) R is running

func(hourFilt = 1:5, minutesFilt = 31:35)

And thus you get your errors. All elements of the list are being passed at once to the arguments of your func().

I think mapply() might be the best solution to your problem. If you are concerned about performance, you could try mcmapply() from the parallel package, which can parallelize the computation (not on Windows).

corenum <- parallel::detectCores()-1
parallel::mcmapply(func,hourFilt=1:5,minutesFilt=31:35,mc.cores=corenum)

Or if you are working on Windows, the more cumbersome parSapply() might work for you:

corenum <- parallel::detectCores()-1
cl<-parallel::makeCluster(corenum)

#export the objects so that the parallel sockets can use them
parallel::clusterExport(cl,c("func","hourFilt","minutesFilt","data"))

result<-parallel::parSapply(cl,1:length(hourFilt),function(i){
  func(hourFilt[i],minutesFilt[i])
})

PS: With your dataset the expected result should be 0 0 0 0 0

Filter data within function over do.call

1 Answers