How to use custom function in dplyr group_by

Question

I'm trying to use a custom function (which returns a scalar) in summarise or mutate in a dplyr flow which includes a group_by. The function works when I call it directly, however whenever it follows a group_by it doesn't work.

See my code for what I've tried. I managed to get it working but I feel it's a kind of hacky way - I want to understand why it isn't working as I'd expect it to:

Load data and define Gini coefficient function

## Load required libraries
library(dplyr)
library(tidyr)
library(ROCR)

set.seed(0)
## Generate fake data
df1 <- data.frame(predictions = seq(0,1,.01), date = seq(as.Date("2009-01-01"), by = "month", length.out = 101), labels = sample(c(0,1), replace=TRUE, size=101), grouping = rep('a',101))
df2 <- data.frame(predictions = seq(0,1,.01), date = seq(as.Date("2010-01-01"), by = "month", length.out = 101), labels = sample(c(0,1), replace=TRUE, size=101), grouping = rep('b',101))

df <- rbind(df1,df2)

## Gini coefficient calculation function
dplyr_Gini <- function(df, predictions, labels, label.ordering = NULL,...){
  predictions = enquo(predictions)
  labels = enquo(labels)

  predictions <- df %>% select(!!predictions) %>% .[[1]]
  labels <- df %>% select(!!labels) %>% .[[1]]

  if(length(unique(labels)) != 2){
    return(NA)
  }

  pred <- prediction(predictions, labels, label.ordering)
  auc.perf = performance(pred, measure = "auc")
  gini =  2*[email protected][[1]] - 1
  return(gini)
}

## test dplyr_Gini - works as expected
dplyr_Gini(df1,predictions, labels)
> [1] -0.05494505
dplyr_Gini(df2,predictions, labels)
> [1] 0.09456265

Not working - use dplyr_Gini after a group_by.

## Wrapper function for using dplyr_Gini in group_by
calc_Gini <- function(df, group, predictions, labels){
  predictions <- enquo(predictions)
  labels = enquo(labels)

  df %>% filter(grouping %in% group) %>%
    group_by(grouping) %>% 
    summarise(min.date = min(date),
              max.date = max(date),
              Gini = dplyr_Gini(.,!!predictions, !!labels)) %>% 
    ungroup()  
}

calc_Gini(df,group = c('a','b'),predictions, labels)
> # Adding missing grouping variables: `grouping`
> # Adding missing grouping variables: `grouping`
> # Error in prediction(predictions, labels, label.ordering) : 
> # Format of predictions is invalid.

Working - use dplyr_Gini after a group_by using do and unnest

## Wrapper function that works for using dplyr_Gini in group_by - but is kind of hacky.
calc_Gini_working <- function(df, group, predictions, labels){
  predictions <- enquo(predictions)
  labels = enquo(labels)

  df %>% filter(grouping %in% group) %>%
    group_by(grouping) %>% 
    mutate(min.date = min(date),
              max.date = max(date)) %>% 
    group_by(grouping, min.date, max.date) %>% 
    do(Gini = dplyr_Gini(.,!!predictions, !!labels)) %>% 
    unnest() %>% 
    ungroup()

}

calc_Gini_working(df,group = c('a','b'),predictions, labels)
>
# A tibble: 2 x 4
  grouping min.date   max.date      Gini
  <fct>    <date>     <date>       <dbl>
1 a        2009-01-01 2017-05-01 -0.0549
2 b        2010-01-01 2018-05-01  0.0946

I'm under the impression that the calc_Gini function would work as I've just added my custom function (dplyr_Gini) inside the summarise which follows a group_by.

As you can see, if I wrap dplyr_Gini in a do and then unnest the result it works - but I'm not sure why.

akrun akrun · Accepted Answer · 2019-04-30T03:00:07

Based on the way the dplyr_Gini is constructed, one option is to group_split and then use map

library(tidyverse)
calc_Gini <- function(df, group, predictions, labels){
  predictions <- enquo(predictions)
  labels = enquo(labels)

  df %>% filter(grouping %in% group) %>%
    group_split(grouping, remove = FALSE) %>% 
    map_dfr(., ~               
              tibble(grouping = first(.x$grouping), min.date = min(.x$date), 
                     max.date = max(.x$date), 
                     Gini = dplyr_Gini(.x, !!predictions, !!labels)))

}


calc_Gini(df,group = c('a','b'),predictions, labels)
# A tibble: 2 x 4
#  grouping min.date   max.date      Gini
#  <fct>    <date>     <date>       <dbl>
#1 a        2009-01-01 2017-05-01 -0.0549
#2 b        2010-01-01 2018-05-01  0.0946

How to use custom function in dplyr group_by

Load data and define Gini coefficient function

Not working - use dplyr_Gini after a group_by.

Working - use dplyr_Gini after a group_by using do and unnest

1 Answers