Multiple summary statistics on grouped column in Julia

Question

I am trying below code to work with Julia(1.5.3), Its just a representation of what I am trying to do.

using DataFrames
using DataFramesMeta
using RDatasets

## setup
iris = dataset("datasets", "iris")
gdf = groupby(iris, :Species)

## Applying the split combine
## This code works fine
combine(gdf, nrow, (valuecols(gdf) .=> mean))

But, when I try to do it for multiple summary it fails

 combine(gdf, nrow, (valuecols(gdf) .=> [mean, sum]))

Error:

ERROR: DimensionMismatch("arrays could not be broadcast to a common size; got a dimension with lengths 4 and 2")

Little debug on error suggests that If I change my code to this:

combine(gdf, nrow, ([:SepalLength, :PetalLength] .=> [mean,sum]))
## This code works but its still not correct as it doesn't tell me the mean and sum of both the columns , rather mean for SepalLength and sum for PetalLength, which was expected as per previous error

A little more research into it and I realized that, we can do something like this, this result is correct but the outcome is in long form of table not the wide form. I was expecting this would have given me the answer to my question, but it seems it doesn't work as expected.

 combine(gdf, ([:SepalWidth, :PetalWidth] .=>  x -> ([sum(x), mean(x)])))

 ## The code above works but output is 6x3 DataFrame, I was expecting 3x6 DataFrame

My question is:

Is there any way to use split combine in such a way that I get a wide table like below (I have used "do end" with "combine" to generate it). I am okay with this solution, but I need to type out all the column here, Is there any way such that I can get all the summary stats(sum, median, mean etc) as columns for all the column provided in combine. I hope my question is clear, Please point out in case its a duplicate or its not well communicated. Thanks

combine(gdf) do x
    return(sw_sum = sum(x.SepalWidth), 
           sw_mean = mean(x.SepalWidth), 
           sp_mean = mean(x.PetalWidth), 
           sp_sum = sum(x.PetalWidth)
          )
end



## My expected answer should be similar to this
#3×5 DataFrame
# Row │ Species     sw_sum   sw_mean  sp_mean  sp_sum
#     │ Cat…        Float64  Float64  Float64  Float64
#─────┼────────────────────────────────────────────────
#   1 │ setosa        171.4    3.428    0.246     12.3
#   2 │ versicolor    138.5    2.77     1.326     66.3
#   3 │ virginica     148.7    2.974    2.026    101.3

Also, this works:

 combine(gdf, [:1] .=> [mean, sum, minimum, maximum,median])

But this doesn't and throws the dimension error like above, still scratching my head over this:

combine(gdf, [:1, :2] .=> [mean, sum, minimum, maximum,median])

Bogumił Kamiński Bogumił Kamiński · Accepted Answer · 2021-02-24T13:57:51

Do:

 combine(gdf, nrow, vec(valuecols(gdf) .=> [mean sum]))

or

 combine(gdf, nrow, (valuecols(gdf) .=> [mean sum])...)

or

 combine(gdf, nrow, [n => f for n in valuecols(gdf) for f in [mean sum]])

(note that there is no comma between mean and sum)

The reason is that you need to add an additional dimension to broadcasted .=> in order to get all combinations of inputs.

EDIT:

... just iterates a collection and passes its elements as consecutive positional arguments to the function, e.g.:

julia> f(x...) = x
f (generic function with 1 method)

julia> f(1, [2,3,4]...)
(1, 2, 3, 4)

Multiple summary statistics on grouped column in Julia

1 Answers