How to collapse data with duplicate timestamps in Julia `DataFrame`

3
votes

I have a DataFrame object looking as follows:

| Row | timestamp           | price | volume |
|-----|---------------------|-------|--------|
| 1   | 2011-08-14T14:14:40 | 10.40 | 0.779  |
| 2   | 2011-08-14T15:15:17 | 10.40 | 0.101  |
| 3   | 2011-08-14T15:15:17 | 10.40 | 0.316  |
| ... | ................... | ..... | .....  |

The timestamps are non-unique, so I cannot convert to a TimeArray before resolving this. How can I collapse duplicate timestamps, taking the mean of the prices and the sum of the volumes?

Thank you for any pointers!

1
julia

1 Answers

3
votes

You can use by:

df = DataFrame(
  cat = ["a", "b", "c","a"],
  prices = [1,2,3,4],
  vol    = [10,20,30,40],
)

df2 = by(df, :cat) do sub
      t = DataFrame(prices=mean(sub[:prices]), vol=sum(sub[:vol]))
end

df2

3×3 DataFrames.DataFrame
│ Row │ cat │ prices │ vol │
├─────┼─────┼────────┼─────┤
│ 1   │ "a" │ 2.5    │ 50  │
│ 2   │ "b" │ 2.0    │ 20  │
│ 3   │ "c" │ 3.0    │ 30  │

If you have to make some totals by day/months/etc you may be interested also in this so answer.