Mean of variables by factors with missing values

Question

I am using a big dataset to calculate means of variables by factors. Example simple dataset looks like below.

 +------+-----+------+--------+------+-------+------+------+
| year | mon | site | region |  rf  | avg 1 | avg2 | avg3 |
+------+-----+------+--------+------+-------+------+------+
| 2000 | jan | A    | high   | 28.2 |       |      |      |
| 2000 | feb | A    | high   | 26.6 |       |      |      |
| 2000 | mar | A    | high   | 30.3 |       |      |      |
| 2000 | apr | A    | high   | 33.2 |       |      |      |
| 2000 | may | A    | high   |      |       |      |      |
| 2000 | jun | A    | high   | 28.3 |       |      |      |
| 2000 | jul | A    | high   | 28.6 |       |      |      |
| 2000 | aug | A    | high   | 28.9 |       |      |      |
| 2000 | sep | A    | high   | 28.1 |       |      |      |
| 2000 | oct | A    | high   | 28.8 |       |      |      |
| 2000 | nov | A    | high   | 31.6 |       |      |      |
| 2000 | dec | A    | high   | 26.9 |       |      |      |
| 2001 | jan | A    | high   | 28.6 |       |      |      |
| 2001 | feb | A    | high   | 29.6 |       |      |      |
| 2002 | jan | B    | mid    | 21.4 |       |      |      |
| 2002 | feb | B    | mid    | 24.5 |       |      |      |
| 2002 | mar | B    | mid    | 24.2 |       |      |      |
+------+-----+------+--------+------+-------+------+------+

But the main variable (rf) has got some missing values. But I want to calculate the means (avg 1, avg2 avg3) removing the missing values. My data set can be accessed using following dput codes.

structure(list(year = c(2000L, 2000L, 2000L, 2000L, 2000L, 2000L, 
2000L, 2000L, 2000L, 2000L, 2000L, 2000L, 2001L, 2001L, 2002L, 
2002L, 2002L), mon = structure(c(5L, 4L, 8L, 1L, 9L, 7L, 6L, 
2L, 12L, 11L, 10L, 3L, 5L, 4L, 5L, 4L, 8L), .Label = c("apr", 
"aug", "dec", "feb", "jan", "jul", "jun", "mar", "may", "nov", 
"oct", "sep"), class = "factor"), site = structure(c(1L, 1L, 
1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 2L, 2L, 2L), .Label = c("A", 
"B"), class = "factor"), region = structure(c(1L, 1L, 1L, 1L, 
1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 2L, 2L, 2L), .Label = c("high", 
"mid"), class = "factor"), rf = c(28.2, 26.6, 30.3, 33.2, NA, 
28.3, 28.6, 28.9, 28.1, 28.8, 31.6, 26.9, 28.6, 29.6, 21.4, 24.5, 
24.2), avg_rf_site_allyears = c(NA, NA, NA, NA, NA, NA, NA, NA, 
NA, NA, NA, NA, NA, NA, NA, NA, NA), avg_mon_rf_all_site = c(NA, 
NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA
), avg_rf_year_ele = c(NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, 
NA, NA, NA, NA, NA, NA, NA)), .Names = c("year", "mon", "site", 
"region", "rf", "avg_rf_site_allyears", "avg_mon_rf_all_site", 
"avg_rf_year_ele"), class = "data.frame", row.names = c(NA, -17L
))

avg 1 is the mean rainfall by site across all years (I have 15 years by monthly).

avg 2 is the mean monthly rainfall across all sites over all years

avg 3 is the mean rainfall by region by year

I am using following codes but these codes are not working for the sites with missing values.

avg 1

df$avg.1<- with(df,ave(rf, site)) # mean rf by sites across all years. This does not calculate values for sites if it has got even one missing value.

avg2

df$avg2<- with(df,ave(rf, mon))#this works in this example but not with my    big dataset. When I run with my dataset, it gives all NAs.

It would be great if anybody can tell me a potential cause for this problem too.

avg 3 - I need to calculate means by regions by years. But could not find a way for this.

Any help regarding above would be much appreciated.

akrun akrun · Accepted Answer · 2016-03-25T01:34:22

We can specify the FUN argument in ave. By default, i.e. without specifying, it gives the mean with na.rm=FALSE. So, using FUN, any other function like min, max etc. can be used.

df$avg.1 <- with(df, ave(rf, site, 
        FUN= function(x) mean(x, na.rm=TRUE)))

and similarly for 'avg.2'.

For the third case

df$avg.3 <- with(df, ave(rf, region, year, 
        FUN= function(x) mean(x, na.rm=TRUE))

If we are using dplyr

library(dplyr)
df %>%
  group_by(site) %>% 
  mutate(avg.1 = mean(rf, na.rm=TRUE)) %>% 
  group_by(mon) %>% 
  mutate(avg.2 = mean(rf, na.rm=TRUE)) %>% 
  group_by(region, year) %>%
  mutate(avg.3= mean(rf, na.rm=TRUE))

Mean of variables by factors with missing values

1 Answers