Create plot with averages and standard deviation computed after every iteration

Question

I have a data frame with this row:

read.table(text = "            V1        V2        V3        V4        V5        V6        V7        V8        V9       V10
 BACC 0.6203474 0.4466501 0.4820099 0.7003722 0.4851117 0.5173697 0.6073201 0.5911911 0.3759305 0.4627792")

Each of the columns is the same calculation being done repeatedly (random sample changes, so value of balanced accuracy changes). I want to generate a plot which shows an iterative mean value with error bars (standard deviation). So for example, at x = 1 iteration, it just plots the point given in column V1. For the second, it averages the first and second columns and calculates the std. dev. and plots this. For the third, it averages first, second, and third columns and does the same all the way until it eventually averages all and computes standard deviation of all.

How can this plot be made? Or a data frame which contains each iterative mean/standard deviation? My real data has 1000 columns (not 10) so I would like something that can be expanded to as large of a data set as possible. Also, if I only want to compute average/standard deviation every 5 iterations, how would that be done?

I would really appreciate if you could help me with this.

Thanks!

Jake Kaupp Jake Kaupp · Accepted Answer · 2017-11-14T15:14:42

You can do this using some tidyverse packages and TTR

Since ggplot2 likes tidy data in a long format, this solution uses gather to get all of your measurement columns into rows. This also makes it easier to perform a running mean and running standard deviation on your data.

library(tidyverse) #for dplyr, tidyr, ggplot2
library(TTR) # for runMean and runSD

df <- read.table(text = "            V1        V2        V3        V4        V5        V6        V7        V8        V9       V10
 BACC 0.6203474 0.4466501 0.4820099 0.7003722 0.4851117 0.5173697 0.6073201 0.5911911 0.3759305 0.4627792")


plot_data <- df %>%
  gather(measurement, value) %>%
  mutate(cum_mean = ifelse(value == first(value), value, runMean(value, 1, cumulative = TRUE)),
         cum_sd = runSD(value, 1, cumulative = TRUE))

ggplot(plot_data, aes(x = measurement, y = cum_mean)) +
  geom_col() +
  geom_errorbar(aes(ymin = cum_mean - cum_sd, ymax = cum_mean + cum_sd), size = 0.2)

If you want to do this over a sequence of rows, e.g every 5th row, you can achieve this using slice and including the sequence over the number of rows in the data frame.

plot_data <- df %>%
  gather(measurement, value) %>%
  slice(seq(1, nrow(.), 5)) %>% 
  mutate(cum_mean = ifelse(value == first(value), value, runMean(value, 1, cumulative = TRUE)),
         cum_sd = runSD(value, 1, cumulative = TRUE))

Create plot with averages and standard deviation computed after every iteration

1 Answers