bootstrap ICC in R

Question

Being new to R I am having difficulties to bootstrap an ICC output. I first managed to calculate a "normal" ICC using the package ICC without any problems (ICCbare(subject, variable, icc)), but when I tried to get some bootstrapped estimates it became worse...

I started by

icc_boot<-function(icc, i)ICCbare(subject [i], variable [i], icc)

And entered the icc_boot in the bootstrap as shown below:

testicc<-boot(icc, icc_boot, 1000)

However I got an error message saying that "undefined columns selected", where did I go wrong?

Here is a small output of my data

structure(list(subject_id = c(2001, 2001, 2001, 2001, 2001, 2001, 2001, 2001, 2001, 2001, 2001, 2001, 2001, 2001, 2001, 2001, 2001, 2001, 2001, 2001, 2001, 2001, 2001, 2001, 2001, 2001, 2001, 2001, 2002, 2002, 2002, 2002, 2002, 2002, 2002, 2002, 2002, 2002, 2002, 2002, 2002, 2002, 2002, 2002, 2002, 2002, 2002, 2002, 2002, 2002, 2002, 2002, 2002, 2002, 2002, 2002, 2002, 2003, 2003, 2003, 2003, 2003, 2003, 2003, 2003, 2003, 2003, 2003, 2003, 2003, 2003, 2003, 2003, 2003, 2003, 2003, 2003, 2003, 2003, 2003, 2003, 2003, 2003, 2003, 2003, 2004, 2004, 2004, 2004, 2004, 2004, 2004, 2004, 2004, 2004, 2004, 2004, 2004, 2004, 2004, 2004, 2004, 2004, 2004, 2004, 2004, 2004, 2004, 2004, 2004, 2004, 2004, 2005, 2005, 2005, 2005, 2005, 2005, 2005, 2005, 2005, 2005, 2005, 2005, 2005, 2005, 2005, 2005, 2005, 2005, 2005, 2005, 2005, 2005, 2005, 2005, 2005, 2006, 2006, 2006, 2006, 2006, 2006, 2006, 2006, 2006, 2006, 2006, 2006, 2006, 2006, 2006, 2006, 2006, 2006, 2006, 2006, 2006, 2006, 2006, 2006, 2006, 2006, 2006, 2006, 2006, 2007, 2007, 2007, 2007, 2007, 2007, 2007, 2007, 2007, 2007, 2007, 2007, 2007, 2007, 2007, 2007, 2007, 2007, 2007, 2007, 2007, 2007, 2007, 2007, 2007, 2007, 2007, 2008, 2008, 2008, 2008, 2008, 2008, 2008, 2008, 2008, 2008, 2008, 2008, 2008, 2008, 2008, 2008, 2008, 2008, 2008, 2008, 2008, 2008, 2008, 2009, 2009, 2009, 2009, 2009, 2009, 2009, 2009, 2009, 2009, 2009, 2009, 2009, 2009, 2009, 2009, 2009, 2009, 2009, 2009, 2009, 2009, 2009, 2009, 2009, 2009, 2009, 2013, 2013, 2013, 2013, 2013, 2013, 2013, 2013, 2013, 2013, 2013, 2013, 2013, 2013, 2013, 2013, 2013, 2013, 2013, 2013, 2013, 2013, 2013, 2013, 2013, 2013, 2013, 2013, 2014, 2014, 2014, 2014, 2014, 2014, 2014, 2014, 2014, 2014, 2014, 2014, 2014, 2014, 2014, 2014, 2014, 2014, 2014, 2014, 2014, 2014, 2015, 2015, 2015, 2015, 2015, 2015, 2015, 2015, 2015, 2015, 2015, 2015, 2015, 2015, 2015, 2015, 2015, 2015, 2015, 2015, 2015, 2015, 2015, 2015, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2016, 2018, 2018, 2018, 2018, 2018, 2018, 2018, 2018, 2018, 2018, 2018, 2018, 2018, 2018, 2018, 2018, 2018, 2018, 2018, 2018, 2018, 2018, 2018, 2018, 2018, 2018, 2018, 2019, 2019, 2019, 2019, 2019, 2019, 2019, 2019, 2019, 2019, 2019, 2019, 2019, 2019, 2019, 2019, 2019, 2019, 2019, 2019, 2019, 2019, 2019, 2020, 2020, 2020, 2020, 2020, 2020, 2020, 2020, 2020, 2020, 2020, 2020, 2020, 2020, 2020, 2020, 2020, 2020, 2020, 2020, 2020, 2020, 2020, 2020, 2020, 2020, 2020, 2020, 2020, 2020, 2022, 2022, 2022, 2022, 2022, 2022, 2022, 2022, 2022, 2022, 2022, 2022, 2022, 2022, 2022, 2022, 2022, 2022, 2022, 2022, 2022, 2022, 2022, 2022, 2022, 2022, 2022, 2022, 2022, 2023, 2023, 2023, 2023, 2023, 2023, 2023, 2023, 2023, 2023, 2023, 2023, 2023, 2023, 2023, 2023, 2023, 2023, 2023, 2023, 2023, 2023, 2023, 2023, 2023, 2023, 2023, 2024, 2024, 2024, 2024, 2024, 2024, 2024, 2024, 2024, 2024, 2024, 2024, 2024, 2024, 2024, 2024, 2024, 2024, 2024, 2024, 2024, 2024, 2024, 2024, 2024, 2025, 2025, 2025, 2025, 2025, 2025, 2025, 2025, 2025, 2025, 2025, 2025, 2025, 2025, 2025, 2025, 2025, 2025, 2025, 2025, 2025, 2025, 2025, 2025, 2025, 2025, 2025, 2026, 2026, 2026, 2026, 2026, 2026, 2026, 2026, 2026, 2026, 2026, 2026, 2026, 2026, 2026, 2026, 2026, 2026, 2026, 2026, 2026, 2026, 2026, 2026, 2026, 2026, 2026, 2026, 2028, 2028, 2028, 2028, 2028, 2028, 2028, 2028, 2028, 2028, 2028, 2028, 2028, 2028, 2028, 2028, 2028, 2028, 2028, 2028, 2028, 2028, 2028, 2028, 2028, 2028, 2028, 2028, 2028, 2029, 2029, 2029, 2029, 2029, 2029, 2029, 2029, 2029, 2029, 2029, 2029, 2029, 2029, 2029, 2029, 2029, 2029, 2029, 2029, 2029, 2029, 2029, 2029, 2029, 2029, 2029, 2029, 2029, 2030, 2030, 2030, 2030, 2030, 2030, 2030, 2030, 2030, 2030, 2030, 2030, 2030, 2030, 2030, 2030, 2030, 2030, 2030, 2030, 2030, 2030, 2030, 2030, 2030, 2030, 2030, 2030, 2031, 2031, 2031, 2031, 2031, 2031, 2031, 2031, 2031, 2031, 2031, 2031, 2031, 2031, 2031, 2031, 2031, 2031, 2031, 2031, 2031, 2031, 2031, 2031, 2031, 2031, 2031, 2032, 2032, 2032, 2032, 2032, 2032, 2032, 2032, 2032, 2032, 2032, 2032, 2032, 2032, 2032, 2032, 2032, 2032, 2032, 2032, 2032, 2032, 2032, 2032, 2033, 2033, 2033, 2033, 2033, 2033, 2033, 2033, 2033, 2033, 2033, 2033, 2033, 2033, 2033, 2033, 2033, 2033, 2033, 2033, 2033, 2033, 2033, 2033, 2033, 2033, 2033, 2033, 2033), interval_0_epochs = c(7139, 7964, 7775, 6756, 6075, 7184, 7965, 4730, 5957, 7574, 5649, 6496, 9266, 5052, 7090, 7680, 6992, 7151, 6022, 6592, 7310, 7785, 6714, 7311, 7636, 7015, 7482, 8860, 6997, 9034, 9553, 8326, 10015, 10252, 8463, 8612, 8388, 7648, 9503, 9978, 8014, 7125, 8261, 8818, 7733, 9518, 9099, 9368, 8468, 8899, 8984, 10304, 10064, 10398, 9193, 9331, 6866, 7423, 7745, 8525, 7362, 9115, 9509, 8804, 8726, 8668, 8361, 8513, 7670, 9077, 7375, 8148, 5897, 5507, 6321, 7695, 6222, 9024, 7096, 6490, 6319, 6142, 5225, 6081, 6314, 6391, 7319, 7598, 7921, 7324, 9289, 8792, 7980, 6650, 9045, 7896, 11727, 5953, 7087, 7624, 9317, 8086, 8307, 9486, 7380, 6613, 7033, 8647, 8366, 8418, 7721, 7516, 7337, 9151, 8360, 8149, 8172, 8697, 7589, 7563, 7780, 7108, 8080, 9582, 8381, 6790, 7983, 7047, 8986, 8643, 8941, 7422, 7403, 8101, 8563, 8375, 9329, 7192, 9250, 8455, 9013, 8342, 8889, 7306, 9580, 9110, 8256, 8547, 8852, 9636, 8893, 5919, 9139, 9146, 7476, 8004, 7755, 6243, 7804, 8797, 7330, 9280, 9893, 9603, 8839, 6793, 9207, 6338, 8098, 9792, 8984, 8139, 7523, 7976, 7085, 9578, 9935, 5320, 5766, 5379, 5486, 8777, 7699, 8868, 6066, 7084, 6751, 8531, 8740, 8666, 8641, 7429, 8089, 9053, 9535, 8384, 8616, 9236, 6682, 9290, 9121, 7705, 8984, 7770, 8059, 7396, 7239, 7165, 8142, 8547, 8541, 6333, 6738, 7727, 6161, 7610, 9438, 6989, 9837, 7362, 10216, 7489, 6298, 7429, 8232, 11105, 9298, 9894, 9979, 7333, 7177, 7931, 7637, 9569, 7022, 7950, 8208, 6622, 8778, 8957, 6581, 5673, 9307, 9508, 8102, 8919, 8062, 7506, 7336, 8108, 7289, 7981, 7178, 7832, 8883, 7385, 7457, 9251, 7352, 8616, 8627, 9424, 8164, 8165, 5929, 8102, 7898, 5869, 7621, 7810, 7036, 9570, 9641, 9374, 9176, 7041, 8969, 10233, 7924, 9788, 7966, 8258, 8052, 8032, 6406, 9471, 5572, 8715, 9241, 8367, 8131, 8436, 6280, 7713, 7611, 8414, 9221, 7105, 8558, 9015, 9878, 8586, 7785, 7505, 7725, 8303, 8562, 10182, 7215, 7741, 9008, 7688, 9240, 9275, 8710, 6537, 5577, 9483, 7891, 7567, 9287, 10318, 6518, 8174, 9924, 8830, 9662, 8229, 9448, 7637, 6615, 7959, 9973, 7997, 6366, 6227, 8287, 8225, 6409, 7757, 7847, 8833, 7231, 9390, 8580, 9323, 9875, 8365, 9464, 9128, 9569, 8701, 7740, 8837, 7505, 6812, 7441, 8182, 10256, 10006, 10039, 5824, 7456, 8994, 8455, 9557, 9335, 9455, 9202, 6225, 9778, 8514, 8623, 7856, 7466, 6204, 8513, 8222, 8059, 8708, 8562, 8495, 9074, 7950, 5699, 7206, 6195, 8916, 5548, 7735, 6529, 6950, 7749, 6069, 11814, 9947, 8095, 5562, 6545, 7928, 7363, 5744, 8539, 11186, 9010, 6770, 6353, 8728, 6658, 9627, 7557, 9935, 9522, 8999, 9101, 8817, 7560, 7496, 8383, 7928, 8621, 8909, 6526, 8412, 6986, 6430, 7060, 6658, 7318, 9233, 7737, 6494, 6525, 7728, 7632, 7524, 6598, 6536, 8352, 7007, 8486, 7947, 6545, 7386, 8398, 7715, 7371, 7020, 7102, 5490, 7439, 7970, 8951, 7907, 8338, 6628, 7355, 7263, 7143, 6317, 7054, 8820, 7769, 8847, 8375, 7718, 7145, 8409, 7299, 7330, 6336, 6645, 8336, 6615, 8552, 9186, 7450, 8100, 8839, 9340, 10206, 8046, 9019, 7946, 7132, 8855, 9794, 8526, 6787, 8849, 8543, 7010, 8509, 8985, 10006, 7851, 7777, 7715, 7654, 8957, 7865, 9502, 7920, 7096, 8862, 8533, 9430, 8213, 10100, 8090, 6765, 8363, 8190, 8202, 7147, 9803, 8623, 8018, 8007, 8577, 7853, 9258, 8648, 6743, 8122, 7506, 8103, 8353, 9713, 9234, 8536, 7996, 7801, 8101, 7831, 8615, 5582, 6412, 7942, 7756, 8055, 7733, 9139, 6097, 6522, 8634, 6966, 8747, 8775, 8424, 7336, 6315, 9505, 7881, 9305, 8351, 9152, 7036, 6447, 6842, 7263, 7933, 8341, 8561, 8260, 7014, 7736, 7140, 9504, 8168, 7849, 7569, 5815, 7675, 8182, 8645, 8136, 8825, 7330, 6910, 8251, 7572, 7488, 7942, 7405, 8453, 7153, 6338, 9165, 9081, 9906, 10001, 13219, 6428, 8823, 8095, 7829, 6467, 9490, 14308, 6622, 8723, 7766, 7510, 7786, 10570, 10981, 7845, 7492, 8920, 8471, 9309, 8894, 10783, 6277, 9295, 8056, 7700, 8387, 7652, 8966, 8037, 6797, 7036, 8350, 8173, 8377, 8089, 8515, 8950, 8549, 6849, 8293, 6532, 8928, 7450, 8357, 8742, 8255, 7542, 8289, 9325, 7570, 7743, 7104, 7844, 8656, 8301, 8481, 8868, 5067, 5670, 8848, 7913, 7782, 8492, 6021, 5938, 4595, 7965, 8776, 7360, 8822, 6776, 4204, 8615, 7717, 8500, 7300, 4311, 6504, 7568, 6402, 8955, 9376, 9580, 7262, 6800, 7713, 8651, 8459, 7432, 7706, 9216, 8085, 9191, 8001, 7503, 8905, 8036, 8882, 6344, 10359, 5644, 6319, 7890, 6502, 6354, 9649, 7130, 8179, 6596, 7122, 8357, 7465, 8471, 7603, 9387, 7391, 6547, 7959, 7137, 8401, 9041, 9833, 8190, 7886, 8190, 8406, 8435, 9415, 8985, 6143, 6586, 8742)), .Names = c("subject_id", "interval_0_epochs"), row.names = c(NA, 725L), class = "data.frame")

R functions are usually defined as function(arguments){do something with arguments}. Perhaps this is not the case with ICC. What is the index i for? — martin
Thanks for a quick response! The function is basically copied from "discovering statistics using R" however then it was used to create a bootstrap function but for a correlation. The i stands for index and means that the data should be used as is. there is also possibilities to use weights (w), and frequencies (f) in the boot package. But I am not sure if the fucntion is correct when I want to bootstrap ICC. — Begga

user20650 user20650 · Accepted Answer · 2013-10-20T19:04:38

In defining the statistic for the boot function 'The first argument passed will always be the original data. The second will be a vector of indices..' See ?boot

An example:

library(ICC)
library(MASS)
library(boot)

# Data
Sigma <- matrix(c(10,3,3,2),2,2)
df <- data.frame(mvrnorm(n=20, rep(0, 2), Sigma))

#ICC on data
m.df <- reshape(df , dir = "long" , varying = list(1:2))
ICCbare(id , X1 , data = m.df)

# Bootstrap function ---------------------------------------------
boot.fun <- function(dat , i) {
  newdf <- dat[i , ]
  m.newdf <- reshape(newdf , dir = "long" , varying = list(1:2) , new.row.names=1:40)  
  ICCbare(id , X1 , data = m.newdf)$ICC
}

boo1 <- boot(df , boot.fun , 2000)
boot.ci(boo1)

EDIT: Based on your data, this would be my approach. Hopefully someone will present a nicer solution. As you data is already in long format i would transform it to wide so that the within-subject correlations are preserved when resampling (there must be a way to sample in the long format).

If your data is df.

#ICC on your data
ICCbare(subject_id , interval_0_epochs , data = df)

# Reshape your data to wide - preserve wothin subject correlation when resampling
df$time <- ave(df$subject_id, list(df$subject_id), FUN=seq_along)
w.df <- reshape(df , timevar = "time" , idvar = "subject_id"  , direction = "wide" )  

# Quick check ---------------------------------------------------------
l.df <- reshape(w.df , direction = "long")
# Define new grouping factor for repeat id's when resamples (later)
l.df$grp <- 1:27
ICCbare(grp , interval_0_epochs.1 , data = l.df)$ICC #same as before

# Bootstrap function ---------------------------------------------
boot.fun <- function(dat , i) {
 newdf <- dat[i , ]
 m.newdf <- reshape(newdf , dir = "long", new.row.names = seq((ncol(dat)-1)*nrow(dat))) 
 m.newdf$grp <- seq(nrow(dat))
 ICCbare(grp , interval_0_epochs.1 , data = m.newdf)$ICC
}

boo1 <- boot(w.df , boot.fun , 2000)
boot.ci(boo1)

bootstrap ICC in R

1 Answers