Linear interpolation for missing data in R

Question

I have a data set that has recorded pupil size across conditions. As expected blink artifiacts are present in the data (represented by extreme drop in pupil size value during blink-onset and '-1' when the eye is completely closed, followed by a gradual rise in pupil size again).

To the best of my understanding a linear interpolation of blink artefacts based on values before and after the blink onset is an acceptable way to smooth blinks in pupil size data.

Example Data:

df<-structure(list(Pupil_Avg = c(8.984, 8.984, 8.988001, 8.988001, 
8.978001, 8.978001, 8.9780005, 8.9780005, 8.9780005, 8.9780005, 
8.9780005, 8.9800005, 8.981, 8.9810005, 8.979, 8.979, 8.979, 
8.979, 8.979, 8.979, 8.979, 8.979, 8.979, 8.979, 8.9750005, 8.964, 
8.964001, 8.9660005, 8.9650005, 8.964001, 8.9610005, 8.9620005, 
8.9630005, 8.9630005, 8.963001, 8.963001, 8.96, 8.96, 8.9600005, 
8.962, 8.962, 8.969001, 8.9730005, 8.9670005, 8.9610005, 8.9610005, 
8.9610005, 8.9610005, 8.9610005, 8.9520005, 8.949001, 8.9450005, 
8.9450005, 8.9400005, 8.933001, 8.938001, 8.9510005, 8.956001, 
8.956001, 8.956001, 8.956001, 8.956001, 8.943001, 8.9280005, 
8.9280005, 8.9280005, 8.9280005, 8.9280005, 8.9350005, 8.9470005, 
8.95, 8.9530005, 8.957001, 8.9480005, 8.946, 8.944, 8.944, 8.9460005, 
8.9460005, 8.9480005, 8.9440005, 8.941, 8.938, 8.9280005, 8.9280005, 
8.9280005, 8.9280005, 8.9280005, 8.9280005, 8.929, 8.929, 8.9280005, 
8.9280005, 8.9210005, 8.918, 8.919, 8.92, 8.92, 8.92, 8.9170005, 
8.9100005, 8.9100005, 8.92, 8.9220005, 8.9220005, 8.9100005, 
8.9100005, 8.912, 8.912, 8.912, 8.912, 8.912, 8.9340005, 8.9610005, 
8.958001, 8.985, 8.978, 8.9880005, 8.9880005, 9.014, 9.014, 9.014, 
9.014, 9.014, 8.9740005, 8.9520005, 8.789, 8.6460005, 8.471001, 
8.326, 8.129001, 7.862, 7.862, 7.862, 7.862, 7.862, 7.862, 7.174, 
6.6910005, 6.518, 2.461, 2.182, 1.942, 1.942, -1, -1, -1, -1, 
-1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, 
-1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, 
-1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, 
-1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, 
-1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, 
-1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, 
-1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, 
-1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, 
-1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, 
-1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, 
-1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, 
-1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, 
-1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, 
-1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, 
-1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, 
-1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, 
-1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, 
-1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, 
-1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, 
-1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, 
-1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, 
-1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, 
-1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, 
-1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, 
-1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, 
-1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, 
-1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, 
-1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, 
-1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, 1.487, -1, 
-1, -1, -1, 2.202, 2.202, 2.281, 2.344, 6.265, 6.378, 6.4910005, 
6.8980005, 6.925, 7.04, 7.591, 7.7900005, 7.8470005, 7.978001, 
7.978001, 7.978001, 7.978001, 7.978001, 8.159, 8.1300005, 8.154, 
8.227, 8.281, 8.3160005, 8.353, 8.4430005, 8.4970005, 8.4970005, 
8.4970005, 8.4970005, 8.5150005, 8.6390005, 8.9930005, 9.0110005, 
9.0330005, 9.035, 9.0360005), BLINK_IDENTIFICATION = c(NA, NA, 
NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, 
NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, 
NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, 
NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, 
NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, 
NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, 
NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, 
NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, 
NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, "Blink Onset", NA, NA, 
NA, "Eye Closed", "Eye Closed", "Eye Closed", "Eye Closed", "Eye Closed", 
"Eye Closed", "Eye Closed", "Eye Closed", "Eye Closed", "Eye Closed", 
"Eye Closed", "Eye Closed", "Eye Closed", "Eye Closed", "Eye Closed", 
"Eye Closed", "Eye Closed", "Eye Closed", "Eye Closed", "Eye Closed", 
"Eye Closed", "Eye Closed", "Eye Closed", "Eye Closed", "Eye Closed", 
"Eye Closed", "Eye Closed", "Eye Closed", "Eye Closed", "Eye Closed", 
"Eye Closed", "Eye Closed", "Eye Closed", "Eye Closed", "Eye Closed", 
"Eye Closed", "Eye Closed", "Eye Closed", "Eye Closed", "Eye Closed", 
"Eye Closed", "Eye Closed", "Eye Closed", "Eye Closed", "Eye Closed", 
"Eye Closed", "Eye Closed", "Eye Closed", "Eye Closed", "Eye Closed", 
"Eye Closed", "Eye Closed", "Eye Closed", "Eye Closed", "Eye Closed", 
"Eye Closed", "Eye Closed", "Eye Closed", "Eye Closed", "Eye Closed", 
"Eye Closed", "Eye Closed", "Eye Closed", "Eye Closed", "Eye Closed", 
"Eye Closed", "Eye Closed", "Eye Closed", "Eye Closed", "Eye Closed", 
"Eye Closed", "Eye Closed", "Eye Closed", "Eye Closed", "Eye Closed", 
"Eye Closed", "Eye Closed", "Eye Closed", "Eye Closed", "Eye Closed", 
"Eye Closed", "Eye Closed", "Eye Closed", "Eye Closed", "Eye Closed", 
"Eye Closed", "Eye Closed", "Eye Closed", "Eye Closed", "Eye Closed", 
"Eye Closed", "Eye Closed", "Eye Closed", "Eye Closed", "Eye Closed", 
"Eye Closed", "Eye Closed", "Eye Closed", "Eye Closed", "Eye Closed", 
"Eye Closed", "Eye Closed", "Eye Closed", "Eye Closed", "Eye Closed", 
"Eye Closed", "Eye Closed", "Eye Closed", "Eye Closed", "Eye Closed", 
"Eye Closed", "Eye Closed", "Eye Closed", "Eye Closed", "Eye Closed", 
"Eye Closed", "Eye Closed", "Eye Closed", "Eye Closed", "Eye Closed", 
"Eye Closed", "Eye Closed", "Eye Closed", "Eye Closed", "Eye Closed", 
"Eye Closed", "Eye Closed", "Eye Closed", "Eye Closed", "Eye Closed", 
"Eye Closed", "Eye Closed", "Eye Closed", "Eye Closed", "Eye Closed", 
"Eye Closed", "Eye Closed", "Eye Closed", "Eye Closed", "Eye Closed", 
"Eye Closed", "Eye Closed", "Eye Closed", "Eye Closed", "Eye Closed", 
"Eye Closed", "Eye Closed", "Eye Closed", "Eye Closed", "Eye Closed", 
"Eye Closed", "Eye Closed", "Eye Closed", "Eye Closed", "Eye Closed", 
"Eye Closed", "Eye Closed", "Eye Closed", "Eye Closed", "Eye Closed", 
"Eye Closed", "Eye Closed", "Eye Closed", "Eye Closed", "Eye Closed", 
"Eye Closed", "Eye Closed", "Eye Closed", "Eye Closed", "Eye Closed", 
"Eye Closed", "Eye Closed", "Eye Closed", "Eye Closed", "Eye Closed", 
"Eye Closed", "Eye Closed", "Eye Closed", "Eye Closed", "Eye Closed", 
"Eye Closed", "Eye Closed", "Eye Closed", "Eye Closed", "Eye Closed", 
"Eye Closed", "Eye Closed", "Eye Closed", "Eye Closed", "Eye Closed", 
"Eye Closed", "Eye Closed", "Eye Closed", "Eye Closed", "Eye Closed", 
"Eye Closed", "Eye Closed", "Eye Closed", "Eye Closed", "Eye Closed", 
"Eye Closed", "Eye Closed", "Eye Closed", "Eye Closed", "Eye Closed", 
"Eye Closed", "Eye Closed", "Eye Closed", "Eye Closed", "Eye Closed", 
"Eye Closed", "Eye Closed", "Eye Closed", "Eye Closed", "Eye Closed", 
"Eye Closed", "Eye Closed", "Eye Closed", "Eye Closed", "Eye Closed", 
"Eye Closed", "Eye Closed", "Eye Closed", "Eye Closed", "Eye Closed", 
"Eye Closed", "Eye Closed", "Eye Closed", "Eye Closed", "Eye Closed", 
"Eye Closed", "Eye Closed", "Eye Closed", "Eye Closed", "Eye Closed", 
"Eye Closed", "Eye Closed", "Eye Closed", "Eye Closed", "Eye Closed", 
"Eye Closed", "Eye Closed", "Eye Closed", "Eye Closed", "Eye Closed", 
"Eye Closed", "Eye Closed", "Eye Closed", "Eye Closed", "Eye Closed", 
"Eye Closed", "Eye Closed", "Eye Closed", "Eye Closed", "Eye Closed", 
"Eye Closed", "Eye Closed", "Eye Closed", "Eye Closed", "Eye Closed", 
"Eye Closed", "Eye Closed", "Eye Closed", "Eye Closed", "Eye Closed", 
"Eye Closed", "Eye Closed", "Eye Closed", "Eye Closed", "Eye Closed", 
"Eye Closed", "Eye Closed", "Eye Closed", "Eye Closed", "Eye Closed", 
"Eye Closed", "Eye Closed", "Eye Closed", "Eye Closed", "Eye Closed", 
"Eye Closed", "Eye Closed", "Eye Closed", "Eye Closed", "Eye Closed", 
"Eye Closed", "Eye Closed", "Eye Closed", "Eye Closed", "Eye Closed", 
"Eye Closed", "Eye Closed", "Eye Closed", "Eye Closed", "Eye Closed", 
"Eye Closed", "Eye Closed", "Eye Closed", "Eye Closed", "Eye Closed", 
"Eye Closed", "Eye Closed", "Eye Closed", "Eye Closed", "Eye Closed", 
"Eye Closed", "Eye Closed", "Eye Closed", "Eye Closed", "Eye Closed", 
"Eye Closed", "Eye Closed", "Eye Closed", "Eye Closed", "Eye Closed", 
"Eye Closed", "Eye Closed", "Eye Closed", "Eye Closed", "Eye Closed", 
"Eye Closed", "Eye Closed", "Eye Closed", "Eye Closed", "Eye Closed", 
"Eye Closed", "Eye Closed", "Eye Closed", "Eye Closed", "Eye Closed", 
"Eye Closed", "Eye Closed", "Eye Closed", "Eye Closed", "Eye Closed", 
"Eye Closed", "Eye Closed", "Eye Closed", "Eye Closed", "Eye Closed", 
"Eye Closed", "Eye Closed", "Eye Closed", "Eye Closed", "Eye Closed", 
"Eye Closed", "Eye Closed", "Eye Closed", "Eye Closed", "Eye Closed", 
"Eye Closed", "Eye Closed", "Eye Closed", "Eye Closed", "Eye Closed", 
"Eye Closed", "Eye Closed", "Eye Closed", "Eye Closed", "Eye Closed", 
"Eye Closed", "Eye Closed", "Eye Closed", "Eye Closed", "Eye Closed", 
"Eye Closed", "Eye Closed", "Eye Closed", "Eye Closed", "Eye Closed", 
"Eye Closed", "Eye Closed", "Eye Closed", "Eye Closed", "Eye Closed", 
"Eye Closed", "Eye Closed", "Eye Closed", "Eye Closed", "Eye Closed", 
"Eye Closed", "Eye Closed", "Eye Closed", "Eye Closed", "Eye Closed", 
"Eye Closed", "Eye Closed", "Eye Closed", "Eye Closed", "Eye Closed", 
"Eye Closed", "Eye Closed", "Eye Closed", "Eye Closed", "Eye Closed", 
"Eye Closed", "Eye Closed", "Eye Closed", "Eye Closed", "Eye Closed", 
"Eye Closed", "Eye Closed", "Eye Closed", "Eye Closed", "Eye Closed", 
"Eye Closed", "Eye Closed", "Eye Closed", "Eye Closed", "Eye Closed", 
"Eye Closed", "Eye Closed", "Eye Closed", "Eye Closed", "Eye Closed", 
"Eye Closed", "Eye Closed", "Eye Closed", "Eye Closed", "Eye Closed", 
"Eye Closed", "Eye Closed", "Eye Closed", "Eye Closed", "Eye Closed", 
"Eye Closed", "Eye Closed", "Eye Closed", "Eye Closed", "Eye Closed", 
"Eye Closed", "Eye Closed", "Eye Closed", "Eye Closed", "Eye Closed", 
"Eye Closed", "Eye Closed", "Eye Closed", "Eye Closed", "Eye Closed", 
"Eye Closed", "Eye Closed", "Eye Closed", "Eye Closed", "Eye Closed", 
"Eye Closed", "Eye Closed", "Eye Closed", "Eye Closed", "Eye Closed", 
"Eye Closed", "Eye Closed", "Eye Closed", "Eye Closed", "Eye Closed", 
"Eye Closed", "Eye Closed", "Eye Closed", "Eye Closed", "Eye Closed", 
"Eye Closed", "Eye Closed", "Eye Closed", "Eye Closed", "Eye Closed", 
NA, "Eye Closed", "Eye Closed", "Eye Closed", "Eye Closed", "Blink Offset", 
NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, 
NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, 
NA, NA, NA, NA)), class = c("tbl_df", "tbl", "data.frame"), row.names = c(NA, 
-651L))

As you can see in the BLINK_IDENTIFICATION column the onset (row 141) and offset (row 615) of a blink have been identified. When the eye is closed (-1 in column Pupil_Avg) has also been identified between these labels.

My main aim is to linear interpolate the data from "blink onset" until x-number of observations after "offset" based on values before and after the blink starts and ends.

Can anyone suggest a way to do this? There appear to be various interpolation functions in R (for example, approx()). I'm just not 100% sure how to implement them. Ideally I'd like to avoid using a for loop if possible because my full data set is millions of rows long, but I'll work with a for loop if there's no other way.

EDIT: Here's an example of a function working with the above data set to interpolate the blink:

install.packages("zoo")
library('zoo')
library(dplyr)

# replace every '-1' value with 'NA'
df$Pupil_Avg[df$Pupil_Avg == -1] <- NA

df<-df%>%mutate(approx = na.approx(Pupil_Avg))

The above results in interpolation from the first eye-closed (NA in column Pupil_Avg onwards). This is a start but I need the interpolation to begin at the row before "Blink Onset" as previously stated.

Thanks for your time.

Answer 1

You can do it this way:

library("imputeTS")
onset <- which("Blink Onset" == df$BLINK_IDENTIFICATION)[[1]]
offset <- which("Blink Offset" == df$BLINK_IDENTIFICATION)[[1]]

df$BLINK_IDENTIFICATION[onset:offset] <- -1
df$Pupil_Avg[df$BLINK_IDENTIFICATION == -1] <- NA
df <- na.interpolation(df$Pupil_Avg, option ="linear")

This works, if you have exactly 1 blink_onset and offset in your time series. If you have multiple in your series just remove the [[1]] in the which function, then you get a list of all occurrences. You then use this two vectors of occurrences to set the respective parts of your series to NA.

The example is now with the interpolation function of imputeTS instead of zoo. Both do the job. The function of imputeTS is slightly faster, but probably the other code takes most of the computing time anyway. You can set option to "spline" and "stine" to change from linear interpolation to Spline or Stineman interpolation. (na.spline if you are using zoo)