Adding predictions to tidy models and plotting

Question

My goal is quite simple, it is to predict (with confidence intervals) future values from many linear models then plot them with ggplot2 as an extension to the real data.

I have a datset with four variables: Area (factor), date, series (nominal; A, B etc), and value(continuous). I have estimated a simple linear model per area-series combination like this, using broom, tidyr and purrr:

  group_by(area, series) %>%
  nest()

le_model<-function(df){
  lm(value ~ date, data=df)
}


models <-by_area_series %>% 
  mutate(leGrad=map(data, le_model))


models<-models%>%
  mutate(
    tidy_le = map(leGrad, broom::tidy),
    glance_le = map(leGrad, broom::glance),
    augment_le = map(leGrad, broom::augment),
    rsq_le = glance_le %>% map_dbl("r.squared"),
  )

I have new dates out-with the existing data range e.g. new.dates=data.frame(date=c((Sys.Date()+7), (Sys.Date()+10), (Sys.Date()+14)))

I wish to predict the corresponding value at each new date for each model, and plot the data and predictions on the same plot (facet-wrapped by area), so I need the predictions and their SE to be put back into the same dataset (I presume). So far I've just added this:

  predict((lm(value~date, data=df)), newdata=new.dates, se.fit=T)
}
preds<-by_area_series %>%
  mutate(preds = map(data, pred))

I've been fiddling about with list notations like preds[4][[1]][[1]]$fit but I'm not able to find a neat solution that applies to all models in one call.

Any help on a)getting the predicted values and their SE neatly into the original data, then b) using ggplot2 to plot the lm and the predicted values on the same plot greatly appreciated!!

Arthur Yip Arthur Yip · Accepted Answer · 2021-04-16T11:43:18

Without a reprex, it's hard to show a replicable solution. But you should be able to use predict the same way you've used tidy, glance, augment from your models object rather than from your by_area_series data. Get both the predicted value and se using options here: https://parsnip.tidymodels.org/reference/predict.model_fit.html

Adding predictions to tidy models and plotting

1 Answers