Time Series Cross Validation for LASSO model

Question

As the title might suggest, my aim is to perform a Time Series Cross Validation using a L1 penalty (Lasso). As the data frame is a time series format, clearly the results of a time series analysis should be more appropriate than a normal cv.

Here are the lines of codes I tried

library(caret)
library(ggplot2)
library(pls)
economics
timeSlices <- createTimeSlices(1:nrow(economics), 
                               initialWindow = 36, horizon = 12, fixedWindow = FALSE)
trainSlices <- timeSlices[[1]]
testSlices <- timeSlices[[2]]

This allows to create two slices for a training set (always getting one more observations) while keeping constant the lenght of the test set. The problem arises here

plsFitTime <- train(unemploy ~ pce + pop + psavert,
                data = economics[trainSlices[[1]],],
                method = "glmnet",
                alpha = 1)

Here the error

Something is wrong; all the RMSE metric values are missing:

      RMSE        Rsquared        MAE     
 Min.   : NA   Min.   : NA   Min.   : NA  
 1st Qu.: NA   1st Qu.: NA   1st Qu.: NA  
 Median : NA   Median : NA   Median : NA  
 Mean   :NaN   Mean   :NaN   Mean   :NaN  
 3rd Qu.: NA   3rd Qu.: NA   3rd Qu.: NA  
 Max.   : NA   Max.   : NA   Max.   : NA  
 NA's   :9     NA's   :9     NA's   :9    
Error: Stopping
In addition: Warning message:
In nominalTrainWorkflow(x = x, y = y, wts = weights, info = trainInfo,  :
  There were missing values in resampled performance measures.

I really don t get what might be wrong with that.

My final objective would be then to perform

pred <- predict(plsFitTime,economics[testSlices[[1]],])
true <- economics$unemploy[testSlices[[1]]]

Any suggestion with that?

This appears to be primarily a methods question for which solid answers are best asked for on CrossValidated.com. In the meantime you might want to do your own searching there. And check out: books.google.es/books/about/… — IRTFM

StupidWolf StupidWolf · Accepted Answer · 2020-02-14T18:55:21

If you look at the warnings:

50: model fit failed for Resample17: alpha=0.55, lambda=42.28 Error in (function (x, y, family = c("gaussian", "binomial", "poisson", :
formal argument "alpha" matched by multiple actual arguments

By default, training for glmnet is done on lambda and alpha. And the alpha you have specified clashes with that generated by caret.

If you want a fixed alpha, it should be specified in the tuneGrid.

t_grid = expand.grid(lambda=seq(0,1,by=0.1),alpha=1)
plsFitTime <- train(unemploy ~ pce + pop + psavert,
                    data = economics[trainSlices[[1]],],
                    method = "glmnet",
                    tuneGrid = t_grid)

You can check the hyper parameters here

Time Series Cross Validation for LASSO model

1 Answers