Missing values - Arima model

2
votes

I have a daily time series about the sales of a product, my series start from 01/01/2016 until 31/08/2017.

Considering that it is a six-day week (my week starts on Monday and ends Saturday) there is no data for Sundays, I understand that before running an Arima model I need first to fill the missing values. This is where I need help: I've read that I can fill the missing values with na.approx or NA, but I do not know how to do that.

You could see my series here:

https://drive.google.com/file/d/0BzIf8XvzKOGWSm1ucUdYUVhfVGs/view?usp=sharing

As you can see, there is no data for Sundays. I need to know how to fill the missing values to run an Arima model and be able to forecast what's left of 2017.

2
rtime-seriesmissing-datazooarima
See if the solutions here work for you? - Z.Lin
So you essentially want to remove all Sundays from calender and assume Monday follows Saturday directly? Perhaps you can just number your days like 1 = first Mon, 7 = next Mon, and so on? As a side note: I am a little skeptical that pure ARIMA can describe the weekly cycle well. - Ott Toomet
Hi, thanks for your comment. I meant, my series has zero-values each Sunday, because it's about the sales of a product, and this shop doesn't work on Sundays, so the expected value for all Sundays in the future is zero. I do have the data from monday to saturday, but I am really confused about how to run an arima model considering that I have these zero-values on Sundays - Jose R.
Does my answer seem helpful to you? - acylam
Yes !! Thank you so so much for your asnwer !! .. It was exactly what I needed !!! Thank you so much !!! - Jose R.

2 Answers

6
votes

Here're three ways of doing it:

library(lubridate)
library(xts)
library(dplyr)
library(forecast)

df$Date = mdy(df$Date)

Removing Sundays:

ts_no_sunday = df %>%
  filter(wday(df$Date) != 1) %>%
  {xts(.$Units, .$Date)}

plot(ts_no_sunday)

no_sunday_arima = auto.arima(ts_no_sunday)

plot(forecast(no_sunday_arima, h = 10))

enter image description here enter image description here

Replace Sundays with NAs:

ts_sunday = df %>%
  mutate(Units = replace(Units, which(wday(df$Date) == 1), NA)) %>%
  {xts(.$Units, .$Date)}

plot(ts_sunday)

sunday_arima = auto.arima(ts_sunday)

plot(forecast(sunday_arima, h = 10))

enter image description here enter image description here

Interpolate Sundays:

ts_interp = df %>%
  mutate(Units = replace(Units, which(wday(df$Date) == 1), NA),
         Units = na.approx(Units)) %>%
  {xts(.$Units, .$Date)}

plot(ts_interp)

interp_arima = auto.arima(ts_interp)

plot(forecast(interp_arima, h = 10))

enter image description here enter image description here

Notes:

As one can see, they produce different forecasts. This is because the first time series is irregular, the second is a regular time series with missing values, and the third is a regular time series with interpolated data. In my opinion, a better way to deal with missing values is to interpolate before fitting an ARIMA, since ARIMA assumes that the time series is regularly spaced. This however, also depends on whether your "missing" data points are actually missing, and not a stop in activity. The former should be treated with interpolation, while for the latter you might just be better off removing Sundays and treat the time series as if Sundays don't exist.

See this discussion on How to handle nonexistent or missing data? and this on Using the R forecast package with missing values and/or irregular time series

0
votes

In principle you could use a imputeTS (for filling the NAs) - forecast (for doing the forecast) combination.

It can be done quite easily:

library("imputeTS")
library("forecast")
ts_sunday %>% na_kalman() %>% auto.arima() %>% forecast(h=10)

Would do the job. But in this specific case this would be a bad idea. If the data would be missing at random you could consider this solution. But it is not - it's always Sundays that are missing. Some time series models can also deal with NAs and still build a model.(but the drawbacks are nearly the same as with the previous solution). How should a model treat Sundays, since it is never observed... Probably the best solution (from a statistics perspective) is what avid_useR described in another answer as removing Sundays completely. If you don't need Sundays - and anyway have no values for Sundays then just remove them. But usually this sooner or later leads to the next question ..'how to treat public holidays' - which are also often NA. Also always keep your problem in mind - one solution might be a fit for one setting - for another it might not make sense.