Thursday, 10 October 2019

How to perform multi-step out-of-time forecast which does not involve refitting the ARIMA model?

I have an already existing ARIMA (p,d,q) model fit to a time series data (for ex, data[0:100]) using python. I would like to do forecasts (forecast[100:120]) with this model. However, given that I also have the future true data (eg: data[100:120]), how do i ensure that the multi-step forecast takes into account the future true data that I have instead of using the data it forecasted?

In essence, when forecasting i would like forecast[101] to be computed using data[100] instead of forecast[100].

I would like to avoid refitting the entire ARIMA model at every time step with the updated "history".

I fit the ARIMAX model as follows:

train, test = data[:100], data[100:]
ext_train, ext_test = external[:100], external[100:]
model = ARIMA(train, order=(p, d, q), exog=ext_train)
model_fit = model.fit(displ=False)

Now, the following code allows me to predict values for the entire dataset, including the test

forecast = model_fit.predict(end=len(data)-1, exog=external, dynamic=False)

However in this case after 100 steps, the ARIMAX predicted values quickly converge to the long-run mean (as expected, since after 100 time steps it is using the forecasted values only). I would like to know if there is a way to provide the "future" true values to give better online predictions. Something along the lines of:

forecast = model_fit.predict_fn(end = len(data)-1, exog=external, true=data, dynamic=False)

I know I can always keep refitting the ARIMAX model by doing

historical = train
historical_ext = ext_train
predictions = []

for t in range(len(test)):
    model = ARIMA(historical, order=(p,d,q), exog=historical_ext)
    model_fit = model.fit(disp=False)
    output = model_fit.forecast(exog=ext_test[t])[0]
    predictions.append(output)
    observed = test[t]
    historical.append(observed)
    historical_ext.append(ext_test[t])

but this leads to me training the ARIMAX model again and again which doesn't make a lot of sense to me. It leads to using a lot of computational resources and is quite impractical. It further makes it difficult to evaluate the ARIMAX model cause the fitted params keep on changing every iteration.

Is there something incorrect about my understanding/use of the ARIMAX model?



from How to perform multi-step out-of-time forecast which does not involve refitting the ARIMA model?

No comments:

Post a Comment