Time Series Model Score MAPE vs Forecast MAPE very different

Hello Alteryx Community,

I need help understanding why the MAPE for my ARIMA forecast is so much higher than the MAPE in the TS Compare model tool using the same ARIMA tool.

I am trying to forecast revenue(y) by day using units sold(cv) as a covariate. In both the TS Model and the TS Forecast I use the same training data and covariate data but the MAPE jumps from 22% to 50% and I cannot understand why for the life of me.

I have attached the workflow and data as a packaged workbook. Any help is greatly appreciated. Thank you so much.

TS Model vs Forecast MAPE.yxzp

Time Series

Accepted answers

RolandSchubert

You have to create a prediction for cv first (as you did) and use the predicted value in forecast calculate (you also did that). But it sees, ARIMA (or at least ARIMA in the configuration you used) does not produce a reliable prediction for cv. I see basically two options now. You can use ARIMA and customize the model manually (e.g. modify parameters for seasonal and non-seasonal components) or you try ETS for cv and feed the results to your ARIMA model. What do you think?

All comments

joshuaburkhow

Hey @Billigans

Take a look at some of the modifications I made here for you.

a) I did change a bit of the layout so you don't have to do any of the MAPE calcs as the tools can do those for you 😉

b) I also took a look at your data and found that you will benefit from tuning your model to use the box-cox transformation.

c) overall the message is much the same (your covariates have a decent impact) but with a better model.

Let me know if this helps!

Joshua

TS Model vs Forecast MAPE_JB.yxzp

Billigans

@joshuaburkhow,

Thank you for taking the time to look at my model. I really appreciate it.

As a point of clarification, I was using the manual calculations to calculate the MAPE of the output of the forecast because I needed to use the forecasted values of units(cv) as the covariate, not the actual unit sales themselves. I won't have the future period unit sales when I'm doing the actual forecast.

I am trying to understand why the ARIMA tool, which doesn't use the future periods, when fed into the TS Model Compare tool creates such a good forecast, but when I feed the information to a TS Forecast tool the forecast flatlines.

I have attached a simplified workflow to help show the problem if you look at the forecast in the TS Model vs the Forecast from the TS Forecast tool you can see a major difference.

Thanks again for your help.

TS Model vs Forecast MAPE_v2.yxzp

RolandSchubert

Hi @Billigans ,

the difference seems to be the data for the covariate. The same model is used for both the TS Covariate Forecast tool and the TS Compare tool, but the data for the covariate is different. Covariate data for the TS Covariate Forecast tool is generated using a TS Arima + TS Forecast tool resulting in a widely flat line ("O" anchor of TS Forecast tool) thereby leading to a flatline for the forecast. Covariate data for the TS Compare tool is the "original" test data (records 1540-1600) - these data vary widely and the result is a varying forecast for the target value.

I've modified the workflow and added a second TS Covariate Forecast tool, using the same model as the first one, but the covariate data used for the TS Compare tool. In my opinion, forecast should be closer to what you expect. Hope this is helpful.

Best,

Roland

TS Model vs Forecast MAPE_v3.yxzp

Quick Links

This months top contributors

atcodedog05 19458

Qiu 15866

binu_acs 15708

MarqueeCrew 13708

apathetichell 13703