Hourly predictions

[gianip]

Hi

I have a data set which looks like this:

ID	Date-Hour-Minute	VALUE
1	03.08.2020 00:50	42621
2	03.08.2020 01:50	69828
3	03.08.2020 02:50	70897
4	03.08.2020 03:50	73074
5	03.08.2020 04:50	741136
6	03.08.2020 05:50	741491
7	03.08.2020 06:50	818614
8	03.08.2020 07:50	845002
9	03.08.2020 08:50	852243
10	03.08.2020 09:50	862989
11	03.08.2020 10:50	878809
12	03.08.2020 11:50	886333
13	03.08.2020 12:50	905734
14	03.08.2020 13:50	908544
15	03.08.2020 14:50	909821
16	03.08.2020 15:50	910864
17	03.08.2020 16:50	911989
18	03.08.2020 17:50	912165
19	03.08.2020 18:50	912341
20	03.08.2020 20:50	931256
21	03.08.2020 21:50	931410
22	03.08.2020 22:50	931654
23	03.08.2020 23:50	931848
24	04.08.2020 00:50	6298
25	04.08.2020 01:50	83324

The dataset contains data back 1 year (rolling) and ID (random number ascending), Datetime and the Value. I have a value for every hour per day (24 per day).

What I want to achieve is to predict the values for the upcoming days considering the hours, means I like to predict 24 values per day in near future considering the seasonality (e.g. higher values at month ends etc.)

Can somebody bring me on the right path how I can achieve this? I have tried by using the tools ARIMA or ETS, but was not able to achieve the output i want.
The output should be in the same format as the input, so that I can use the daily / hourly values of the future as thresholds and feed into a DB and then do a comparisonwith the real values during the days in future.

Hope, somebody can help.

Regards

Giani

[jrgo]

Hi @gianip 

These type of questions don't usually get much replies since designing a predictive model is a bit more involved than just dropping a few tools and getting your prediction, or a good prediction to be exact. 

That said, and with some down time at the moment, here's a general approach (workflow attached). The first part is looking that the quality of your data is solid and that your data is not missing any intervals you're looking to create your forecast on. in this case, there was one missing. This would have caused the issues with the forecast data as the periods and sub-periods it produced would have been misrepresented. I.e. a forecast for hour 0500 should actually have been for hour 0600. 

The forecast it generated is not ideal, but likely because there wasn't enough historical data to produce a good prediction. That said, you'll likely have to go through some trial/error to fine-tune the model configurations till you get a good fit, which is where my help stops 🙂. 

Hope this helps!

Jimmy

[gianip]

Hi @jrgo 

Wow, I'm impressed how fast I get a first reply on my post. 

I think with your help I understand more about how to start with this project and how to use the tools. The thing with filling the gaps is something that I didn't know.

So, thank you very much for guiding me in a direction. Very appreciated.

Maybe I'll come back in later stage with some additional questions when the workflow development is more mature 🙂

Regards