Hello team,
I am using an iterative macro to generate statistical sales forecasts iteratively on subsets of my data. I have an R Tool embedded in the iterative macro with different predictive methods. The macro is pretty lightweight and mainly is just the R tool. I don't use the existing time series tools because I wanted more forecasting methods and my outputs formatted a certain way.
Here are the facts on my current state situation and what I am looking for in my future state situation. I think AMP Engine could the solution, but I am have not seen the results by enabling the engine in my workflow and I do not see any documentation or other threads addressing the R Tool with AMP engine.
Current State:
Forecast at business segment and item family level
300,000 records
~3000 iterations per workflow run
3 seconds per iteration
~3 hours workflow run time
Ideal Future State:
Forecast at business segment, item family, distributor, customer level
6.7 mil records
435,000 iterations per workflow
Minimal iteration time
These iterations would take the workflow 15 days to run given current iteration time/specs.
Not sure if I'm a mad man for wanting these gains. The AMP engine seems to be the ticket to solving this problem using Alteryx though. I have 12 worker threads and 15.6GB RAM available for my workflow on my desktop. I can get more than that on my company's Alteryx server. Dividing the iterations by 12 (given the worker threads available), it would take only 30 hours to realize the 435K iterations if 3 seconds per iteration.
The R Tool doesn't appear to be leveraging the AMP engine. Am I correct in this assessment (no information on the wiki)? Is there a plan to add the R tool in the future? Does anyone have an ideas on how to speed up these iterations? I have loosely looked into the handoff between R and Alteryx being less efficient, so that may be an opportunity for gains.
I appreciate any insight! I can't share the workflow due to company IP, but I can answer questions.
