Iteration Macro taking 20 hours to run! Please help...

[bryantv_autonation]

Hi,

This is the first macro I've written, and I think it is not that complicated of a macro.  However, it took just over 20 hours to run!  I'm not sure if it is my computer that doesn't have the computing resources, or if there is something wrong with my macro.  The goal is to take the first record of the bottom input and match it to a record in the top data set.  Then, perform a series of formulas and output the result back to the workflow.  The second output from the top is to output any records that did not match.  The bottom output loops back to the bottom input.  It is the original data set, less the records that have ran through the process. Below is a screenshot of the macro. 

For security purposes, I cannot share the data.  However, the top input has 37 fields and 576,710 records.  The bottom input has 14 fields and 58,621 records.  It took just over 20 hours for the macro to run, and the output was as expected.  I just want to know if anyone sees anything wrong with the macro that would cause it to take so long to run.

Thanks so much!

[apathetichell]

for the joins can you confirm that no new columns from the top datastream are flowing to the looped data output (ie the join directly prior the looped macro output  consists entirely of the fields in the original input - ie 14 fields with the dynamic/unknown box unchecked)?

How many gb of ram do you have - and after 20 hours did the output look correct?

[bryantv_autonation]

Yes, in the last join prior to the looped output, I am keeping only the fields from the right join input.  The final data output did look correct, it just took forever.  My laptop has 8GB of RAM.  Our company does most computing of this type on our servers, but my Alteryx license is a local install license so it is running on my laptop.  I can ask for a more powerful laptop, similar to what our IT associates use...but, I wanted to make sure the problem wasn't with my macro before making such a request.

[gabrielvilella]

It took 20h, but for how many iterations? You set the limit to like 2 iterations and see if what you are getting is expected or not.

[apathetichell · Accepted solution]

8gb of ram isn't enough. This performance is bad but not totally unexpected. If you need to run this locally shut down every other system on the machine - like everything. 

Alteryx works by reading everything into ram and that usually makes things faster - I'm surprised it even finished. A good test might be keeping open resource monitor and seeing how long it takes for the ram to basically max out.