Alteryx Designer Desktop Discussions
Find answers, ask questions, and share expertise about Alteryx Designer Desktop and Intelligence Suite.
I am in the process to examining the runtime of our Alteryx model and trying to find areas where we could optimize.
I am running through a large set of data (~32mil records with ~100 fields, roughly 40GBs) in a workflow that contains multiple macros. During my testing, I found that that for all of the macro used in the model, the Macro Output tools seem to take much longer than I expect, and often be one of the highest costing in the macros.
I have attached an example of one of the macros, which has the top container turned off via Dynamic Container. So the run would simply take the bottom path, which just assign 1s and 0s to pad out fields,
However, examining the Profiling in the results window, the line that corresponds to the Macro Output tool here (#45) took 4.3mins to complete. The Formula tool (#131) and Union tool (#150) took 33secs and 50secs, respectively. The latter two tools are were I would expect most of the time should be taken.
Info: MACRO_DYNAMIC_JOIN_ACTUALS_ADJUSTMENTS (1050): Record #1: Tool #45: Profile Time: 258168.14ms, 67.83%
Info: MACRO_DYNAMIC_JOIN_ACTUALS_ADJUSTMENTS (1050): Record #1: Tool #150: Profile Time: 49628.81ms, 13.04%
Info: MACRO_DYNAMIC_JOIN_ACTUALS_ADJUSTMENTS (1050): Record #1: Tool #131: Profile Time: 33370.99ms, 8.77%
Info: MACRO_DYNAMIC_JOIN_ACTUALS_ADJUSTMENTS (1050): Record #1: Tool #107: Profile Time: 28035.28ms, 7.37%
Info: MACRO_DYNAMIC_JOIN_ACTUALS_ADJUSTMENTS (1050): Record #1: Tool #130: Profile Time: 8601.93ms, 2.26%
Info: MACRO_DYNAMIC_JOIN_ACTUALS_ADJUSTMENTS (1050): Record #1: Tool #153: Profile Time: 2803.76ms, 0.74%
Another example (macro not attached here) is
Info: MACRO_IBNR_CHOICE_OF_CALCULATION (883): Record #1: Output IBNR Factors: Profile Time: 157926.52ms, 20.50%
This is slightly better, but still taking 2.6mins to run through.
Note: This is run on 2020.3 and not using AMP engine (it's causing us a lot of error atm).
Hi @Harley ,
The chances are the macro output tools take the longest because these are the end result of a batch macro. The batch macro will loop through every record based on the unique values of a control parameter and will then, crucially, union the outcome of all records together. This union process is carried out in the Macro Output tool. In effect, that is unioning the entirety of the batch macro process together into a single output stream. This is most likely why that tool takes the longest to run.
M.
@mceleavey Appreciate your response.
However, although this is a batch macro, we have not grouped the data by anything. So it should just run everything in one go.
Hi @Harley ,
I can't see your data so I can't really help but looking at the macro, you're grouping by Run Controller Location, Product Type and Non-Standard Claim Adjustments Toggle. Each combination will be unioned together.
If you aren't grouping, then that represents a single row, which is difficult to believe given your data size.
Can you confirm there is only one combination of those things?
M.
