Standard macro performance issues?

[apozzy]

I've been performance tuning some workflows, and noticed that some macros are the big performance problem. I tuned one batch macro and turned it into a standard macro, which reduced processing time from 13 minutes to 5, which was great. But then I noticed while working on the new standard macro in design mode, just running the dataset through the tools, that it runs in about a minute. And then I save and run it as a macro with the same data going in, and it takes almost 5 minutes.

I wasn't surprised the batch macro was much slower, it's iterating one batch at a time and I'm not surprised there's a performance hit, but the standard one makes no sense. It's the same tools running, but takes nearly 5 times as long. Is this normal? If so it makes macros fairly useless, which is unfortunate because I've been using them everywhere that needs this repeating code.

I could post the macro itself, but was really just wondering if this is a known flaw with Alteryx, and that we can always expect standard macros to run much slower than if they weren't macros. Is that the case?

thanks!

[IraWatt]

Hey @apozzy,

That is an interesting observation, Alteryx has performance profiling tools which you can enable from the runtime menu:

I did a quick test generating 10000000 and I seemed to get the same result from both the macro and non macro workflows:

This would suggest there is something more complex going on but I'm not sure. It will be interesting to see a breakdown from your workflow as to were all the time is being spent.

[apozzy]

Attached is my test. I feed the same data first to the macro in the container, and then later to the "details", which is the components from the macro itself. The time difference is dramatic. If I work with my entire dataset it's the difference between about 20 minutes and maybe 5. In this example I've limited it to a smaller batch of records, and the macro runs in 2+ minutes, the process itself runs in about 40 seconds when not wrapped in the macro. 

[IraWatt]

That is really odd, I'm wondering are all the tools in the macro taking 5x longer to run or is it one tool in particular?

[apozzy]

I'm not sure about the details yet, I haven't dug that far into it. I was kind of hoping to not have to dig into it, this shouldn't be an issue in the first place, but if it's essentially a flaw or bug with Alteryx then I'm hoping to find that out so I don't waste time on it. It seems like others would have noticed this, but maybe not. I've yet to be able to make it NOT run more slowly in a macro. Did you happen to run what I posted to see if you get the same type of results?

[IraWatt · Accepted solution]

@apozzy Ah didn't noticed you linked it sorry. I had a look, what I think is causing this is this:

In your bottom workflow it is set to 3 

In your macro it is set to 52

When I made the bottom workflow 52 it ran for 5 seconds not 1.

[apozzy]

Oh wow thank you that's a nice catch! The original code was built for months "3" being the typical default, and now after making it a more flexible parameter driven macro, I'm using it for weeks in this instance - thus the 52. I could see how the large difference might impact performance. I'm going to run some tests and play with those variables now - thank you!

[apozzy]

@IraWatt Thank you so much for catching that! It's definitely what was causing the issue, and I'm pleased to know this isn't an Alteryx issue. I figured others would have reported it before me if it were, but I just couldn't see something so obvious - it often takes a fresh set of eyes! Thanks again.

[IraWatt]

No worries @apozzy ! I am also glad its not an Alteryx issue 😅