Large nr of tools = slow down?

[jeffv]

Hi,

I've noticed as I add larger number of tools to a workflow 200+  that going from one tool to another takes 10-15 seconds.  Is this normal?  I have a beefy computer with 32Gb memory but it slows down quite a bit to make me want to split it up into multiple workflows..  Might there be a setting or some known tip to help the situation?

Thanks,

Jeff

[MarqueeCrew]

As I understand Alteryx.  Each time you add a tool (an angel gets his/her wings) the canvas passes data (1 record) through the tools.  Having 200+ tools gives you ample opportunities to wait.  Perhaps you might want to use containers to disable sections of the workflow to speed things up.  Another idea might be to CACHE the workflow just prior to where you are.

Cheers,

Mark

[jpoz]

This sounds like a defect. We want the Designer to perform quickly on large workflows, even with thousands of tools.

[jeffv]

Mark,

Thanks for response.  Just so I understand.  The slow down I'm concerned with is in development time not execution time.  Adding containers, as I understand it, will allow me to disable/enable as I need them I have to enable all the containers to get it to work and thus the slow down when I need to go from one tool to another for checking or updating something minor.

Jeff

[jpoz]

Can you provide a workflow that demonstrates the issue?

[jeffv]

JPoz,

Thanks for response.  How to I investigate it further with someone to see if a defect?

Jeff

[MarqueeCrew]

I agree that this looks like a defect (support@alteryx.com) and you can send them a packaged version of the workflow.  My container suggestion may help if there are multiple threads of processes.  Disable of tools that don't lead to the tools that you're adding might help you.  If you are accessing data from a database source, that too might contribute to the slowdown.

Cheers,

Mark

[danilang · Accepted solution]

Hi @jeffv 

As I understand it, Alteryx keeps the underlying metadata consistent as you're changing and debugging the workflow.  That's how a new field that you create in a formula is cascaded down the tool chain and is displayed in a Join tool even though there may be 50 tools between them.   In the traditional coding world, it's equivalent to your development environment recompiling your app every time you change a line of source code, just so it can tell you if you've just introduced error in a module that is dependent on the one you've just changed.  This process obviously takes time.  

There is a way to disable this though.  In the Advanced tab of the user settings, check the Disable Auto Configure option.  As the help file states " Select this option to manually configure tools in a workflow. With this option selected, press the F5 key to manually refresh tool configurations. This option may optimize performance."

Dan

[MikeD]

Are there any new solutions to try for this issue?  I have a large workflow (>2000 tools) and I've done everything in the solution and more and it does not change the wait time after a tool is altered in some way. It takes about 20 seconds from clicking to finish in a tool until the time the screen frees up again. It frequently goes into "not responding" mode when connecting tools. 

Here are the things I have tried:  1)  More than 2/3rds of my workflow is sectioned off in disabled containers, 2) AlteryxGUI.exe is set to the highest priority on the operating system, 3) 32GB RAM and plenty of open HD space, 4) Version 2019.2.7, 5) AutoConfigure disabled, 6) autosave interval increased, 7) display XML in properties is off, 😎 Display asset management is off, 9) dedicated sort join memory is about 1/3rd of physical RAM, 10) over 300 megabits/second internet connection, 11) Optimized the workflow as much as feasible, dropping variables, eliminating spatial variables, etc.

[jpoz]

@MikeD Would it be possible for you to send me your workflow? We have made some performance improvements for the upcoming 19.3 release and I would love to look into your workflow and see if there are other improvements we could make.