Ability to execute tools in parallel within the same workflow

[sandeep_chayanam]

Tools within a workflow needs to be able to run in parallel whereever applicable.

For example: Extracting 10 million rows from one source, 12 million rows from a different source to perform blending.

currently the order of execution is the order in which tools are dragged into the canvas. Hence Source1 first, Source2 second and then the JOIN.

Here Source1 & Source2 are completely independent, hence can be run in parallel. Thus saving the workflow execution time.

Execution time is quite crucial when you have tight data loading window.

Hopefully alteryx considers this in the next release!

[NicoleJ]

Cleaning up some Idea statuses - this has been implemented with AMP (Alteryx Multi-Threaded Processing) engine since 2020.2! For more information on AMP, see the links below 😁

AMP Engine webinar (32 minutes): https://community.alteryx.com/t5/Analytics/AMP-Engine-Discover-the-Untapped-Performance-of-Your-Anal...

https://help.alteryx.com/current/designer/alteryx-amp-engine

https://help.alteryx.com/current/designer/Alteryx-Engine-and-AMP-Main-Differences

https://help.alteryx.com/current/designer/AMP-Memory-Use

https://help.alteryx.com/current/designer/tool-use-amp

https://community.alteryx.com/t5/Analytics/Accelerate-Your-Analytic-Processes-with-the-New-AMP-Engin...

https://community.alteryx.com/t5/Engine-Works/AMPlify-your-Workflows/ba-p/617590

https://community.alteryx.com/t5/Alteryx-Designer-Discussions/Best-practice-with-the-new-Multi-threa...

AlterEverything Podcast: https://community.alteryx.com/t5/Alter-Everything-Podcast/66-The-Alteryx-AMP-Engine-Explained/ba-p/5...

https://community.alteryx.com/t5/Engine-Works/AMP-Engine-Technical-Deep-Dive-Part-1-Why-AMP/ba-p/570...  

https://community.alteryx.com/t5/Engine-Works/AMP-Engine-Technical-Deep-Dive-Part-2-Key-concepts-of-... 

Using AMP with Server:

https://help.alteryx.com/current/server/controller#configuration

https://help.alteryx.com/current/server/worker

https://help.alteryx.com/current/server/engine

[hroderick-thr]

Hold on there pardner!

AMP give a single tool parallel process, but it doesn't multiple tools ability to run simultaneous.

This is for tools

[NicoleJ]

@hroderick-thr - On the contrary! AMP absolutely runs tools in parallel - check out the explanatory graphic on this page: https://community.alteryx.com/t5/Engine-Works/AMP-Engine-Technical-Deep-Dive-Part-2-Key-concepts-of-...

You'll still see the results window only output the completion of a tool at a time (will just save that message until all records have passed through that tool), but tools are indeed able to execute in parallel when using AMP. 😊 NOTE: There may also be some scenarios where a tool has not been converted to AMP and will still use the E1 engine, so it's possible you'll also still see that single-tool behavior on specific tools that haven't been converted to AMP. You can get a list of converted/AMP-enabled tools here: https://help.alteryx.com/20223/designer/tool-use-amp 

Thanks for keeping us accurate!

cc: @TonyaS