Alteryx Designer Discussions

Find answers, ask questions, and share expertise about Alteryx Designer.

Best practice with the new Multi-threaded processing (AMP)

JamesFo
8 - Asteroid

Hi all,

 

I've been excitedly toggling on and off the AMP engine on some of my workflows (my PC has a 16 core Xeon Gold) and was really pleased to see that some of the workflows that were previously only using one core, have jumped up to processing across all 16 (at an overclocked 4.0Ghz).

 

JamesFo_0-1594727328724.jpeg

 

It appears to be a game changer, and I've seen a reduction in some file times of 10x, which I'm really pleased with.

 

With that said, I'm keen to make the most of this, and wondering if there is any documentation / blogs out there that describe best practice. Have you found anything works particularly well?

 

So I've far I've come across:

  • Documentation (here)
  • Blog on the technical side of things (part 1)

 

Should I be building my workflows to look a certain way to make the most of it? Are some tools now preferable to others (when there is two ways of doing something)?

 

Basically a shout out for any good sources of information for someone who is keen to take advantage, but not (yet?) skilled enough to be writing the information!

 

Thanks,

 

James

5 REPLIES 5
TonyaS
Alteryx
Alteryx

@JamesFo 

Wow thank you so much for jumping right in and using AMP as soon as it was delivered in the 2020.2 Release and for the early feedback on the new AMP Engine. 

 

I encourage you to continue to report good feedback as well as any specific use case issues or optimization opportunities that you may encounter with running workflows with AMP Engine enabled. We worked hard to identify differences from the original Engine as well as provide guidance on how to better optimize workflows to run with AMP. 

 

Below I will link our documentation and articles related to using the new AMP Engine. The help documentation will be updated as we make enhancements, convert more tools and learn more about AMP from users like you.

 

Marketing Blog announcing the AMP Engine: https://community.alteryx.com/t5/Analytics/Accelerate-Your-Analytic-Processes-with-the-New-AMP-Engin...

 

Help articles:

https://help.alteryx.com/current/designer/alteryx-amp-engine

https://help.alteryx.com/current/designer/Alteryx-Engine-and-AMP-Main-Differences

https://help.alteryx.com/current/designer/AMP-Memory-Use

A list of tools that have been fully or partially converted to AMP: https://help.alteryx.com/current/designer/tool-use-amp

 

Adam Riley's Blog Series:

  1. https://community.alteryx.com/t5/Engine-Works/AMP-Engine-Technical-Deep-Dive-Part-1-Why-AMP/ba-p/570...

I will add links to these as more get published.

 

2020.2 Release Notes:

 

 

Tonya Smith
Sr. Technical Product Manager, Alteryx Engines
JamesFo
8 - Asteroid

Thanks Tonya,

 

All helpful - I need to keep developing my understanding of it. For example, if you want to do a series of XML parses, is it better to put these in series, or all parallel & then join them together again at the end. Does this actually make a difference?

 

Maybe i'll just have to try and experiment - but it'd be useful if someone was able to do the thinking first 😄

 

James

warrencowan
8 - Asteroid

Great to see some validation and getting under the hood. thanks for sharing.

 

I'd be keen to know how deep the rabbit hole goes with mutli threading. I am about to upgrade a machine and am pondering going all out with an amd threadripper with 64 cores on it and 256 of RAM.

 

That will give it about 128 threads assuming there's enough ram, which is a very tanatlising prospect. But I am curious to know if the multi threading tops out before that, and if the extra oomph will be wasted,

 

Have you or anyone else plummed the depth on this.

JamesFo
8 - Asteroid

Hi @warrencowan ,

 

Sorry for the delay on this - for some reason didn't get a notification. I've found workflows using all my cores (Even overclocking - see picture) + maxing out my 64gb of RAM. Whether its doing it efficiently, I've got no idea!

 

James

warrencowan
8 - Asteroid

Thanks @JamesFo . I havent managed to max my ram yet, but that might be due to keeping the max memory setting set at 16gb for the workflow. 

 

I have seen my processor utilisation max out though so alteryx clearly will use it if it can.

 

I had a support call with alteryx recently to try and solve a bug in a worklflow using amp , and they suggested it should eat whatever resources you set aside for it, even way up to the levels of cores on a threadripper cpu.

Labels