Switching to AMP Engine with In-DB Tools: Is It Worth It?

Question

Hello,

90% of our workflows primarily use In-DB tools, while the remaining 10% of in-memory workflows handle low-volume data.

When considering the effort to switch to AMP:

- Enormous machine resources (CPU/RAM) are recommended to run a few parallel processes.
- Increasing the number of CPUs translates to higher licensing costs.
- There's uncertainty about whether processes from the original engine and AMP yield the same results, necessitating extensive verification work.

Given all these factors, in our context, is there any advantage to transitioning to the AMP engine?

sraynal · Answer

Thanks for your responses.

Considering our context (Resources, Licenses, low-volume in-memory workflow, result consistency), it's probably not worth using AMP.

Another concern is the sustainability of the original engine.

I'm worried it might be deprecated and no longer supported at some point.

Hammad_Rashid · Answer

Enormous Machine Resources:

Consideration: If your workflows primarily use In-DB tools and handle low-volume data, it's important to assess whether the enormous machine resources required by the AMP engine are justified for your use case. If your datasets are relatively small and your workflows are not highly complex, the benefits of parallel processing may not outweigh the resource investment.

Increasing CPUs and Licensing Costs:

Consideration: If increasing the number of CPUs translates to higher licensing costs, it becomes a crucial factor in your decision. Evaluate whether the potential performance gains from the AMP engine justify the additional licensing expenses, especially considering that the majority of your workflows use In-DB tools.

Uncertainty about Results Consistency:

Consideration: The uncertainty about whether processes from the original engine and AMP yield the same results is a valid concern. Given that 90% of your workflows use In-DB tools, extensive verification work may be required to ensure consistency. Consider the time and effort required for verification in your decision-making process.

Considering these factors:

* Advantage of Transitioning to AMP:

* If your organization frequently deals with large datasets and complex workflows that could benefit from parallel processing.
* If you have the resources for additional CPUs and are willing to invest in verification to ensure consistent results.

* Consideration Against Transitioning:

* If the majority of your workflows are In-DB and handle low-volume data, the performance gains from parallel processing may not be as significant.
* If the cost of additional resources and licensing outweighs the perceived benefits.

fmvizcaino · Answer

Hi @sraynal ,

You have some benchmarks here and there about the processes that perform better with AMP. Still, here you can see a new comparison between the E1 and AMP engines for data prep and predictive analytics based on the number of processing threads (logical cores).

https://help.alteryx.com/current/en/server/best-practices/amp-engine-best-practices.html#idm45579902914720

The server license is based on the number of physical cores, so depending on your provider, we could be talking about 8 logical cores, which is the minimum but more than enough depending on the workflows you have there (not only the number of workflows but runtime and types of tools as well). Of course, there is still the RAM cost and there is nothing we can do there.

https://knowledge.alteryx.com/index/s/article/How-Alteryx-defines-cores-for-licensing-our-products-1583461081373

To your question, if you will always have only these low-volume data workflows, I don`t think you need to go through this change. And yes, for a good performance and to not create bottlenecks, better to have 16 logical cores (8-core license) and 2 simultaneous workflows at least.

Best,

Fernando Vizcaino