Issues faced when using Alteryx AMP engine
- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark
- Subscribe
- Mute
- Printer Friendly Page
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Notify Moderator
Below are issues which we faced when working with AMP engine feature. Can anyone help us to resolve the below issue and explain more about AMP feature.
- Delimiter (\0):- We cannot use AMP engine in cases where there are spills or splits in data and single row gets split to multiple rows. In such cases we import files without splitting on delimiters and later on combine rows to create final table. AMP data is processed in parallel packets which are comprised of rows chosen randomly from data instead of grouping rows sequentially. This leads to distortion of data.
- File Format:- AMP reads CSV and YXDB files considerable faster whereas for other data types files is almost the same.
- Record Limit:- There is no provision of test run by limiting records at initial run mostly to test workflow. As AMP overrides the limit and reads whole file.
- Data Field Length:- We cannot limit the data length as AMP allocates good memory to each column so there is no room for blockage or truncation of data. Might be a challenge incase it is being used as input to database resulting in more space consumption.
- Multiple Joins:- We have observed that incases where we had multiple joins in the huge data sets then it created a deadlock which resulted in no output resulting in loss of 3-4 hours. This might be due to parallel sharing of resources whereas in normal case it would happen sequentially
- Iterative Macros:- It results in unnecessary or even wrong output if iteration is too high. Errors are ignored and processing continues. We don’t have experience on this however its based on my experience from Alteryx Community.
- Excel Output : Error while writing in multiple sheets of same excel file as AMP attempts to write multiple outputs simultaneously.
- Labels:
- AMP Engine
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Notify Moderator
First, I want to thank you for using the AMP Engine and for providing feedback.
I encourage you to continue to report any use case issues that you find with running workflows with AMP Engine enabled. We worked hard to identify differences from the original Engine as well as provide guidance on how to better optimize workflows to run with AMP. Some of these differences have already been addressed in our documentation.
2. File Formats: AMP supports yxdb, csv, xlsx and sqlLite so those formats should be faster, all others must fall back to the original engine.
3. Record Limit: We have a feature on the roadmap to enable Record Limit configuration with AMP.
4. AMP attempts to accommodate for future processing of data.
5. Deadlocks have been addressed with 21.1, 21.2 and there will be additional edge-case fixes in 21.3 that have been identified with the help of ACEs.
6. Iterative Macros - differences are addressed in the "Main Differences" documentation.
7. Excel Output - this can be addressed by placing a Block Until Done tool in the branch with the lowest numbered Input tool id.
I also wanted to provide some helpful links to available documentation about AMP Engine:
AMP Engine webinar (32 minutes): https://community.alteryx.com/t5/Analytics/AMP-Engine-Discover-the-Untapped-Performance-of-Your-Anal...
https://help.alteryx.com/current/designer/alteryx-amp-engine
https://help.alteryx.com/current/designer/Alteryx-Engine-and-AMP-Main-Differences
https://help.alteryx.com/current/designer/AMP-Memory-Use
https://help.alteryx.com/current/designer/tool-use-amp
https://community.alteryx.com/t5/Engine-Works/AMPlify-your-Workflows/ba-p/617590
AlterEverything Podcast: https://community.alteryx.com/t5/Alter-Everything-Podcast/66-The-Alteryx-AMP-Engine-Explained/ba-p/5...
Server:
https://help.alteryx.com/current/server/controller#configuration
https://help.alteryx.com/current/server/worker
https://help.alteryx.com/current/server/engine
Sr. Product Manager, cloud App Builder
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Notify Moderator
I asked around, and we're not sure what you mean exactly by the first item in your list:
- Delimiter (\0):- We cannot use AMP engine in cases where there are spills or splits in data and single row gets split to multiple rows. In such cases we import files without splitting on delimiters and later on combine rows to create final table. AMP data is processed in parallel packets which are comprised of rows chosen randomly from data instead of grouping rows sequentially. This leads to distortion of data.
If you could supply a short workflow that demonstrates the issue (as a yxzp so that the data is included), that would allow me to reproduce it and see the issue.
Sr. Product Manager, cloud App Builder
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Notify Moderator
I've faced 2 more issues when using AMP.
- Data Cleansing tool: the tool's output sorts the data in an inexplicable way, almost randomly. I isolated the issue to the "Remove Null rows" option, for some reason when ticking this option the data sorting order is not the same as it originally is. However when disabling the use of AMP engine, the issue is solved.
- When running a workflow with a Batch Macro, the workflow errors on the macro showing the message Record #1: Tool #79: Parse Error at char(4): Unknown variable "Variable", I've checked this "Variable" field and everything appears to be alright with its data, expressions, field names, etc., in fact when disabling the use of AMP engine the workflows runs correctly.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Notify Moderator
For the Data Cleansing tool - we have documented that difference: https://help.alteryx.com/20221/designer/alteryx-engine-and-amp-main-differences
and the order difference can be easily solved by using the "Engine Compatibility Mode" to maintain the record order.
For the Batch Macro issue - please go through Support and get a defect created, with a workflow to reproduce the issue. Then it can be escalated to my team to look into. We do have a few issues related to Batch Macros, some are fixed for the next Release - so if there are additional issues we need to understand what exactly they are in order to address them.
Sr. Product Manager, cloud App Builder
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Notify Moderator
@TonyaS, just wanted to let you know that having AMP Engine on caused my Dynamic Input with a csv file with tab (\t) as a delimiter to go haywire.
Instead of creating a NULL cell it just filled the cell with the next bit of data it found.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Notify Moderator
Is there a chance you have quoted fields in your CSV?
There are 2 optional additional AMP specific configurations for CSV files:
Sr. Product Manager, cloud App Builder
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Notify Moderator
@TonyaS we are testing version 2021.4.2.35356 with the AMP engine on. Two issues are below, will be reported to support today.
1) With the AMP engine on, my workflow does not read any data from two Input Data tools that read YXDB files. With AMP off, the data is read in as expected. After I significantly reduce the tools in my workflow, to simplify it before sending it to support, I can find a few ways to get the YXDB files read in.... like deleting unrelated containers, or changing other unrelated input tools to read a Text Input tool instead of a text file.
But in my full workflow, those same changes will not force a read of the YXDB file, so I can't test the AMP engine with my full workflow.
2) With the AMP engine on, when I open the workflow I get the two errors in the screenshots below. I also have another version of the same workflow that does not give the memory error, but still gives the
Error: Designer x64: The Designer x64 reported: InboundNamedPipe GetOverlappedResult: The pipe has been ended.
Notes:
Our issue with "InboundNamedPipe" may have been fixed under defect TDSL-4131. We'll need to test in version 2022.1.
But in the 2022.1 release notes I don't see any defect mentioned for reading YXDB files.
Chris
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Notify Moderator
Hi @ChrisTX
There are 2 formats of YXDB for original Engine and AMP.
If you are trying to read a YXDB that was written with the original Engine using AMP it may be much slower.
The 2 formats of YXDB are documented here: https://help.alteryx.com/current/designer/Alteryx-Engine-and-AMP-Main-Differences
Here is more documentation on AMP:
Help Documentation
https://help.alteryx.com/current/designer/alteryx-amp-engine
https://help.alteryx.com/current/designer/Alteryx-Engine-and-AMP-Main-Differences
https://help.alteryx.com/current/designer/AMP-Memory-Use
https://help.alteryx.com/current/designer/tool-use-amp
https://help.alteryx.com/20221/designer/engine-compatibility-mode
https://help.alteryx.com/20221/designer/performance-profiling-amp-engine
https://help.alteryx.com/20221/server/amp-engine-best-practices
•AMP Engine Webinar (32 minutes)
•AlterEverything Podcast
•Community Blog posts
https://community.alteryx.com/t5/Engine-Works/AMPlify-your-Workflows/ba-p/617590
Sr. Product Manager, cloud App Builder
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Notify Moderator
@TonyaS thanks
Using version 2021.4.2.35356 with the AMP engine on, I created a small workflow to read in my existing YXDB file (created with our old version 2020.4 with AMP off).
I wrote out a new YXDB file.
Using version 2021.4.2.35356 with the AMP engine on: in a brand new workflow, data is read in from the new YXDB file as expected.
But my existing workflow, created under our old version 2020.4 with AMP off, will not read either the original or the new YXDB file. Even dragging in the YXDB file to create a new Input Data tool doesn't work. I cannot find any way to get my large workflow to read data from the YXDB file.
Chris
![](/skins/images/7C0E562E11459B4559A865ABC88017AF/responsive_peak/images/icon_anonymous_message.png)