This site uses different types of cookies, including analytics and functional cookies (its own and from other sites). To change your cookie settings or find out more, click here. If you continue browsing our website, you accept these cookies.
Below are issues which we faced when working with AMP engine feature. Can anyone help us to resolve the below issue and explain more about AMP feature.
Delimiter (\0):- We cannot use AMP engine in cases where there are spills or splits in data and single row gets split to multiple rows. In such cases we import files without splitting on delimiters and later on combine rows to create final table. AMP data is processed in parallel packets which are comprised of rows chosen randomly from data instead of grouping rows sequentially. This leads to distortion of data.
File Format:- AMP reads CSV and YXDB files considerable faster whereas for other data types files is almost the same.
Record Limit:- There is no provision of test run by limiting records at initial run mostly to test workflow. As AMP overrides the limit and reads whole file.
Data Field Length:- We cannot limit the data length as AMP allocates good memory to each column so there is no room for blockage or truncation of data. Might be a challenge incase it is being used as input to database resulting in more space consumption.
Multiple Joins:- We have observed that incases where we had multiple joins in the huge data sets then it created a deadlock which resulted in no output resulting in loss of 3-4 hours. This might be due to parallel sharing of resources whereas in normal case it would happen sequentially
Iterative Macros:- It results in unnecessary or even wrong output if iteration is too high. Errors are ignored and processing continues. We don’t have experience on this however its based on my experience from Alteryx Community.
Excel Output : Error while writing in multiple sheets of same excel file as AMP attempts to write multiple outputs simultaneously.
First, I want to thank you for using the AMP Engine and for providing feedback.
I encourage you to continue to report any use case issues that you find with running workflows with AMP Engine enabled. We worked hard to identify differences from the original Engine as well as provide guidance on how to better optimize workflows to run with AMP. Some of these differences have already been addressed in our documentation.
2. File Formats: AMP supports yxdb, csv, xlsx and sqlLite so those formats should be faster, all others must fall back to the original engine.
3. Record Limit: We have a feature on the roadmap to enable Record Limit configuration with AMP.
4. AMP attempts to accommodate for future processing of data.
5. Deadlocks have been addressed with 21.1, 21.2 and there will be additional edge-case fixes in 21.3 that have been identified with the help of ACEs.
6. Iterative Macros - differences are addressed in the "Main Differences" documentation.
7. Excel Output - this can be addressed by placing a Block Until Done tool in the branch with the lowest numbered Input tool id.
I also wanted to provide some helpful links to available documentation about AMP Engine:
I asked around, and we're not sure what you mean exactly by the first item in your list:
Delimiter (\0):-We cannot use AMP engine in cases where there are spills or splits in data and single row gets split to multiple rows. In such cases we import files without splitting on delimiters and later on combine rows to create final table. AMP data is processed in parallel packets which are comprised of rows chosen randomly from data instead of grouping rows sequentially. This leads to distortion of data.
If you could supply a short workflow that demonstrates the issue (as a yxzp so that the data is included), that would allow me to reproduce it and see the issue.
Tonya Smith Sr. Technical Product Manager, Alteryx Engines
Data Cleansing tool: the tool's output sorts the data in an inexplicable way, almost randomly. I isolated the issue to the "Remove Null rows" option, for some reason when ticking this option the data sorting order is not the same as it originally is. However when disabling the use of AMP engine, the issue is solved.
When running a workflow with a Batch Macro, the workflow errors on the macro showing the message Record #1: Tool #79: Parse Error at char(4): Unknown variable "Variable", I've checked this "Variable" field and everything appears to be alright with its data, expressions, field names, etc., in fact when disabling the use of AMP engine the workflows runs correctly.
For the Batch Macro issue - please go through Support and get a defect created, with a workflow to reproduce the issue. Then it can be escalated to my team to look into. We do have a few issues related to Batch Macros, some are fixed for the next Release - so if there are additional issues we need to understand what exactly they are in order to address them.
Tonya Smith Sr. Technical Product Manager, Alteryx Engines
@TonyaS we are testing version 2021.4.2.35356 with the AMP engine on. Two issues are below, will be reported to support today.
1) With the AMP engine on, my workflow does not read any data from two Input Data tools that read YXDB files. With AMP off, the data is read in as expected. After I significantly reduce the tools in my workflow, to simplify it before sending it to support, I can find a few ways to get the YXDB files read in.... like deleting unrelated containers, or changing other unrelated input tools to read a Text Input tool instead of a text file.
But in my full workflow, those same changes will not force a read of the YXDB file, so I can't test the AMP engine with my full workflow.
2) With the AMP engine on, when I open the workflow I get the two errors in the screenshots below. I also have another version of the same workflow that does not give the memory error, but still gives the
Error: Designer x64: The Designer x64 reported: InboundNamedPipe GetOverlappedResult: The pipe has been ended.
Our issue with "InboundNamedPipe" may have been fixed under defect TDSL-4131. We'll need to test in version 2022.1.
But in the 2022.1 release notes I don't see any defect mentioned for reading YXDB files.
Using version 2021.4.2.35356 with the AMP engine on, I created a small workflow to read in my existing YXDB file (created with our old version 2020.4 with AMP off).
I wrote out a new YXDB file.
Using version 2021.4.2.35356 with the AMP engine on: in a brand new workflow, data is read in from the new YXDB file as expected.
But my existing workflow, created under our old version 2020.4 with AMP off, will not read either the original or the new YXDB file. Even dragging in the YXDB file to create a new Input Data tool doesn't work. I cannot find any way to get my large workflow to read data from the YXDB file.