Today I was testing the AMP engine with a workflow and noticed that the file size of the YXDB output file was significantly larger. I ran the workflow a few times under each setting to ensure it wasn't a one-time occurrence.
The YXDB file has 16,654,982 records and 23 fields.
Not using the AMP engine, the file size is 240 MB.
Using the AMP engine, the file size is 789 MB.
Has anyone experienced this?
Solved! Go to Solution.
First, thank you for using the AMP Engine and for providing feedback.
Are you by chance using a Text Input tool in your workflow?
There are some differences in how AMP determines the data types on data read using a Text Input tool. Those are described in this documentation: https://help.alteryx.com/current/designer/Alteryx-Engine-and-AMP-Main-Differences
Text Input Tool and AutoField
AMP addresses a historic issue where the size of the field may not be large enough when processed by a downstream tool. You don't need to add Select tools to change data types when resulting data will exceed the length of the original data type. AMP creates the maximum size field for strings and integers so that subsequent operations will have the necessary room to hold larger downstream values.
Here are more helpful links to available documentation about AMP Engine:
https://help.alteryx.com/current/designer/alteryx-amp-engine
https://help.alteryx.com/current/designer/AMP-Memory-Use
https://help.alteryx.com/current/designer/tool-use-amp
https://community.alteryx.com/t5/Engine-Works/AMPlify-your-Workflows/ba-p/617590
AlterEverything Podcast: https://community.alteryx.com/t5/Alter-Everything-Podcast/66-The-Alteryx-AMP-Engine-Explained/ba-p/5...
Thanks @TonyaS ! I do have a couple of Text Input tools in the workflow and I am seeing them coming through as the maximum size now. I should be able to correct that and get the file size down.