YXDB File Size Using AMP Engine

bzuerlein

Today I was testing the AMP engine with a workflow and noticed that the file size of the YXDB output file was significantly larger. I ran the workflow a few times under each setting to ensure it wasn't a one-time occurrence.

The YXDB file has 16,654,982 records and 23 fields.

Not using the AMP engine, the file size is 240 MB.

Using the AMP engine, the file size is 789 MB.

Has anyone experienced this?

Tonya_Smith

@bzuerlein 

First, thank you for using the AMP Engine and for providing feedback. 

Are you by chance using a Text Input tool in your workflow? 

There are some differences in how AMP determines the data types on data read using a Text Input tool. Those are described in this documentation: https://help.alteryx.com/current/designer/Alteryx-Engine-and-AMP-Main-Differences

Text Input Tool and AutoField

AMP addresses a historic issue where the size of the field may not be large enough when processed by a downstream tool. You don't need to add Select tools to change data types when resulting data will exceed the length of the original data type. AMP  creates the maximum size field for strings and integers so that subsequent operations will have the necessary room to hold larger downstream values.

Here are more helpful links to available documentation about AMP Engine:

https://help.alteryx.com/current/designer/alteryx-amp-engine

https://help.alteryx.com/current/designer/AMP-Memory-Use

https://help.alteryx.com/current/designer/tool-use-amp

https://community.alteryx.com/t5/Analytics/Accelerate-Your-Analytic-Processes-with-the-New-AMP-Engine/ba-p/574809

https://community.alteryx.com/t5/Engine-Works/AMPlify-your-Workflows/ba-p/617590

https://community.alteryx.com/t5/Alteryx-Designer-Discussions/Best-practice-with-the-new-Multi-threaded-processing-AMP/m-p/603329#M134951

AlterEverything Podcast: https://community.alteryx.com/t5/Alter-Everything-Podcast/66-The-Alteryx-AMP-Engine-Explained/ba-p/599058

https://community.alteryx.com/t5/Engine-Works/AMP-Engine-Technical-Deep-Dive-Part-1-Why-AMP/ba-p/570964  

https://community.alteryx.com/t5/Engine-Works/AMP-Engine-Technical-Deep-Dive-Part-2-Key-concepts-of-the-AMP/ba-p/611500