Massive txt file taking an hour or longer to read into Designer 2023.1

[dmpope]

Hi there,

I have a txt file that is used in a workflow that I built a while back that can be anywhere from 5-10 million rows long. When I first built the flow (on the last 2022 version), it took about 4-10 minutes to run the entire workflow. Now on 2023.1 it takes about an hour or more to read the file in and get to the point where I have a batch macro. Is there any faster ways to bring in a massive file? Currently using an Input Data tool and reading it in as a flat file to parse out from there because the delimiters are inconsistent. The only thing that has changed was the version of Alteryx and that I'm using control containers. 

For context, this workflow is basically extracting a set of reports out of our system's master report file. We don't get these reports individually and the only other format we get them in is on a disc, so this massive file is our only way of retrieving this data without having to go day-by-day and copy & paste from the system itself. And since this is customer activity data, there is no specific line to start pulling in this file from.

Thank you!

[apathetichell]

turn off amp. this is a ram issue - and the ram isn't optimized for single large jobs.

[caltang]

An alternative would also to look at upgrading your setup with RAM modules or even a new PC? Increase the specs if you can... goes a long way with Alteryx.

[apathetichell]

@caltang - i'd posit that AMP is optimized for multi-threaded processes - not single threaded large memory processes. the difference in behavior between 2022.1 and 2023.1 makes me think it's not their machine.