Advice Needed: Best Practices for Optimizing Large Dataset Processing in Alteryx

[BusRentalDubai]

Hello Alteryx Community,

I’m currently working on a project for Bus Rental Dubai, where we’re analyzing large datasets to forecast vehicle demand and optimize route efficiency. However, I’m facing challenges with Alteryx’s performance when processing these large datasets—it’s causing workflows to slow down significantly.

Does anyone have advice on optimizing Alteryx workflows for large datasets? I’m particularly interested in any tips for managing memory usage, structuring workflows, or using specific tools in Alteryx that help with processing efficiency. If you’ve worked with similar transportation or logistics data, I’d also love to hear about any custom solutions you’ve implemented to manage or analyze large volumes of data effectively.

Thanks in advance for any insights or strategies you can share! Your experience would be incredibly helpful for refining the data analytics approach at Bus Rental Dubai.

[DanM]

@BusRentalDubai 

Alteryx is a memory intensive tool, so obviously the more memory available the better. However, there are tools that can enhance the process as well as hinder the process.

Tools that can help are the Block Until Done tool. This tool is great for when you are pulling data into Alteryx typically used after an Input tool or a large macro is used. The only caveat with this tool is that it does not block other Input tools from executing. There is also the Throttle tool which allows you to manage how many records are being pushed into Alteryx at a time. The other piece is using the cache option. This is great for building as you can right click on a tool and select "cache" which will then cache the data to a point. The other piece is using the tool containers to only execute processes when needed during the building process. You can also use tool containers to manage what gets executed when in your workflow.

Tools that can use up your memory are the orange tool category that include Summarize, Transpose, etc. If you are using these tools the Block Until Done can also assist after these tools.

Lastly, make sure your Amp engine is being used for all your workflows. You select the canvas and look under Runtime and at the bottom the Amp engine should be selected. You should also make sure your machine is meeting the minimum requirements to run Alteryx.