This site uses different types of cookies, including analytics and functional cookies (its own and from other sites). To change your cookie settings or find out more, click here. If you continue browsing our website, you accept these cookies.
We are quite new to Alteryx with our company, using it for a year for simple workflows.
Recently we've created a lot of process automatization with Alteryx - some of them became a quite huge and resource intensive workflow, especially one that creates our daily reporting from sales data. (we learned our lesson and next time we'll split into smaller parts).
However the question is:
We set up a workstation computer (128 gigs of ddr4 ram, 512 GB PCie SSD, Intel Xenon processor) and scheduled our workflows for a daily run. We noticed two things:
- If I run the workflow on this machine manually (from designer) it runs 65-70 minutes
- If I schedule the workflow via the scheduler it runs 90-95 minutes
In both cases I run a performance log in windows and the MAX memory usage is ~30 gb of ram and MAX CPU usage is 35%, while SSD usage is minimal, nearly never reaches 10%.
There are no parallel workflows running and the machine is a corporate windows 10 with no other software running.
Anyone has any idea:
- Why the scheduled workflow runs way slower than the same from designer manual run?
- How could we "make" Alteryx use more RAM / CPU to run faster?
As an additional information our workflow mainly operates with joins and formulas - what I've read are single thread processes in Alteryx (we use 2018.3). Does it have anything to do with it?
32GB should be the value to go with based on the recommendations in the article.
(64 bit machine): Set this in the system settings to half your physical memory divided by the number of simultaneous processes you expect to run. If you have 8 GB of RAM and run 2 processes at a time, your Sort/Join memory should be set to 2GB. You might set it lower if you expect to be doing a lot of memory intensive stuff on the machine besides Alteryx.
As for the Designer vs scheduler speed, is there a download tool in your workflow? I have seen on very rare occasions that the service running under a system account rather than a user account could subject a download tool to different firewall rules and increase processing time.
It might also be useful to check the box for "enable performance profiling" to see what tools in your workflow are taking the most time. This will show each tool's % of total time and time spent processing down in the results window after workflow run.
I read this article hence I did the settings according to this. However even if I raise the sort / join memory limit to 64 gigs the system (incl. Windows processes) still only uses ~ 30 of it. It has absolutely no effect.
I also tried to adjust Alteryx application priority level to "high" based on this article:
The workflow indeed feeds it's input data from a shared drive. We have already checked the performance profiling that indicated the time difference is coming from general running time of nearly all tools. All the joins / formulas just run a little bit longer, so it's definitely not the network load time.