Belgium

Memory usage

JoBa
7 - Meteor

Hey guys

 

I find myself wondering a bit on how Alteryx uses the memory.

The documentation I find online is not really helping me; in case you have some tips on the below situation and/or material that I can read to better understand, then any of that is welcome!

 

Thanks in advance

 

Joachim

 

The workflow gives me the following message:

 

Capture1.PNG

 

But the machine has "plenty" of memory free - allright, it's only a 4 GB machine, but why is Alteryx not using it ?!

 

capture2.PNG

9 REPLIES 9
JefBus
7 - Meteor

I get these a lot too!

I'm setting up a little experiment as i now have

- a  8GB intel i5 laptop and a 16GB intel i7 desktop side by side

But I am unclear on what to use as a benchmark as most of my flows run in 3 seconds or less,

 

Maybe could use  my #weeklychallenge89 data set, but  for now unclear what to look for.

 

Interested to see where this topic goes!

KoCo
7 - Meteor

Hi,

 

The memory on your machine has a big influence on your Alteryx performance, so first of all, if you're going to treat a larger set of data, or doing a lot of in memory sorting, working a lot with Alteryx, consider putting in some extra memory in your machine. 16GB is nice, 32GB better - things just run a whole lot faster then for a very small price (if IT agrees).

 

I think the default settings are to take 25% of what is available - just not to lock up all your other applications.

 

The rest if it overruns it writes out to your Alteryx Temp drive (which can run out the space on your C drive at times - in which case set it to elsewhere). So when there is too much ... it swaps in and out. - you can see it somewhere under your local app ... (Can be changed in system settings I think) Then on workflow level you can override this Global Level - Under Runtime on the left of your canvas - Set to use a specific amount (but if you go there, it allocates this block of memory by default all the time during that workflow)...and Alteryx tends to mention, don't go there unless you know what you're doing ;) Could give it a try allocating there.

 

Could it be that you have your specific amount on the workflow or in the global default set too high for what's left? - Was it perhaps for this flow set on a higher grade machine and your now running it on a lower spec. Then you might want to try : Set it back to Global Default perhaps?

 

... a few thoughts only, I'm not a memory expert.

 

 

KoCo
7 - Meteor

There's a few other posts : Like https://community.alteryx.com/t5/Alteryx-Knowledge-Base/What-is-the-Sort-Join-Memory-Setting/ta-p/13..., gives some advice, but you might already have read that...

JoBa
7 - Meteor

Yes indeed I've seen this (and other comparable) posts.

 

What I don't understand is that my machine have > 1GB free; So when I set the sort/join memory to 700 MB (I've been trying various combinations), why is Alteryx telling me that this would exceed the available physical memory ? There's 1 GB to take which is not being used, resulting in a lot of IO waste of that.

 

I'll have a difficult time requesting ore RAM memory for the machine when theres is consistently 1GB (=25%) free !

 

 

 

KoCo
7 - Meteor
https://help.alteryx.com/9.5/Memory_Utilization.htm

Is a pretty good one...

Alteryx 32 or 64 bit? 32 uses max 2GB in any case.

Beside that most go towards setting it low. My interpretation is that eg the 1Gb you reserve for the sorts and joins comes on top of the other tools. And the sum of the two is max x pct of the total? Is that what it is. Ie if you want it to run faster in general, take less dedicated. So the reverting would be something like : you dedicated x gb, process requires y. Y plus x is bigger then 50% of available so you scale down... just a guess.

Koco
KoCo
7 - Meteor
By the way ... minimum spec advice is 8 gb for alteryx. https://www.alteryx.com/techspecs
Hakimipous
10 - Fireball

https://superuser.com/questions/44003/how-do-i-give-a-program-high-priority-for-memory-and-processor...

 

You could check that but honnetsly 4GB of RAM really is limited. If you are going to run predictives test I'm not even sure it will be able to run,as it is really ressources consuming.

 

You should consider an upgrade and get at least 8 GB of RAM to be confortable 

PatrickVandeputte
7 - Meteor

Hey all,

 

Not technical but from experience. We started with standard Portables with 4Gb. Being not allowed to plug in another card (100 eur?) to expand to 8 Gb, we got a second 8 Gb machine, and then finally also have 16Gb Virtual Machines. And that allowed us to do the test Jef was referring to on larger fuzzy matches... and each time we double, the run time halves:-) Maybe someone could do a predictive on what 32GB will give.

 

Working with a 4 GB machine has the advantage that you start optimising your flows to be able to go home in time. The data cleaninsing Icon... Very nice and very very slow. Probably better just to trim what you need and use formulas etc...

 

Cheers,

 

 

Hakimipous
10 - Fireball
Agreed!

Working with a limited capacity machine taught me how to cut my workflows into several pieces, so that when I had to change a parameter, I could test it without making it run the same long workflow again and again.

A real time gain x)