Hi all,
We're a little concerned about how people publish to our prod scheduler and gallery, in-case one of the jobs is incredibly resource intensive, and takes down the server or starves all other jobs.
Many thanks
Sean
Solved! Go to Solution.
Sounds like product ideas to me. You might also be seeing indications that you are growing and might need to get another server instance.
Cheers,
Mark
:-) Hola Mark, happy Sunday!
Am I right that based on your response, Alteryx server doesn't yet have the capabity to do this kind of resource control?
I'm sorry to be asking such a basic question - I would have mocked it up myself on a server at home, but I can't figure out how to get a trial license for the server that I can use at home for learning?
I'll give more color for the two resource questions below - and possibly (based on your guidance) get these into the Ideas section:
Is there a way, within the server setup, to limit the priority of certain jobs so that they allow other jobs to go ahead of them?
This is like setting the priority on a windows thread to "Below Normal" so that more important jobs can go ahead. Useful if you have a job that you know is pretty intense (like an analytical job, or a big ETL or dimension conforming job) - you want to let the smaller and quicker jobs overtake you.
Is there a way to choke the total resource usage so that this automatically forces other jobs to wait or slow down until resources are released?
This is a similar issue - but even the small and quick jobs may all end up being scheduled by users at the same time (for example - users tend to schedule things at even multiples of the hour, so there's a cluster of jobs that all kick off at 08:00; 09:00; etc). Does Alteryx server have the ability to queue these up so that even though they are all scheduled to start at 08:00, they only start when there is spare capacity on a worker node?
Thanks Mark
Sean
Thank you Mark - have a good Sunday!
@MarqueeCrew...it seems like you're familiar with Control M jobs... I'm not too familiar with them...
1) It seems like (as an Admin) I don't have visibility to them since they're not scheduled thru the scheduler
2) The only way I see them is thru the engine log files
3) And I have a case where there's a sequence of workflows that run every 2 hours, but during the time one of our Control M jobs is running those workflows fail due to memory allocation issues. Even though, the log file for the Control M job says it was completed in 4 minutes or less.
Can you shed any light on this? Is it not a best practice to use Alteryx and Control M jobs together?
Any input/additional reading material would be greatly appreciated...
Thanks!